======================================================================================================================== [START] 2026-10-16T18:34:40+00:00 Suite32 selective borrowing stage=development run=20261016T183440_329208 [CODE] /root/disk_3tb/earlywarning/30072026/suite32_selective_borrowing [ROOT] /root/disk_3tb/earlywarning/30072026 [PYTHON] /root/miniconda3/envs/Huy/bin/python3 [LOG] /root/disk_3tb/earlywarning/30072026/log/suite32_selective_borrowing.log [STAGE LOG] /root/disk_3tb/earlywarning/30072026/log/suite32_selective_borrowing_development.log [RUN LOG] /root/disk_3tb/earlywarning/30072026/log/suite32_selective_borrowing_runs/development_20261016T183440_329208.log [RESULTS] /root/disk_3tb/earlywarning/30072026/results/suite32_selective_borrowing [ARGS] --stage development --root /root/disk_3tb/earlywarning/30072026 --mask-audit [ENV] python=3.13.5 numpy=2.2.6 pandas=2.3.1 sklearn=1.7.1 root=/root/disk_3tb/earlywarning/30072026 cache=/root/disk_3tb/earlywarning/30072026/results/cross_faculty_residual_moe/cache structured=/root/disk_3tb/earlywarning/30072026/results/suite12_structured_cache folds=[1, 2, 3, 4, 5] targets=14 donors=14 seed=42 [SCIENTIFIC_CONTRACT] labels=grade10_lt5 prediction=semester_origin selection=historical_only evaluation=faculty_fold_macro_AP council=13_faculty_archived_metrics_only holdout=retrospective_not_fresh masks=cache_nan_is_not_true_observation_provenance borrow_mass=0.250000 [FOLD_START] fold=1 stage=development [DATA] fold=1 common_raw=136 course_features=17 common_total=153 faculties=14 history_rows=240672 target_rows=45800 target_failures=1083 history_last=6065 target_first=6066 [OBSERVATION_AUDIT] fold=1 faculty=khoa_cong_nghe_thong_tin raw_columns=275 common_raw_columns=136 native_raw_columns=139 history_rows=19090 history_failures=576 target_rows=5222 target_failures=189 target_prevalence=0.036193 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.660650 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich raw_columns=301 common_raw_columns=136 native_raw_columns=165 history_rows=33453 history_failures=915 target_rows=5672 target_failures=219 target_prevalence=0.038611 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.693662 history_latest_semester=6065 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_quoc_te_hoc raw_columns=254 common_raw_columns=136 native_raw_columns=118 history_rows=10439 history_failures=98 target_rows=2363 target_failures=31 target_prevalence=0.013119 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.623864 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_anh raw_columns=511 common_raw_columns=136 native_raw_columns=375 history_rows=47997 history_failures=821 target_rows=8164 target_failures=148 target_prevalence=0.018128 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.813756 history_latest_semester=6065 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_bo_dao_nha raw_columns=400 common_raw_columns=136 native_raw_columns=264 history_rows=4117 history_failures=123 target_rows=768 target_failures=30 target_prevalence=0.039062 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.751248 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_duc raw_columns=381 common_raw_columns=136 native_raw_columns=245 history_rows=10883 history_failures=172 target_rows=1900 target_failures=46 target_prevalence=0.024211 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.749647 history_latest_semester=6065 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_han_quoc raw_columns=333 common_raw_columns=136 native_raw_columns=197 history_rows=17102 history_failures=75 target_rows=2904 target_failures=20 target_prevalence=0.006887 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.686133 history_latest_semester=6065 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_italia raw_columns=420 common_raw_columns=136 native_raw_columns=284 history_rows=10650 history_failures=326 target_rows=2166 target_failures=111 target_prevalence=0.051247 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.745552 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_nga raw_columns=469 common_raw_columns=136 native_raw_columns=333 history_rows=6608 history_failures=270 target_rows=1588 target_failures=102 target_prevalence=0.064232 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.791949 history_latest_semester=6065 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_nhat raw_columns=362 common_raw_columns=136 native_raw_columns=226 history_rows=19043 history_failures=199 target_rows=2840 target_failures=17 target_prevalence=0.005986 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.742401 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_phap raw_columns=495 common_raw_columns=136 native_raw_columns=359 history_rows=15265 history_failures=178 target_rows=2606 target_failures=42 target_prevalence=0.016117 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.788273 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_tay_ban_nha raw_columns=396 common_raw_columns=136 native_raw_columns=260 history_rows=8025 history_failures=170 target_rows=1401 target_failures=63 target_prevalence=0.044968 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.747781 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_tieng_trung_quoc raw_columns=436 common_raw_columns=136 native_raw_columns=300 history_rows=32781 history_failures=185 target_rows=7393 target_failures=35 target_prevalence=0.004734 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.760859 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=1 faculty=khoa_viet_nam_hoc raw_columns=181 common_raw_columns=136 native_raw_columns=45 history_rows=5219 history_failures=337 target_rows=813 target_failures=30 target_prevalence=0.036900 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.504808 history_latest_semester=6064 target_first_semester=6066 availability_provenance=semester_proxy; source release timestamps not verified [SELECTION_ORIGINS] target=khoa_cong_nghe_thong_tin fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=1 origin=6060 ap=0.152427 rows=3115 failures=96 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.005928 ap=0.158355 base_ap=0.152427 donor_failures=94 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6060 delta_ap=0.016016 ap=0.168442 base_ap=0.152427 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6060 delta_ap=0.062389 ap=0.214816 base_ap=0.152427 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=-0.017442 ap=0.134985 base_ap=0.152427 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6060 delta_ap=0.040216 ap=0.192643 base_ap=0.152427 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6060 delta_ap=0.055143 ap=0.207570 base_ap=0.152427 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6060 delta_ap=0.025333 ap=0.177760 base_ap=0.152427 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6060 delta_ap=-0.090470 ap=0.061957 base_ap=0.152427 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6060 delta_ap=0.083790 ap=0.236216 base_ap=0.152427 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6060 delta_ap=0.041474 ap=0.193900 base_ap=0.152427 donor_failures=71 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.017578 ap=0.170004 base_ap=0.152427 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6060 delta_ap=0.061895 ap=0.214322 base_ap=0.152427 donor_failures=38 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6060 delta_ap=0.033825 ap=0.186251 base_ap=0.152427 donor_failures=121 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=1 origin=6061 ap=0.408067 rows=2815 failures=77 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.012046 ap=0.420113 base_ap=0.408067 donor_failures=77 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6061 delta_ap=0.011707 ap=0.419774 base_ap=0.408067 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6061 delta_ap=0.023101 ap=0.431168 base_ap=0.408067 donor_failures=52 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.039193 ap=0.447260 base_ap=0.408067 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6061 delta_ap=0.001375 ap=0.409442 base_ap=0.408067 donor_failures=77 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6061 delta_ap=0.021506 ap=0.429573 base_ap=0.408067 donor_failures=26 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6061 delta_ap=0.034381 ap=0.442448 base_ap=0.408067 donor_failures=161 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6061 delta_ap=0.022404 ap=0.430471 base_ap=0.408067 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6061 delta_ap=0.018659 ap=0.426726 base_ap=0.408067 donor_failures=62 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6061 delta_ap=0.006299 ap=0.414366 base_ap=0.408067 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.036742 ap=0.444809 base_ap=0.408067 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.011138 ap=0.419205 base_ap=0.408067 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6061 delta_ap=0.015468 ap=0.423535 base_ap=0.408067 donor_failures=180 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=1 origin=6063 ap=0.053064 rows=5277 failures=63 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.041514 ap=0.094578 base_ap=0.053064 donor_failures=95 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.000484 ap=0.052580 base_ap=0.053064 donor_failures=37 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6063 delta_ap=0.012684 ap=0.065748 base_ap=0.053064 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.000463 ap=0.053527 base_ap=0.053064 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6063 delta_ap=-0.007450 ap=0.045615 base_ap=0.053064 donor_failures=70 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6063 delta_ap=0.009693 ap=0.062758 base_ap=0.053064 donor_failures=13 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6063 delta_ap=0.014535 ap=0.067599 base_ap=0.053064 donor_failures=146 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6063 delta_ap=0.023053 ap=0.076118 base_ap=0.053064 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6063 delta_ap=0.021900 ap=0.074965 base_ap=0.053064 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6063 delta_ap=0.015457 ap=0.068521 base_ap=0.053064 donor_failures=50 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.005476 ap=0.058541 base_ap=0.053064 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.011897 ap=0.064962 base_ap=0.053064 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6063 delta_ap=0.004353 ap=0.057418 base_ap=0.053064 donor_failures=217 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=1 origin=6064 ap=0.290741 rows=3992 failures=116 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.029362 ap=0.261380 base_ap=0.290741 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.010987 ap=0.279754 base_ap=0.290741 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6064 delta_ap=0.015336 ap=0.306077 base_ap=0.290741 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.001419 ap=0.289322 base_ap=0.290741 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6064 delta_ap=-0.018414 ap=0.272327 base_ap=0.290741 donor_failures=50 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6064 delta_ap=0.016822 ap=0.307563 base_ap=0.290741 donor_failures=19 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6064 delta_ap=-0.031327 ap=0.259414 base_ap=0.290741 donor_failures=128 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6064 delta_ap=-0.001215 ap=0.289526 base_ap=0.290741 donor_failures=138 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6064 delta_ap=-0.026698 ap=0.264043 base_ap=0.290741 donor_failures=41 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6064 delta_ap=-0.030966 ap=0.259776 base_ap=0.290741 donor_failures=48 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.047175 ap=0.243566 base_ap=0.290741 donor_failures=79 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.026500 ap=0.264242 base_ap=0.290741 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.091764 ap=0.198978 base_ap=0.290741 donor_failures=269 [POLICY_REPLAY] fold=1 faculty=khoa_cong_nghe_thong_tin origin=6063 policy_sources=khoa_tieng_nhat,khoa_tieng_anh,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.053064 policy_ap=0.075094 delta_ap=0.022030 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_cong_nghe_thong_tin origin=6064 policy_sources=khoa_tieng_nhat,khoa_tieng_anh,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.290741 policy_ap=0.279860 delta_ap=-0.010882 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_cong_nghe_thong_tin selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.005574 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common rows=5222 failures=189 prevalence=0.036193 ap=0.234575 auc=0.809290 f2_at20=0.344253 brier=0.033141 ece10=0.026494 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=0.621602 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common in_sample_bce=0.049430 local_bce=0.049430 donor_bce=nan in_sample_local_ap=0.824765 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native rows=5222 failures=189 prevalence=0.036193 ap=0.259189 auc=0.821192 f2_at20=0.347029 brier=0.032553 ece10=0.024785 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=0.930301 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native in_sample_bce=0.047943 local_bce=0.047943 donor_bce=nan in_sample_local_ap=0.837533 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all rows=5222 failures=189 prevalence=0.036193 ap=0.258426 auc=0.807754 f2_at20=0.341477 brier=0.032363 ece10=0.023704 rho=0.250000 donor_rows=4095 effective_n=18789.264710 seconds=0.678884 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all in_sample_bce=0.047049 local_bce=0.053686 donor_bce=0.047639 in_sample_local_ap=0.793527 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.037743363630619815, 'khoa_quoc_te_hoc': 0.011777806861568178, 'khoa_tieng_anh': 0.05415263875224522, 'khoa_tieng_bo_dao_nha': 0.004645007265933154, 'khoa_tieng_duc': 0.012278750078977534, 'khoa_tieng_han_quoc': 0.019295339874177506, 'khoa_tieng_italia': 0.012015867714886589, 'khoa_tieng_nga': 0.0074554792356779885, 'khoa_tieng_nhat': 0.02148527407460895, 'khoa_tieng_phap': 0.017222743724670777, 'khoa_tieng_tay_ban_nha': 0.009054210179527218, 'khoa_tieng_trung_quoc': 0.03698517930156782, 'khoa_viet_nam_hoc': 0.005888339305539258} source_failures={'khoa_cong_nghe_thong_tin': 576, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 2, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 26} positive_ess=581.858837 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal rows=5222 failures=189 prevalence=0.036193 ap=0.279948 auc=0.813011 f2_at20=0.349806 brier=0.031772 ece10=0.022223 rho=0.250000 donor_rows=4095 effective_n=22357.258021 seconds=0.687505 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal in_sample_bce=0.051289 local_bce=0.055717 donor_bce=0.038006 in_sample_local_ap=0.778674 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_cong_nghe_thong_tin': 576, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 2, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 26} positive_ess=641.770941 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior rows=5222 failures=189 prevalence=0.036193 ap=0.234575 auc=0.809290 f2_at20=0.344253 brier=0.033141 ece10=0.026494 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=0.619682 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior in_sample_bce=0.049430 local_bce=0.049430 donor_bce=nan in_sample_local_ap=0.824765 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor rows=5222 failures=189 prevalence=0.036193 ap=0.296383 auc=0.814835 f2_at20=0.352582 brier=0.031960 ece10=0.023688 rho=0.250000 donor_rows=4096 effective_n=22359.120706 seconds=0.696297 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor in_sample_bce=0.040868 local_bce=0.051556 donor_bce=0.008807 in_sample_local_ap=0.810142 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor mass={'khoa_cong_nghe_thong_tin': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_cong_nghe_thong_tin': 576, np.str_('khoa_tieng_han_quoc'): 18} positive_ess=588.871335 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common rows=5222 failures=189 prevalence=0.036193 ap=0.234575 auc=0.809290 f2_at20=0.344253 brier=0.033141 ece10=0.026494 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=0.597531 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common in_sample_bce=0.049430 local_bce=0.049430 donor_bce=nan in_sample_local_ap=0.824765 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native rows=5222 failures=189 prevalence=0.036193 ap=0.259189 auc=0.821192 f2_at20=0.347029 brier=0.032553 ece10=0.024785 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=0.930285 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native in_sample_bce=0.047943 local_bce=0.047943 donor_bce=nan in_sample_local_ap=0.837533 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator rows=5222 failures=189 prevalence=0.036193 ap=0.259189 auc=0.821192 f2_at20=0.347029 brier=0.032553 ece10=0.024785 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=1.660521 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator in_sample_bce=0.047943 local_bce=0.047943 donor_bce=nan in_sample_local_ap=0.837533 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator rows=5222 failures=189 prevalence=0.036193 ap=0.259189 auc=0.821192 f2_at20=0.347029 brier=0.032553 ece10=0.024785 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=1.868696 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator in_sample_bce=0.047943 local_bce=0.047943 donor_bce=nan in_sample_local_ap=0.837533 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator rows=5222 failures=189 prevalence=0.036193 ap=0.259189 auc=0.821192 f2_at20=0.347029 brier=0.032553 ece10=0.024785 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=1.814536 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator in_sample_bce=0.047943 local_bce=0.047943 donor_bce=nan in_sample_local_ap=0.837533 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan rows=5222 failures=189 prevalence=0.036193 ap=0.259189 auc=0.821192 f2_at20=0.347029 brier=0.032553 ece10=0.024785 rho=0.000000 donor_rows=0 effective_n=19090.000000 seconds=1.767140 [TRAINING_INSIGHT] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan in_sample_bce=0.047943 local_bce=0.047943 donor_bce=nan in_sample_local_ap=0.837533 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_cong_nghe_thong_tin model=committee_reference ap=0.203171 [FACULTY_COMPLETE] fold=1 faculty=khoa_cong_nghe_thong_tin models=13 positive_rows=189 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=1 origins=[6062, 6063, 6064, 6065] availability=semester_proxy [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=1 origin=6062 ap=0.063163 rows=716 failures=10 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6062 delta_ap=0.005233 ap=0.068395 base_ap=0.063163 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6062 delta_ap=0.025523 ap=0.088685 base_ap=0.063163 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6062 delta_ap=0.106961 ap=0.170123 base_ap=0.063163 donor_failures=87 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6062 delta_ap=0.015033 ap=0.078195 base_ap=0.063163 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6062 delta_ap=0.132020 ap=0.195183 base_ap=0.063163 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6062 delta_ap=-0.007201 ap=0.055961 base_ap=0.063163 donor_failures=26 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6062 delta_ap=0.000409 ap=0.063572 base_ap=0.063163 donor_failures=140 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6062 delta_ap=0.010013 ap=0.073176 base_ap=0.063163 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6062 delta_ap=0.125788 ap=0.188951 base_ap=0.063163 donor_failures=52 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6062 delta_ap=-0.019053 ap=0.044110 base_ap=0.063163 donor_failures=48 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6062 delta_ap=0.120134 ap=0.183296 base_ap=0.063163 donor_failures=85 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6062 delta_ap=0.019468 ap=0.082630 base_ap=0.063163 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6062 delta_ap=-0.026124 ap=0.037038 base_ap=0.063163 donor_failures=217 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=1 origin=6063 ap=0.326431 rows=6115 failures=157 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.024474 ap=0.350905 base_ap=0.326431 donor_failures=169 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.061544 ap=0.264887 base_ap=0.326431 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6063 delta_ap=-0.098828 ap=0.227602 base_ap=0.326431 donor_failures=67 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.001506 ap=0.324925 base_ap=0.326431 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6063 delta_ap=-0.007933 ap=0.318498 base_ap=0.326431 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.037364 ap=0.289067 base_ap=0.326431 donor_failures=20 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6063 delta_ap=-0.050356 ap=0.276075 base_ap=0.326431 donor_failures=139 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6063 delta_ap=0.012116 ap=0.338547 base_ap=0.326431 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6063 delta_ap=-0.048482 ap=0.277948 base_ap=0.326431 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6063 delta_ap=-0.033317 ap=0.293114 base_ap=0.326431 donor_failures=55 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.019018 ap=0.307413 base_ap=0.326431 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.036865 ap=0.289565 base_ap=0.326431 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.044385 ap=0.282046 base_ap=0.326431 donor_failures=217 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=1 origin=6064 ap=0.589997 rows=4784 failures=257 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.028047 ap=0.561950 base_ap=0.589997 donor_failures=115 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.041630 ap=0.548367 base_ap=0.589997 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6064 delta_ap=-0.013676 ap=0.576322 base_ap=0.589997 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.040104 ap=0.549894 base_ap=0.589997 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6064 delta_ap=-0.022712 ap=0.567285 base_ap=0.589997 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.019370 ap=0.570627 base_ap=0.589997 donor_failures=18 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6064 delta_ap=-0.010461 ap=0.579536 base_ap=0.589997 donor_failures=113 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6064 delta_ap=-0.006160 ap=0.583837 base_ap=0.589997 donor_failures=146 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6064 delta_ap=-0.035955 ap=0.554043 base_ap=0.589997 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6064 delta_ap=-0.028034 ap=0.561964 base_ap=0.589997 donor_failures=43 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.002149 ap=0.587849 base_ap=0.589997 donor_failures=76 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.042516 ap=0.547481 base_ap=0.589997 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.032576 ap=0.557421 base_ap=0.589997 donor_failures=275 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=1 origin=6065 ap=0.618680 rows=687 failures=21 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=-0.120137 ap=0.498543 base_ap=0.618680 donor_failures=114 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6065 delta_ap=-0.108452 ap=0.510228 base_ap=0.618680 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6065 delta_ap=-0.075880 ap=0.542800 base_ap=0.618680 donor_failures=84 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=-0.140933 ap=0.477746 base_ap=0.618680 donor_failures=123 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6065 delta_ap=-0.113055 ap=0.505625 base_ap=0.618680 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6065 delta_ap=-0.093839 ap=0.524841 base_ap=0.618680 donor_failures=18 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6065 delta_ap=-0.147341 ap=0.471338 base_ap=0.618680 donor_failures=131 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6065 delta_ap=-0.195466 ap=0.423214 base_ap=0.618680 donor_failures=152 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6065 delta_ap=-0.079449 ap=0.539231 base_ap=0.618680 donor_failures=45 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6065 delta_ap=-0.097694 ap=0.520986 base_ap=0.618680 donor_failures=53 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=-0.087411 ap=0.531269 base_ap=0.618680 donor_failures=79 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6065 delta_ap=-0.101423 ap=0.517257 base_ap=0.618680 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6065 delta_ap=-0.100807 ap=0.517873 base_ap=0.618680 donor_failures=261 [POLICY_REPLAY] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 policy_sources=khoa_tieng_duc,khoa_tieng_tay_ban_nha,khoa_tieng_nhat policy_donor_count=3 base_ap=0.589997 policy_ap=0.526349 delta_ap=-0.063648 policy_donor_rows=4095 selected_using_origins=6062,6063 [POLICY_REPLAY] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6065 policy_sources=khoa_tieng_nga policy_donor_count=1 base_ap=0.618680 policy_ap=0.423214 delta_ap=-0.195466 policy_donor_rows=4096 selected_using_origins=6062,6063,6064 [SELECTION_FINAL] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich selected=[] reason=no_stable_positive_directed_edge replay_mean=-0.129557 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common rows=5672 failures=219 prevalence=0.038611 ap=0.501686 auc=0.918674 f2_at20=0.452511 brier=0.028717 ece10=0.022103 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=0.755422 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common in_sample_bce=0.044193 local_bce=0.044193 donor_bce=nan in_sample_local_ap=0.818899 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native rows=5672 failures=219 prevalence=0.038611 ap=0.484882 auc=0.917476 f2_at20=0.452511 brier=0.029405 ece10=0.022299 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=1.355481 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native in_sample_bce=0.043762 local_bce=0.043762 donor_bce=nan in_sample_local_ap=0.823123 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all rows=5672 failures=219 prevalence=0.038611 ap=0.525325 auc=0.908883 f2_at20=0.469915 brier=0.028874 ece10=0.023349 rho=0.250000 donor_rows=4095 effective_n=24696.819329 seconds=0.804969 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all in_sample_bce=0.043410 local_bce=0.049274 donor_bce=0.041749 in_sample_local_ap=0.775645 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.023031189224926286, 'khoa_quoc_te_hoc': 0.012594163662598508, 'khoa_tieng_anh': 0.057906128299045935, 'khoa_tieng_bo_dao_nha': 0.00496696731477326, 'khoa_tieng_duc': 0.013129828828437547, 'khoa_tieng_han_quoc': 0.020632760509412748, 'khoa_tieng_italia': 0.012848725261679673, 'khoa_tieng_nga': 0.007972241927622466, 'khoa_tieng_nhat': 0.022974485930344225, 'khoa_tieng_phap': 0.01841650620840753, 'khoa_tieng_tay_ban_nha': 0.009681785936617782, 'khoa_tieng_trung_quoc': 0.0395487382913729, 'khoa_viet_nam_hoc': 0.006296478604761146} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 7, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 4, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 21} positive_ess=750.956918 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal rows=5672 failures=219 prevalence=0.038611 ap=0.526468 auc=0.912311 f2_at20=0.464943 brier=0.029333 ece10=0.024255 rho=0.250000 donor_rows=4095 effective_n=31174.838710 seconds=0.833576 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal in_sample_bce=0.045805 local_bce=0.049823 donor_bce=0.033754 in_sample_local_ap=0.771928 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 7, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 4, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 21} positive_ess=850.549437 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior rows=5672 failures=219 prevalence=0.038611 ap=0.501686 auc=0.918674 f2_at20=0.452511 brier=0.028717 ece10=0.022103 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=0.758754 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior in_sample_bce=0.044193 local_bce=0.044193 donor_bce=nan in_sample_local_ap=0.818899 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor rows=5672 failures=219 prevalence=0.038611 ap=0.541859 auc=0.921730 f2_at20=0.462456 brier=0.028291 ece10=0.023047 rho=0.250000 donor_rows=4096 effective_n=31178.460515 seconds=0.793522 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor in_sample_bce=0.043746 local_bce=0.048692 donor_bce=0.028905 in_sample_local_ap=0.777404 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.25} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915, np.str_('khoa_tieng_tay_ban_nha'): 93} positive_ess=850.637894 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common rows=5672 failures=219 prevalence=0.038611 ap=0.501686 auc=0.918674 f2_at20=0.452511 brier=0.028717 ece10=0.022103 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=0.746811 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common in_sample_bce=0.044193 local_bce=0.044193 donor_bce=nan in_sample_local_ap=0.818899 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native rows=5672 failures=219 prevalence=0.038611 ap=0.484882 auc=0.917476 f2_at20=0.452511 brier=0.029405 ece10=0.022299 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=1.368994 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native in_sample_bce=0.043762 local_bce=0.043762 donor_bce=nan in_sample_local_ap=0.823123 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator rows=5672 failures=219 prevalence=0.038611 ap=0.484882 auc=0.917476 f2_at20=0.452511 brier=0.029405 ece10=0.022299 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=2.395963 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator in_sample_bce=0.043762 local_bce=0.043762 donor_bce=nan in_sample_local_ap=0.823123 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator rows=5672 failures=219 prevalence=0.038611 ap=0.486094 auc=0.917673 f2_at20=0.452511 brier=0.029389 ece10=0.022148 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=2.375248 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator in_sample_bce=0.043829 local_bce=0.043829 donor_bce=nan in_sample_local_ap=0.822414 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator rows=5672 failures=219 prevalence=0.038611 ap=0.484882 auc=0.917476 f2_at20=0.452511 brier=0.029405 ece10=0.022299 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=2.355712 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator in_sample_bce=0.043762 local_bce=0.043762 donor_bce=nan in_sample_local_ap=0.823123 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan rows=5672 failures=219 prevalence=0.038611 ap=0.484882 auc=0.917476 f2_at20=0.452511 brier=0.029405 ece10=0.022299 rho=0.000000 donor_rows=0 effective_n=33453.000000 seconds=2.285843 [TRAINING_INSIGHT] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan in_sample_bce=0.043762 local_bce=0.043762 donor_bce=nan in_sample_local_ap=0.823123 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 915} positive_ess=915.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=committee_reference ap=0.517408 [FACULTY_COMPLETE] fold=1 faculty=khoa_quan_tri_kinh_doanh_va_du_lich models=13 positive_rows=219 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quoc_te_hoc fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_quoc_te_hoc fold=1 origin=6060 ap=0.055331 rows=1841 failures=6 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=-0.026676 ap=0.028655 base_ap=0.055331 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=-0.013976 ap=0.041356 base_ap=0.055331 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6060 delta_ap=-0.043005 ap=0.012326 base_ap=0.055331 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.001692 ap=0.057023 base_ap=0.055331 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6060 delta_ap=-0.010015 ap=0.045317 base_ap=0.055331 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6060 delta_ap=0.016129 ap=0.071461 base_ap=0.055331 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6060 delta_ap=-0.035324 ap=0.020007 base_ap=0.055331 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6060 delta_ap=-0.034336 ap=0.020995 base_ap=0.055331 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6060 delta_ap=-0.038873 ap=0.016458 base_ap=0.055331 donor_failures=64 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6060 delta_ap=-0.037921 ap=0.017410 base_ap=0.055331 donor_failures=69 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=-0.035589 ap=0.019743 base_ap=0.055331 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6060 delta_ap=-0.029913 ap=0.025418 base_ap=0.055331 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6060 delta_ap=0.134227 ap=0.189558 base_ap=0.055331 donor_failures=121 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=1 origin=6061 ap=0.549770 rows=1641 failures=12 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.133895 ap=0.683666 base_ap=0.549770 donor_failures=178 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.095908 ap=0.645678 base_ap=0.549770 donor_failures=93 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6061 delta_ap=0.126403 ap=0.676174 base_ap=0.549770 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.017691 ap=0.567461 base_ap=0.549770 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6061 delta_ap=0.118103 ap=0.667874 base_ap=0.549770 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6061 delta_ap=0.080868 ap=0.630639 base_ap=0.549770 donor_failures=24 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6061 delta_ap=0.135219 ap=0.684990 base_ap=0.549770 donor_failures=163 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6061 delta_ap=0.039587 ap=0.589358 base_ap=0.549770 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6061 delta_ap=0.107802 ap=0.657572 base_ap=0.549770 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6061 delta_ap=0.040355 ap=0.590126 base_ap=0.549770 donor_failures=64 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.031063 ap=0.518707 base_ap=0.549770 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.090768 ap=0.640538 base_ap=0.549770 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6061 delta_ap=0.051353 ap=0.601123 base_ap=0.549770 donor_failures=180 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=1 origin=6063 ap=0.048420 rows=2231 failures=17 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.090437 ap=0.138857 base_ap=0.048420 donor_failures=151 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.087828 ap=0.136248 base_ap=0.048420 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6063 delta_ap=0.150195 ap=0.198615 base_ap=0.048420 donor_failures=72 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.142023 ap=0.190443 base_ap=0.048420 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6063 delta_ap=0.197686 ap=0.246107 base_ap=0.048420 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.070773 ap=0.119194 base_ap=0.048420 donor_failures=25 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6063 delta_ap=0.133199 ap=0.181619 base_ap=0.048420 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6063 delta_ap=0.132272 ap=0.180692 base_ap=0.048420 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6063 delta_ap=0.133097 ap=0.181517 base_ap=0.048420 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6063 delta_ap=0.060631 ap=0.109052 base_ap=0.048420 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.187030 ap=0.235450 base_ap=0.048420 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.159449 ap=0.207869 base_ap=0.048420 donor_failures=25 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.063043 ap=0.111464 base_ap=0.048420 donor_failures=217 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=1 origin=6064 ap=0.292454 rows=1690 failures=30 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.073464 ap=0.365919 base_ap=0.292454 donor_failures=132 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.077504 ap=0.369958 base_ap=0.292454 donor_failures=99 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6064 delta_ap=0.082683 ap=0.375137 base_ap=0.292454 donor_failures=73 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.053033 ap=0.345488 base_ap=0.292454 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6064 delta_ap=0.002739 ap=0.295193 base_ap=0.292454 donor_failures=55 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.004982 ap=0.287473 base_ap=0.292454 donor_failures=19 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6064 delta_ap=0.055515 ap=0.347969 base_ap=0.292454 donor_failures=123 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6064 delta_ap=0.080939 ap=0.373394 base_ap=0.292454 donor_failures=151 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6064 delta_ap=0.039217 ap=0.331671 base_ap=0.292454 donor_failures=38 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6064 delta_ap=0.090661 ap=0.383115 base_ap=0.292454 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.011063 ap=0.303517 base_ap=0.292454 donor_failures=72 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.058570 ap=0.351024 base_ap=0.292454 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6064 delta_ap=0.041515 ap=0.333969 base_ap=0.292454 donor_failures=271 [POLICY_REPLAY] fold=1 faculty=khoa_quoc_te_hoc origin=6063 policy_sources=khoa_viet_nam_hoc,khoa_tieng_duc,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.048420 policy_ap=0.167274 delta_ap=0.118853 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_quoc_te_hoc origin=6064 policy_sources=khoa_tieng_duc,khoa_viet_nam_hoc,khoa_tieng_anh policy_donor_count=3 base_ap=0.292454 policy_ap=0.337950 delta_ap=0.045496 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_quoc_te_hoc selected=['khoa_tieng_anh', 'khoa_tieng_duc', 'khoa_viet_nam_hoc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.082175 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_local_common rows=2363 failures=31 prevalence=0.013119 ap=0.036644 auc=0.710632 f2_at20=0.117253 brier=0.013056 ece10=0.011882 rho=0.000000 donor_rows=0 effective_n=10439.000000 seconds=0.500410 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=hgb_local_common in_sample_bce=0.005000 local_bce=0.005000 donor_bce=nan in_sample_local_ap=0.998621 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_local_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 98} positive_ess=98.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_local_native rows=2363 failures=31 prevalence=0.013119 ap=0.032168 auc=0.657403 f2_at20=0.067002 brier=0.013076 ece10=0.011857 rho=0.000000 donor_rows=0 effective_n=10439.000000 seconds=0.682698 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=hgb_local_native in_sample_bce=0.004914 local_bce=0.004914 donor_bce=nan in_sample_local_ap=0.999289 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_local_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 98} positive_ess=98.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_pool_all rows=2363 failures=31 prevalence=0.013119 ap=0.051290 auc=0.787072 f2_at20=0.159129 brier=0.013173 ece10=0.011721 rho=0.250000 donor_rows=4095 effective_n=13013.186599 seconds=0.577953 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=hgb_pool_all in_sample_bce=0.010652 local_bce=0.008457 donor_bce=0.031717 in_sample_local_ap=0.960619 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_pool_all mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.020729000621109918, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.036325157557778424, 'khoa_tieng_anh': 0.0521178545212893, 'khoa_tieng_bo_dao_nha': 0.004470471218287561, 'khoa_tieng_duc': 0.01181737631008587, 'khoa_tieng_han_quoc': 0.018570317895349495, 'khoa_tieng_italia': 0.011564371745145135, 'khoa_tieng_nga': 0.007175339764499442, 'khoa_tieng_nhat': 0.020677965365521016, 'khoa_tieng_phap': 0.016575599501374695, 'khoa_tieng_tay_ban_nha': 0.008713998427679785, 'khoa_tieng_trung_quoc': 0.03559546198850729, 'khoa_viet_nam_hoc': 0.005667085083372062} source_failures={'khoa_quoc_te_hoc': 98, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 6, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 6, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 23} positive_ess=148.857394 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_pool_equal rows=2363 failures=31 prevalence=0.013119 ap=0.049054 auc=0.742198 f2_at20=0.117253 brier=0.013002 ece10=0.011393 rho=0.250000 donor_rows=4095 effective_n=14461.946124 seconds=0.562580 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=hgb_pool_equal in_sample_bce=0.012917 local_bce=0.009177 donor_bce=0.024135 in_sample_local_ap=0.951221 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_pool_equal mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quoc_te_hoc': 98, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 6, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 6, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 23} positive_ess=186.778101 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_language_prior rows=2363 failures=31 prevalence=0.013119 ap=0.036644 auc=0.710632 f2_at20=0.117253 brier=0.013056 ece10=0.011882 rho=0.000000 donor_rows=0 effective_n=10439.000000 seconds=0.484578 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=hgb_language_prior in_sample_bce=0.005000 local_bce=0.005000 donor_bce=nan in_sample_local_ap=0.998621 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_language_prior mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 98} positive_ess=98.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_random_donor rows=2363 failures=31 prevalence=0.013119 ap=0.045097 auc=0.775964 f2_at20=0.117253 brier=0.013030 ece10=0.011265 rho=0.250000 donor_rows=4095 effective_n=14461.946124 seconds=0.593108 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=hgb_random_donor in_sample_bce=0.014130 local_bce=0.009637 donor_bce=0.027608 in_sample_local_ap=0.950063 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=hgb_random_donor mass={'khoa_quoc_te_hoc': 0.75, np.str_('khoa_tieng_bo_dao_nha'): 0.08333333333333333, np.str_('khoa_tieng_anh'): 0.08333333333333333, np.str_('khoa_tieng_duc'): 0.08333333333333333} source_failures={'khoa_quoc_te_hoc': 98, np.str_('khoa_tieng_bo_dao_nha'): 38, np.str_('khoa_tieng_anh'): 19, np.str_('khoa_tieng_duc'): 24} positive_ess=177.854623 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_common rows=2363 failures=31 prevalence=0.013119 ap=0.077257 auc=0.830500 f2_at20=0.167504 brier=0.012933 ece10=0.010721 rho=0.250000 donor_rows=4095 effective_n=14457.824815 seconds=0.580248 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_common in_sample_bce=0.017374 local_bce=0.010388 donor_bce=0.039018 in_sample_local_ap=0.933356 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_common mass={'khoa_quoc_te_hoc': 0.75, 'khoa_tieng_anh': 0.08642094950968471, 'khoa_tieng_duc': 0.08430002927088345, 'khoa_viet_nam_hoc': 0.07927902121943183} source_failures={'khoa_quoc_te_hoc': 98, 'khoa_tieng_anh': 19, 'khoa_tieng_duc': 24, 'khoa_viet_nam_hoc': 97} positive_ess=235.742289 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_native rows=2363 failures=31 prevalence=0.013119 ap=0.076969 auc=0.860704 f2_at20=0.201005 brier=0.012924 ece10=0.010722 rho=0.250000 donor_rows=4095 effective_n=14457.824815 seconds=0.771687 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_native in_sample_bce=0.017389 local_bce=0.009934 donor_bce=0.040449 in_sample_local_ap=0.938882 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_native mass={'khoa_quoc_te_hoc': 0.75, 'khoa_tieng_anh': 0.08642094950968471, 'khoa_tieng_duc': 0.08430002927088345, 'khoa_viet_nam_hoc': 0.07927902121943183} source_failures={'khoa_quoc_te_hoc': 98, 'khoa_tieng_anh': 19, 'khoa_tieng_duc': 24, 'khoa_viet_nam_hoc': 97} positive_ess=235.742289 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator rows=2363 failures=31 prevalence=0.013119 ap=0.032168 auc=0.657403 f2_at20=0.067002 brier=0.013076 ece10=0.011857 rho=0.000000 donor_rows=0 effective_n=10439.000000 seconds=1.184028 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator in_sample_bce=0.004914 local_bce=0.004914 donor_bce=nan in_sample_local_ap=0.999289 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 98} positive_ess=98.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator rows=2363 failures=31 prevalence=0.013119 ap=0.033398 auc=0.667792 f2_at20=0.067002 brier=0.013074 ece10=0.011479 rho=0.000000 donor_rows=0 effective_n=10439.000000 seconds=1.181189 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator in_sample_bce=0.004926 local_bce=0.004926 donor_bce=nan in_sample_local_ap=0.999289 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 98} positive_ess=98.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=native_placebo_indicator rows=2363 failures=31 prevalence=0.013119 ap=0.032168 auc=0.657403 f2_at20=0.067002 brier=0.013076 ece10=0.011857 rho=0.000000 donor_rows=0 effective_n=10439.000000 seconds=1.233217 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=native_placebo_indicator in_sample_bce=0.004914 local_bce=0.004914 donor_bce=nan in_sample_local_ap=0.999289 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=native_placebo_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 98} positive_ess=98.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan rows=2363 failures=31 prevalence=0.013119 ap=0.076969 auc=0.860704 f2_at20=0.201005 brier=0.012924 ece10=0.010722 rho=0.250000 donor_rows=4095 effective_n=14457.824815 seconds=1.308706 [TRAINING_INSIGHT] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.017389 local_bce=0.009934 donor_bce=0.040449 in_sample_local_ap=0.938882 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan mass={'khoa_quoc_te_hoc': 0.75, 'khoa_tieng_anh': 0.08642094950968471, 'khoa_tieng_duc': 0.08430002927088345, 'khoa_viet_nam_hoc': 0.07927902121943183} source_failures={'khoa_quoc_te_hoc': 98, 'khoa_tieng_anh': 19, 'khoa_tieng_duc': 24, 'khoa_viet_nam_hoc': 97} positive_ess=235.742289 [FACULTY_RESULTS] fold=1 faculty=khoa_quoc_te_hoc model=committee_reference ap=0.070421 [FACULTY_COMPLETE] fold=1 faculty=khoa_quoc_te_hoc models=13 positive_rows=31 num_edges=52 num_replays=2 chosen=['khoa_tieng_anh', 'khoa_tieng_duc', 'khoa_viet_nam_hoc'] [SELECTION_ORIGINS] target=khoa_tieng_anh fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_anh fold=1 origin=6060 ap=0.197517 rows=8314 failures=170 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=-0.015080 ap=0.182438 base_ap=0.197517 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=-0.006490 ap=0.191027 base_ap=0.197517 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6060 delta_ap=-0.004730 ap=0.192787 base_ap=0.197517 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.101978 ap=0.299496 base_ap=0.197517 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_duc origin=6060 delta_ap=0.048297 ap=0.245814 base_ap=0.197517 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6060 delta_ap=-0.023540 ap=0.173977 base_ap=0.197517 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_italia origin=6060 delta_ap=-0.067541 ap=0.129976 base_ap=0.197517 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nga origin=6060 delta_ap=-0.015774 ap=0.181744 base_ap=0.197517 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6060 delta_ap=-0.035919 ap=0.161599 base_ap=0.197517 donor_failures=68 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_phap origin=6060 delta_ap=-0.018576 ap=0.178942 base_ap=0.197517 donor_failures=74 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.000292 ap=0.197810 base_ap=0.197517 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6060 delta_ap=-0.000826 ap=0.196691 base_ap=0.197517 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6060 delta_ap=0.008882 ap=0.206400 base_ap=0.197517 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_anh fold=1 origin=6061 ap=0.438113 rows=8489 failures=245 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.063205 ap=0.501318 base_ap=0.438113 donor_failures=179 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.018600 ap=0.419513 base_ap=0.438113 donor_failures=70 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6061 delta_ap=0.089791 ap=0.527904 base_ap=0.438113 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.163080 ap=0.601193 base_ap=0.438113 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_duc origin=6061 delta_ap=0.018342 ap=0.456455 base_ap=0.438113 donor_failures=85 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6061 delta_ap=0.126744 ap=0.564857 base_ap=0.438113 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_italia origin=6061 delta_ap=0.163007 ap=0.601120 base_ap=0.438113 donor_failures=159 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nga origin=6061 delta_ap=0.053278 ap=0.491391 base_ap=0.438113 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6061 delta_ap=0.168067 ap=0.606180 base_ap=0.438113 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_phap origin=6061 delta_ap=0.104548 ap=0.542661 base_ap=0.438113 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.039196 ap=0.477309 base_ap=0.438113 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.032514 ap=0.470627 base_ap=0.438113 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.071845 ap=0.366268 base_ap=0.438113 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_anh fold=1 origin=6063 ap=0.258771 rows=9030 failures=67 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.090426 ap=0.349196 base_ap=0.258771 donor_failures=148 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.037048 ap=0.295819 base_ap=0.258771 donor_failures=76 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6063 delta_ap=0.163875 ap=0.422646 base_ap=0.258771 donor_failures=35 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.046913 ap=0.211857 base_ap=0.258771 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_duc origin=6063 delta_ap=0.047458 ap=0.306228 base_ap=0.258771 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.038816 ap=0.219954 base_ap=0.258771 donor_failures=13 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_italia origin=6063 delta_ap=0.090240 ap=0.349011 base_ap=0.258771 donor_failures=143 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nga origin=6063 delta_ap=0.005206 ap=0.263977 base_ap=0.258771 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6063 delta_ap=0.099951 ap=0.358722 base_ap=0.258771 donor_failures=64 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_phap origin=6063 delta_ap=-0.046920 ap=0.211851 base_ap=0.258771 donor_failures=57 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.092960 ap=0.165811 base_ap=0.258771 donor_failures=92 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.020633 ap=0.238138 base_ap=0.258771 donor_failures=21 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6063 delta_ap=0.063666 ap=0.322436 base_ap=0.258771 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_anh fold=1 origin=6064 ap=0.581902 rows=9726 failures=240 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.016702 ap=0.565201 base_ap=0.581902 donor_failures=133 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.062019 ap=0.519884 base_ap=0.581902 donor_failures=107 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.008511 ap=0.573392 base_ap=0.581902 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.030029 ap=0.551873 base_ap=0.581902 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_duc origin=6064 delta_ap=-0.021434 ap=0.560468 base_ap=0.581902 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.013722 ap=0.568180 base_ap=0.581902 donor_failures=20 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_italia origin=6064 delta_ap=-0.016359 ap=0.565543 base_ap=0.581902 donor_failures=118 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nga origin=6064 delta_ap=-0.008722 ap=0.573180 base_ap=0.581902 donor_failures=144 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6064 delta_ap=-0.015739 ap=0.566163 base_ap=0.581902 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_phap origin=6064 delta_ap=-0.020038 ap=0.561864 base_ap=0.581902 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.027061 ap=0.554841 base_ap=0.581902 donor_failures=70 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.014787 ap=0.567115 base_ap=0.581902 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.042469 ap=0.539433 base_ap=0.581902 donor_failures=267 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_anh origin=6063 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_nhat,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.258771 policy_ap=0.312554 delta_ap=0.053783 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_anh origin=6064 policy_sources=khoa_quoc_te_hoc,khoa_tieng_nhat,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.581902 policy_ap=0.557798 delta_ap=-0.024104 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_anh selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.014840 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=hgb_local_common rows=8164 failures=148 prevalence=0.018128 ap=0.533165 auc=0.918749 f2_at20=0.294382 brier=0.010954 ece10=0.004961 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=0.870995 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=hgb_local_common in_sample_bce=0.019914 local_bce=0.019914 donor_bce=nan in_sample_local_ap=0.921535 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=hgb_local_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=hgb_local_native rows=8164 failures=148 prevalence=0.018128 ap=0.532250 auc=0.919480 f2_at20=0.292135 brier=0.011518 ece10=0.006232 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=2.452517 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=hgb_local_native in_sample_bce=0.020052 local_bce=0.020052 donor_bce=nan in_sample_local_ap=0.920644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=hgb_local_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=hgb_pool_all rows=8164 failures=148 prevalence=0.018128 ap=0.568922 auc=0.928966 f2_at20=0.287640 brier=0.010535 ece10=0.004231 rho=0.250000 donor_rows=4095 effective_n=30512.088342 seconds=0.895559 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=hgb_pool_all in_sample_bce=0.026089 local_bce=0.026005 donor_bce=0.039122 in_sample_local_ap=0.871164 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=hgb_pool_all mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.024769689892305696, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04340599455040872, 'khoa_quoc_te_hoc': 0.013544829375892046, 'khoa_tieng_bo_dao_nha': 0.005341896976774361, 'khoa_tieng_duc': 0.014120929025561179, 'khoa_tieng_han_quoc': 0.022190216686129492, 'khoa_tieng_italia': 0.013818606461658233, 'khoa_tieng_nga': 0.00857402361489555, 'khoa_tieng_nhat': 0.024708706370831713, 'khoa_tieng_phap': 0.019806669261710135, 'khoa_tieng_tay_ban_nha': 0.010412611911249514, 'khoa_tieng_trung_quoc': 0.042534059945504084, 'khoa_viet_nam_hoc': 0.006771765927079279} source_failures={'khoa_tieng_anh': 821, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 8, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 21} positive_ess=534.848085 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=hgb_pool_equal rows=8164 failures=148 prevalence=0.018128 ap=0.582659 auc=0.921691 f2_at20=0.289888 brier=0.010571 ece10=0.004784 rho=0.250000 donor_rows=4095 effective_n=37061.747985 seconds=0.924179 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=hgb_pool_equal in_sample_bce=0.027685 local_bce=0.025995 donor_bce=0.032755 in_sample_local_ap=0.870604 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=hgb_pool_equal mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_anh': 821, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 8, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 21} positive_ess=625.422262 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=hgb_language_prior rows=8164 failures=148 prevalence=0.018128 ap=0.578287 auc=0.922226 f2_at20=0.285393 brier=0.010521 ece10=0.002917 rho=0.250000 donor_rows=4095 effective_n=37061.747985 seconds=0.915617 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=hgb_language_prior in_sample_bce=0.025921 local_bce=0.025580 donor_bce=0.026944 in_sample_local_ap=0.878966 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=hgb_language_prior mass={'khoa_tieng_anh': 0.75, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_anh': 821, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 16, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 11, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3} positive_ess=628.860111 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=hgb_random_donor rows=8164 failures=148 prevalence=0.018128 ap=0.573418 auc=0.916041 f2_at20=0.289888 brier=0.010502 ece10=0.004443 rho=0.250000 donor_rows=4096 effective_n=37066.866900 seconds=0.908395 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=hgb_random_donor in_sample_bce=0.034229 local_bce=0.028467 donor_bce=0.051516 in_sample_local_ap=0.850962 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=hgb_random_donor mass={'khoa_tieng_anh': 0.75, np.str_('khoa_cong_nghe_thong_tin'): 0.25} source_failures={'khoa_tieng_anh': 821, np.str_('khoa_cong_nghe_thong_tin'): 136} positive_ess=631.398731 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=suite32_selective_common rows=8164 failures=148 prevalence=0.018128 ap=0.533165 auc=0.918749 f2_at20=0.294382 brier=0.010954 ece10=0.004961 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=0.884209 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=suite32_selective_common in_sample_bce=0.019914 local_bce=0.019914 donor_bce=nan in_sample_local_ap=0.921535 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=suite32_selective_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=suite32_selective_native rows=8164 failures=148 prevalence=0.018128 ap=0.532250 auc=0.919480 f2_at20=0.292135 brier=0.011518 ece10=0.006232 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=2.447816 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=suite32_selective_native in_sample_bce=0.020052 local_bce=0.020052 donor_bce=nan in_sample_local_ap=0.920644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=suite32_selective_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=native_cache_nan_indicator rows=8164 failures=148 prevalence=0.018128 ap=0.532250 auc=0.919480 f2_at20=0.292135 brier=0.011518 ece10=0.006232 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=4.652528 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=native_cache_nan_indicator in_sample_bce=0.020052 local_bce=0.020052 donor_bce=nan in_sample_local_ap=0.920644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=native_cache_nan_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=native_naive_zero_indicator rows=8164 failures=148 prevalence=0.018128 ap=0.532250 auc=0.919480 f2_at20=0.292135 brier=0.011518 ece10=0.006232 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=5.043818 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=native_naive_zero_indicator in_sample_bce=0.020052 local_bce=0.020052 donor_bce=nan in_sample_local_ap=0.920644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=native_naive_zero_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=native_placebo_indicator rows=8164 failures=148 prevalence=0.018128 ap=0.532250 auc=0.919480 f2_at20=0.292135 brier=0.011518 ece10=0.006232 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=4.724012 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=native_placebo_indicator in_sample_bce=0.020052 local_bce=0.020052 donor_bce=nan in_sample_local_ap=0.920644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=native_placebo_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan rows=8164 failures=148 prevalence=0.018128 ap=0.532250 auc=0.919480 f2_at20=0.292135 brier=0.011518 ece10=0.006232 rho=0.000000 donor_rows=0 effective_n=47997.000000 seconds=4.879156 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan in_sample_bce=0.020052 local_bce=0.020052 donor_bce=nan in_sample_local_ap=0.920644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 821} positive_ess=821.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_anh model=committee_reference ap=0.585634 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_anh models=13 positive_rows=148 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_bo_dao_nha fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=1 origin=6060 ap=0.260177 rows=703 failures=9 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=-0.075685 ap=0.184493 base_ap=0.260177 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.015610 ap=0.275787 base_ap=0.260177 donor_failures=88 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6060 delta_ap=-0.012633 ap=0.247544 base_ap=0.260177 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6060 delta_ap=-0.044324 ap=0.215853 base_ap=0.260177 donor_failures=41 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6060 delta_ap=0.119113 ap=0.379291 base_ap=0.260177 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6060 delta_ap=-0.025621 ap=0.234556 base_ap=0.260177 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6060 delta_ap=0.087314 ap=0.347491 base_ap=0.260177 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6060 delta_ap=0.002327 ap=0.262504 base_ap=0.260177 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6060 delta_ap=0.018155 ap=0.278332 base_ap=0.260177 donor_failures=64 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6060 delta_ap=-0.123676 ap=0.136501 base_ap=0.260177 donor_failures=75 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.085787 ap=0.345964 base_ap=0.260177 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6060 delta_ap=0.094664 ap=0.354842 base_ap=0.260177 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6060 delta_ap=0.077039 ap=0.337216 base_ap=0.260177 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=1 origin=6061 ap=0.401922 rows=785 failures=24 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.042850 ap=0.444771 base_ap=0.401922 donor_failures=179 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.003154 ap=0.398768 base_ap=0.401922 donor_failures=98 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6061 delta_ap=0.013309 ap=0.415231 base_ap=0.401922 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6061 delta_ap=0.052649 ap=0.454571 base_ap=0.401922 donor_failures=42 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6061 delta_ap=0.011095 ap=0.413017 base_ap=0.401922 donor_failures=79 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.011164 ap=0.390758 base_ap=0.401922 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6061 delta_ap=0.003962 ap=0.405884 base_ap=0.401922 donor_failures=158 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6061 delta_ap=0.003784 ap=0.405706 base_ap=0.401922 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6061 delta_ap=-0.010200 ap=0.391722 base_ap=0.401922 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6061 delta_ap=0.010513 ap=0.412435 base_ap=0.401922 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.010187 ap=0.391735 base_ap=0.401922 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6061 delta_ap=-0.004150 ap=0.397772 base_ap=0.401922 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.040704 ap=0.361218 base_ap=0.401922 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=1 origin=6063 ap=0.094793 rows=856 failures=15 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.069175 ap=0.163968 base_ap=0.094793 donor_failures=151 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.000267 ap=0.095060 base_ap=0.094793 donor_failures=104 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.004454 ap=0.090339 base_ap=0.094793 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6063 delta_ap=0.029289 ap=0.124083 base_ap=0.094793 donor_failures=76 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6063 delta_ap=-0.011214 ap=0.083580 base_ap=0.094793 donor_failures=55 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6063 delta_ap=0.001614 ap=0.096407 base_ap=0.094793 donor_failures=25 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6063 delta_ap=-0.003787 ap=0.091006 base_ap=0.094793 donor_failures=145 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6063 delta_ap=0.085695 ap=0.180488 base_ap=0.094793 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6063 delta_ap=0.028650 ap=0.123443 base_ap=0.094793 donor_failures=42 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6063 delta_ap=-0.006266 ap=0.088527 base_ap=0.094793 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.042910 ap=0.137703 base_ap=0.094793 donor_failures=85 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.031791 ap=0.126585 base_ap=0.094793 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.009589 ap=0.085204 base_ap=0.094793 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=1 origin=6064 ap=0.179113 rows=780 failures=26 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.017634 ap=0.161479 base_ap=0.179113 donor_failures=119 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.047363 ap=0.226476 base_ap=0.179113 donor_failures=96 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.021792 ap=0.157320 base_ap=0.179113 donor_failures=38 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6064 delta_ap=-0.017363 ap=0.161750 base_ap=0.179113 donor_failures=59 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6064 delta_ap=0.034955 ap=0.214067 base_ap=0.179113 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.011476 ap=0.167636 base_ap=0.179113 donor_failures=15 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6064 delta_ap=-0.003543 ap=0.175570 base_ap=0.179113 donor_failures=122 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6064 delta_ap=0.001692 ap=0.180804 base_ap=0.179113 donor_failures=155 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6064 delta_ap=-0.003340 ap=0.175773 base_ap=0.179113 donor_failures=55 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6064 delta_ap=0.017089 ap=0.196201 base_ap=0.179113 donor_failures=43 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.030194 ap=0.148919 base_ap=0.179113 donor_failures=77 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.005883 ap=0.173230 base_ap=0.179113 donor_failures=23 [TRANSFER_EDGE] fold=1 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.012349 ap=0.166764 base_ap=0.179113 donor_failures=272 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_bo_dao_nha origin=6063 policy_sources=khoa_tieng_duc,khoa_tieng_italia,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.094793 policy_ap=0.106550 delta_ap=0.011757 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_bo_dao_nha origin=6064 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_duc,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.179113 policy_ap=0.192387 delta_ap=0.013275 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_bo_dao_nha selected=['khoa_tieng_duc', 'khoa_tieng_trung_quoc', 'khoa_tieng_nga'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.012516 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common rows=768 failures=30 prevalence=0.039062 ap=0.114820 auc=0.792638 f2_at20=0.291971 brier=0.037970 ece10=0.032339 rho=0.000000 donor_rows=0 effective_n=4117.000000 seconds=0.375104 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common in_sample_bce=0.018758 local_bce=0.018758 donor_bce=nan in_sample_local_ap=0.991211 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 123} positive_ess=123.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native rows=768 failures=30 prevalence=0.039062 ap=0.104590 auc=0.764115 f2_at20=0.273723 brier=0.037987 ece10=0.032790 rho=0.000000 donor_rows=0 effective_n=4117.000000 seconds=0.666263 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native in_sample_bce=0.018525 local_bce=0.018525 donor_bce=nan in_sample_local_ap=0.990886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 123} positive_ess=123.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all rows=768 failures=30 prevalence=0.039062 ap=0.142154 auc=0.856052 f2_at20=0.364964 brier=0.037727 ece10=0.032779 rho=0.250000 donor_rows=4095 effective_n=6301.010706 seconds=0.537600 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all in_sample_bce=0.028423 local_bce=0.025410 donor_bce=0.049859 in_sample_local_ap=0.968591 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.020175012153621778, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03535435733761704, 'khoa_quoc_te_hoc': 0.011032318065566147, 'khoa_tieng_anh': 0.05072498996005157, 'khoa_tieng_duc': 0.011501553549914396, 'khoa_tieng_han_quoc': 0.01807402084081926, 'khoa_tieng_italia': 0.011255310604299212, 'khoa_tieng_nga': 0.006983576758047811, 'khoa_tieng_nhat': 0.02012534082982816, 'khoa_tieng_phap': 0.016132611866162203, 'khoa_tieng_tay_ban_nha': 0.008481114328591659, 'khoa_tieng_trung_quoc': 0.034644163091035914, 'khoa_viet_nam_hoc': 0.005515630614444844} source_failures={'khoa_tieng_bo_dao_nha': 123, 'khoa_cong_nghe_thong_tin': 16, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 10, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 13} positive_ess=167.868347 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal rows=768 failures=30 prevalence=0.039062 ap=0.137539 auc=0.842412 f2_at20=0.273723 brier=0.037761 ece10=0.033891 rho=0.250000 donor_rows=4095 effective_n=6583.662989 seconds=0.500907 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal in_sample_bce=0.030737 local_bce=0.025337 donor_bce=0.046936 in_sample_local_ap=0.969814 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_bo_dao_nha': 123, 'khoa_cong_nghe_thong_tin': 16, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 10, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 13} positive_ess=176.235771 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior rows=768 failures=30 prevalence=0.039062 ap=0.150112 auc=0.844535 f2_at20=0.328467 brier=0.037617 ece10=0.032390 rho=0.250000 donor_rows=4095 effective_n=6583.662989 seconds=0.607856 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior in_sample_bce=0.027656 local_bce=0.023809 donor_bce=0.039196 in_sample_local_ap=0.978188 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_bo_dao_nha': 123, 'khoa_tieng_anh': 11, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 7, 'khoa_tieng_phap': 8, 'khoa_tieng_tay_ban_nha': 12, 'khoa_tieng_trung_quoc': 2} positive_ess=165.135049 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor rows=768 failures=30 prevalence=0.039062 ap=0.128982 auc=0.833243 f2_at20=0.328467 brier=0.038173 ece10=0.033033 rho=0.250000 donor_rows=4095 effective_n=6583.662989 seconds=0.479832 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor in_sample_bce=0.028506 local_bce=0.025150 donor_bce=0.038573 in_sample_local_ap=0.970155 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor mass={'khoa_tieng_bo_dao_nha': 0.75, np.str_('khoa_cong_nghe_thong_tin'): 0.08333333333333333, np.str_('khoa_quoc_te_hoc'): 0.08333333333333333, np.str_('khoa_tieng_nhat'): 0.08333333333333333} source_failures={'khoa_tieng_bo_dao_nha': 123, np.str_('khoa_cong_nghe_thong_tin'): 37, np.str_('khoa_quoc_te_hoc'): 15, np.str_('khoa_tieng_nhat'): 19} positive_ess=164.524587 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common rows=768 failures=30 prevalence=0.039062 ap=0.152384 auc=0.807543 f2_at20=0.273723 brier=0.037392 ece10=0.032113 rho=0.250000 donor_rows=4095 effective_n=6555.860305 seconds=0.490433 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common in_sample_bce=0.027914 local_bce=0.025204 donor_bce=0.038582 in_sample_local_ap=0.971207 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_duc': 0.10577177685504496, 'khoa_tieng_trung_quoc': 0.07998951818120807, 'khoa_tieng_nga': 0.06423870496374695} source_failures={'khoa_tieng_bo_dao_nha': 123, 'khoa_tieng_duc': 24, 'khoa_tieng_trung_quoc': 10, 'khoa_tieng_nga': 58} positive_ess=173.346034 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native rows=768 failures=30 prevalence=0.039062 ap=0.153520 auc=0.803930 f2_at20=0.273723 brier=0.037273 ece10=0.031914 rho=0.250000 donor_rows=4095 effective_n=6555.860305 seconds=0.907512 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native in_sample_bce=0.027360 local_bce=0.024221 donor_bce=0.039403 in_sample_local_ap=0.974664 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_duc': 0.10577177685504496, 'khoa_tieng_trung_quoc': 0.07998951818120807, 'khoa_tieng_nga': 0.06423870496374695} source_failures={'khoa_tieng_bo_dao_nha': 123, 'khoa_tieng_duc': 24, 'khoa_tieng_trung_quoc': 10, 'khoa_tieng_nga': 58} positive_ess=173.346034 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator rows=768 failures=30 prevalence=0.039062 ap=0.104590 auc=0.764115 f2_at20=0.273723 brier=0.037987 ece10=0.032790 rho=0.000000 donor_rows=0 effective_n=4117.000000 seconds=1.256930 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator in_sample_bce=0.018525 local_bce=0.018525 donor_bce=nan in_sample_local_ap=0.990886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 123} positive_ess=123.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator rows=768 failures=30 prevalence=0.039062 ap=0.130936 auc=0.770980 f2_at20=0.291971 brier=0.037722 ece10=0.032008 rho=0.000000 donor_rows=0 effective_n=4117.000000 seconds=1.155728 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator in_sample_bce=0.018519 local_bce=0.018519 donor_bce=nan in_sample_local_ap=0.992198 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 123} positive_ess=123.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator rows=768 failures=30 prevalence=0.039062 ap=0.104590 auc=0.764115 f2_at20=0.273723 brier=0.037987 ece10=0.032790 rho=0.000000 donor_rows=0 effective_n=4117.000000 seconds=1.142803 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator in_sample_bce=0.018525 local_bce=0.018525 donor_bce=nan in_sample_local_ap=0.990886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 123} positive_ess=123.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan rows=768 failures=30 prevalence=0.039062 ap=0.153520 auc=0.803930 f2_at20=0.273723 brier=0.037273 ece10=0.031914 rho=0.250000 donor_rows=4095 effective_n=6555.860305 seconds=1.618556 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan in_sample_bce=0.027360 local_bce=0.024221 donor_bce=0.039403 in_sample_local_ap=0.974664 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_duc': 0.10577177685504496, 'khoa_tieng_trung_quoc': 0.07998951818120807, 'khoa_tieng_nga': 0.06423870496374695} source_failures={'khoa_tieng_bo_dao_nha': 123, 'khoa_tieng_duc': 24, 'khoa_tieng_trung_quoc': 10, 'khoa_tieng_nga': 58} positive_ess=173.346034 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_bo_dao_nha model=committee_reference ap=0.314553 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_bo_dao_nha models=13 positive_rows=30 num_edges=52 num_replays=2 chosen=['khoa_tieng_duc', 'khoa_tieng_trung_quoc', 'khoa_tieng_nga'] [SELECTION_ORIGINS] target=khoa_tieng_duc fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_duc fold=1 origin=6060 ap=0.098272 rows=1871 failures=44 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=-0.008692 ap=0.089580 base_ap=0.098272 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.005784 ap=0.104056 base_ap=0.098272 donor_failures=92 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6060 delta_ap=-0.025106 ap=0.073166 base_ap=0.098272 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_anh origin=6060 delta_ap=0.000944 ap=0.099216 base_ap=0.098272 donor_failures=34 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.088382 ap=0.186654 base_ap=0.098272 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6060 delta_ap=0.012812 ap=0.111084 base_ap=0.098272 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_italia origin=6060 delta_ap=0.037984 ap=0.136256 base_ap=0.098272 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nga origin=6060 delta_ap=-0.018310 ap=0.079961 base_ap=0.098272 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6060 delta_ap=0.005067 ap=0.103339 base_ap=0.098272 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_phap origin=6060 delta_ap=-0.043944 ap=0.054328 base_ap=0.098272 donor_failures=74 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.020892 ap=0.119163 base_ap=0.098272 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6060 delta_ap=-0.019903 ap=0.078369 base_ap=0.098272 donor_failures=40 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6060 delta_ap=0.017074 ap=0.115346 base_ap=0.098272 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_duc fold=1 origin=6061 ap=0.089258 rows=1836 failures=10 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.014602 ap=0.074656 base_ap=0.089258 donor_failures=185 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.011391 ap=0.077867 base_ap=0.089258 donor_failures=109 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6061 delta_ap=0.098260 ap=0.187517 base_ap=0.089258 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_anh origin=6061 delta_ap=-0.018687 ap=0.070571 base_ap=0.089258 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.011099 ap=0.078158 base_ap=0.089258 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6061 delta_ap=0.025326 ap=0.114584 base_ap=0.089258 donor_failures=25 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_italia origin=6061 delta_ap=0.079232 ap=0.168490 base_ap=0.089258 donor_failures=159 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nga origin=6061 delta_ap=0.028879 ap=0.118137 base_ap=0.089258 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6061 delta_ap=0.026170 ap=0.115428 base_ap=0.089258 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_phap origin=6061 delta_ap=0.096328 ap=0.185585 base_ap=0.089258 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.016607 ap=0.105864 base_ap=0.089258 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.028526 ap=0.117784 base_ap=0.089258 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.007247 ap=0.082011 base_ap=0.089258 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_duc fold=1 origin=6063 ap=0.249342 rows=2282 failures=19 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.084159 ap=0.333501 base_ap=0.249342 donor_failures=156 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.131210 ap=0.118132 base_ap=0.249342 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.062423 ap=0.311766 base_ap=0.249342 donor_failures=40 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_anh origin=6063 delta_ap=0.094182 ap=0.343524 base_ap=0.249342 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.021666 ap=0.227676 base_ap=0.249342 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.135092 ap=0.384434 base_ap=0.249342 donor_failures=26 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_italia origin=6063 delta_ap=-0.082169 ap=0.167173 base_ap=0.249342 donor_failures=144 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nga origin=6063 delta_ap=0.099141 ap=0.348483 base_ap=0.249342 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6063 delta_ap=-0.111231 ap=0.138111 base_ap=0.249342 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_phap origin=6063 delta_ap=-0.018845 ap=0.230497 base_ap=0.249342 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.157215 ap=0.406557 base_ap=0.249342 donor_failures=90 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.090290 ap=0.339632 base_ap=0.249342 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.129027 ap=0.378369 base_ap=0.249342 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_duc fold=1 origin=6064 ap=0.375846 rows=1984 failures=53 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.029534 ap=0.346312 base_ap=0.375846 donor_failures=146 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.010281 ap=0.386127 base_ap=0.375846 donor_failures=90 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.038703 ap=0.337143 base_ap=0.375846 donor_failures=23 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_anh origin=6064 delta_ap=0.028570 ap=0.404416 base_ap=0.375846 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.038959 ap=0.414805 base_ap=0.375846 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.029762 ap=0.346084 base_ap=0.375846 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_italia origin=6064 delta_ap=-0.013641 ap=0.362205 base_ap=0.375846 donor_failures=126 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nga origin=6064 delta_ap=-0.003297 ap=0.372549 base_ap=0.375846 donor_failures=149 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6064 delta_ap=0.026464 ap=0.402310 base_ap=0.375846 donor_failures=39 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_phap origin=6064 delta_ap=0.006569 ap=0.382415 base_ap=0.375846 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.054344 ap=0.321502 base_ap=0.375846 donor_failures=81 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.033805 ap=0.342041 base_ap=0.375846 donor_failures=24 [TRANSFER_EDGE] fold=1 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.024918 ap=0.350929 base_ap=0.375846 donor_failures=265 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_duc origin=6063 policy_sources=khoa_tieng_italia,khoa_tieng_bo_dao_nha,khoa_quoc_te_hoc policy_donor_count=3 base_ap=0.249342 policy_ap=0.425767 delta_ap=0.176425 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_duc origin=6064 policy_sources=khoa_tieng_tay_ban_nha,khoa_tieng_han_quoc,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.375846 policy_ap=0.330878 delta_ap=-0.044968 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_duc selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.065728 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=hgb_local_common rows=1900 failures=46 prevalence=0.024211 ap=0.145798 auc=0.848917 f2_at20=0.265957 brier=0.023689 ece10=0.021356 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=0.580221 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=hgb_local_common in_sample_bce=0.014081 local_bce=0.014081 donor_bce=nan in_sample_local_ap=0.963634 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=hgb_local_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=hgb_local_native rows=1900 failures=46 prevalence=0.024211 ap=0.148353 auc=0.862307 f2_at20=0.274823 brier=0.023643 ece10=0.021272 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=1.014597 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=hgb_local_native in_sample_bce=0.013844 local_bce=0.013844 donor_bce=nan in_sample_local_ap=0.966357 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=hgb_local_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=hgb_pool_all rows=1900 failures=46 prevalence=0.024211 ap=0.159042 auc=0.860009 f2_at20=0.319149 brier=0.023433 ece10=0.021454 rho=0.250000 donor_rows=4095 effective_n=13386.878141 seconds=0.582362 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=hgb_pool_all in_sample_bce=0.020600 local_bce=0.018610 donor_bce=0.046356 in_sample_local_ap=0.915970 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=hgb_pool_all mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.020769053348941854, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03639534529503153, 'khoa_quoc_te_hoc': 0.011357158088507282, 'khoa_tieng_anh': 0.0522185570240525, 'khoa_tieng_bo_dao_nha': 0.004479109095735653, 'khoa_tieng_han_quoc': 0.01860619960050307, 'khoa_tieng_italia': 0.011586716509493491, 'khoa_tieng_nga': 0.007189204008895117, 'khoa_tieng_nhat': 0.020717919482655826, 'khoa_tieng_phap': 0.016607626996940673, 'khoa_tieng_tay_ban_nha': 0.008730835679688758, 'khoa_tieng_trung_quoc': 0.03566423980260152, 'khoa_viet_nam_hoc': 0.005678035066952726} source_failures={'khoa_tieng_duc': 172, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 23} positive_ess=226.879943 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=hgb_pool_equal rows=1900 failures=46 prevalence=0.024211 ap=0.188882 auc=0.856292 f2_at20=0.310284 brier=0.023173 ece10=0.020290 rho=0.250000 donor_rows=4095 effective_n=14936.825171 seconds=0.637208 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=hgb_pool_equal in_sample_bce=0.023788 local_bce=0.019344 donor_bce=0.037118 in_sample_local_ap=0.910888 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=hgb_pool_equal mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_duc': 172, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 23} positive_ess=270.111816 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=hgb_language_prior rows=1900 failures=46 prevalence=0.024211 ap=0.179506 auc=0.875833 f2_at20=0.319149 brier=0.023260 ece10=0.020083 rho=0.250000 donor_rows=4095 effective_n=14936.825171 seconds=0.609627 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=hgb_language_prior in_sample_bce=0.021129 local_bce=0.018346 donor_bce=0.029480 in_sample_local_ap=0.922731 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=hgb_language_prior mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_duc': 172, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 16, 'khoa_tieng_nga': 19, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1} positive_ess=246.272237 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=hgb_random_donor rows=1900 failures=46 prevalence=0.024211 ap=0.198266 auc=0.871459 f2_at20=0.301418 brier=0.022844 ece10=0.019937 rho=0.250000 donor_rows=4096 effective_n=14937.656565 seconds=0.603965 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=hgb_random_donor in_sample_bce=0.019143 local_bce=0.018301 donor_bce=0.021668 in_sample_local_ap=0.918055 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=hgb_random_donor mass={'khoa_tieng_duc': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.25} source_failures={'khoa_tieng_duc': 172, np.str_('khoa_tieng_tay_ban_nha'): 88} positive_ess=259.179013 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=suite32_selective_common rows=1900 failures=46 prevalence=0.024211 ap=0.145798 auc=0.848917 f2_at20=0.265957 brier=0.023689 ece10=0.021356 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=0.507735 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=suite32_selective_common in_sample_bce=0.014081 local_bce=0.014081 donor_bce=nan in_sample_local_ap=0.963634 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=suite32_selective_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=suite32_selective_native rows=1900 failures=46 prevalence=0.024211 ap=0.148353 auc=0.862307 f2_at20=0.274823 brier=0.023643 ece10=0.021272 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=0.949974 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=suite32_selective_native in_sample_bce=0.013844 local_bce=0.013844 donor_bce=nan in_sample_local_ap=0.966357 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=suite32_selective_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=native_cache_nan_indicator rows=1900 failures=46 prevalence=0.024211 ap=0.148353 auc=0.862307 f2_at20=0.274823 brier=0.023643 ece10=0.021272 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=1.668437 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=native_cache_nan_indicator in_sample_bce=0.013844 local_bce=0.013844 donor_bce=nan in_sample_local_ap=0.966357 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=native_cache_nan_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=native_naive_zero_indicator rows=1900 failures=46 prevalence=0.024211 ap=0.145945 auc=0.867091 f2_at20=0.283688 brier=0.023694 ece10=0.021409 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=1.658375 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=native_naive_zero_indicator in_sample_bce=0.014004 local_bce=0.014004 donor_bce=nan in_sample_local_ap=0.962515 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=native_naive_zero_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=native_placebo_indicator rows=1900 failures=46 prevalence=0.024211 ap=0.148353 auc=0.862307 f2_at20=0.274823 brier=0.023643 ece10=0.021272 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=1.846822 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=native_placebo_indicator in_sample_bce=0.013844 local_bce=0.013844 donor_bce=nan in_sample_local_ap=0.966357 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=native_placebo_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan rows=1900 failures=46 prevalence=0.024211 ap=0.148353 auc=0.862307 f2_at20=0.274823 brier=0.023643 ece10=0.021272 rho=0.000000 donor_rows=0 effective_n=10883.000000 seconds=1.606392 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan in_sample_bce=0.013844 local_bce=0.013844 donor_bce=nan in_sample_local_ap=0.966357 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 172} positive_ess=172.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_duc model=committee_reference ap=0.262205 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_duc models=13 positive_rows=46 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_han_quoc fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_han_quoc fold=1 origin=6060 ap=0.067894 rows=3087 failures=17 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.007421 ap=0.075315 base_ap=0.067894 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=-0.000058 ap=0.067837 base_ap=0.067894 donor_failures=87 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6060 delta_ap=-0.016114 ap=0.051781 base_ap=0.067894 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6060 delta_ap=-0.001982 ap=0.065912 base_ap=0.067894 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.015401 ap=0.083296 base_ap=0.067894 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6060 delta_ap=-0.028676 ap=0.039219 base_ap=0.067894 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6060 delta_ap=0.010178 ap=0.078072 base_ap=0.067894 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6060 delta_ap=-0.020688 ap=0.047207 base_ap=0.067894 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6060 delta_ap=-0.008643 ap=0.059252 base_ap=0.067894 donor_failures=69 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6060 delta_ap=-0.004791 ap=0.063103 base_ap=0.067894 donor_failures=71 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.047259 ap=0.115153 base_ap=0.067894 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6060 delta_ap=0.000392 ap=0.068287 base_ap=0.067894 donor_failures=38 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6060 delta_ap=-0.031006 ap=0.036888 base_ap=0.067894 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=1 origin=6061 ap=0.178374 rows=2789 failures=5 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.048790 ap=0.227164 base_ap=0.178374 donor_failures=202 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.132087 ap=0.310462 base_ap=0.178374 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.058674 ap=0.119701 base_ap=0.178374 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6061 delta_ap=-0.039389 ap=0.138985 base_ap=0.178374 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.042274 ap=0.220648 base_ap=0.178374 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6061 delta_ap=-0.002434 ap=0.175940 base_ap=0.178374 donor_failures=72 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6061 delta_ap=-0.080322 ap=0.098052 base_ap=0.178374 donor_failures=151 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6061 delta_ap=-0.120355 ap=0.058019 base_ap=0.178374 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6061 delta_ap=-0.058444 ap=0.119930 base_ap=0.178374 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6061 delta_ap=0.080001 ap=0.258375 base_ap=0.178374 donor_failures=59 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.060499 ap=0.117876 base_ap=0.178374 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.058186 ap=0.236560 base_ap=0.178374 donor_failures=26 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6061 delta_ap=0.063442 ap=0.241817 base_ap=0.178374 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=1 origin=6063 ap=0.008586 rows=3094 failures=9 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.008684 ap=0.017270 base_ap=0.008586 donor_failures=173 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.001246 ap=0.009832 base_ap=0.008586 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.014814 ap=0.023401 base_ap=0.008586 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6063 delta_ap=0.008348 ap=0.016934 base_ap=0.008586 donor_failures=81 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.015006 ap=0.023592 base_ap=0.008586 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6063 delta_ap=0.012650 ap=0.021236 base_ap=0.008586 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6063 delta_ap=0.000176 ap=0.008762 base_ap=0.008586 donor_failures=156 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6063 delta_ap=0.007471 ap=0.016058 base_ap=0.008586 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6063 delta_ap=0.018542 ap=0.027128 base_ap=0.008586 donor_failures=50 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6063 delta_ap=0.003119 ap=0.011705 base_ap=0.008586 donor_failures=67 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.004740 ap=0.013326 base_ap=0.008586 donor_failures=88 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.020326 ap=0.028912 base_ap=0.008586 donor_failures=35 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.009008 ap=0.017594 base_ap=0.008586 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=1 origin=6064 ap=0.054310 rows=3265 failures=12 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.015323 ap=0.038987 base_ap=0.054310 donor_failures=134 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.021736 ap=0.032574 base_ap=0.054310 donor_failures=104 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.021557 ap=0.032754 base_ap=0.054310 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6064 delta_ap=0.007372 ap=0.061682 base_ap=0.054310 donor_failures=59 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.064528 ap=0.118838 base_ap=0.054310 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6064 delta_ap=0.010293 ap=0.064604 base_ap=0.054310 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6064 delta_ap=-0.014647 ap=0.039664 base_ap=0.054310 donor_failures=121 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6064 delta_ap=0.010883 ap=0.065194 base_ap=0.054310 donor_failures=146 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.047796 ap=0.102106 base_ap=0.054310 donor_failures=48 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.023043 ap=0.031267 base_ap=0.054310 donor_failures=37 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.023009 ap=0.031301 base_ap=0.054310 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.093487 ap=0.147797 base_ap=0.054310 donor_failures=21 [TRANSFER_EDGE] fold=1 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.018752 ap=0.035558 base_ap=0.054310 donor_failures=269 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_han_quoc origin=6063 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_phap,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.008586 policy_ap=0.029037 delta_ap=0.020451 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_han_quoc origin=6064 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_trung_quoc,khoa_tieng_phap policy_donor_count=3 base_ap=0.054310 policy_ap=0.029175 delta_ap=-0.025135 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_han_quoc selected=[] reason=prequential_policy_delta_-0.002342_below_threshold replay_mean=-0.002342 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_local_common rows=2904 failures=20 prevalence=0.006887 ap=0.033364 auc=0.818811 f2_at20=0.090772 brier=0.006898 ece10=0.006558 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=0.586886 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=hgb_local_common in_sample_bce=0.002840 local_bce=0.002840 donor_bce=nan in_sample_local_ap=0.991436 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_local_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_local_native rows=2904 failures=20 prevalence=0.006887 ap=0.031520 auc=0.804438 f2_at20=0.083207 brier=0.006936 ece10=0.006819 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=0.952858 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=hgb_local_native in_sample_bce=0.002708 local_bce=0.002708 donor_bce=nan in_sample_local_ap=0.993382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_local_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_pool_all rows=2904 failures=20 prevalence=0.006887 ap=0.048902 auc=0.874593 f2_at20=0.113464 brier=0.006833 ece10=0.005917 rho=0.250000 donor_rows=4095 effective_n=17695.330068 seconds=0.630615 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=hgb_pool_all in_sample_bce=0.010637 local_bce=0.006333 donor_bce=0.041049 in_sample_local_ap=0.886738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_pool_all mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.021346781768573603, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.037407747014357916, 'khoa_quoc_te_hoc': 0.01167307778324462, 'khoa_tieng_anh': 0.05367110971955092, 'khoa_tieng_bo_dao_nha': 0.004603703538041777, 'khoa_tieng_duc': 0.012169566578700183, 'khoa_tieng_italia': 0.011909021782886792, 'khoa_tieng_nga': 0.007389184595428725, 'khoa_tieng_nhat': 0.021294225522207808, 'khoa_tieng_phap': 0.0170695978888044, 'khoa_tieng_tay_ban_nha': 0.008973699512456948, 'khoa_tieng_trung_quoc': 0.03665630451312788, 'khoa_viet_nam_hoc': 0.0058359797826184195} source_failures={'khoa_tieng_han_quoc': 75, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 14, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 19} positive_ess=120.226348 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_pool_equal rows=2904 failures=20 prevalence=0.006887 ap=0.040266 auc=0.872347 f2_at20=0.128593 brier=0.006838 ece10=0.005673 rho=0.250000 donor_rows=4095 effective_n=20766.963323 seconds=0.627422 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=hgb_pool_equal in_sample_bce=0.012340 local_bce=0.006802 donor_bce=0.028955 in_sample_local_ap=0.870232 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_pool_equal mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_han_quoc': 75, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 14, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 19} positive_ess=181.588156 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_language_prior rows=2904 failures=20 prevalence=0.006887 ap=0.083395 auc=0.865352 f2_at20=0.121029 brier=0.006765 ece10=0.005911 rho=0.250000 donor_rows=4095 effective_n=20766.963323 seconds=0.643216 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=hgb_language_prior in_sample_bce=0.010068 local_bce=0.006430 donor_bce=0.020985 in_sample_local_ap=0.882133 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_language_prior mass={'khoa_tieng_han_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_han_quoc': 75, 'khoa_tieng_anh': 13, 'khoa_tieng_bo_dao_nha': 19, 'khoa_tieng_duc': 6, 'khoa_tieng_italia': 19, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2} positive_ess=165.772234 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_random_donor rows=2904 failures=20 prevalence=0.006887 ap=0.044803 auc=0.873630 f2_at20=0.128593 brier=0.006842 ece10=0.006559 rho=0.250000 donor_rows=4096 effective_n=20768.570433 seconds=0.619828 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=hgb_random_donor in_sample_bce=0.005551 local_bce=0.004857 donor_bce=0.007635 in_sample_local_ap=0.932088 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=hgb_random_donor mass={'khoa_tieng_han_quoc': 0.75, np.str_('khoa_tieng_phap'): 0.25} source_failures={'khoa_tieng_han_quoc': 75, np.str_('khoa_tieng_phap'): 47} positive_ess=118.741649 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_common rows=2904 failures=20 prevalence=0.006887 ap=0.033364 auc=0.818811 f2_at20=0.090772 brier=0.006898 ece10=0.006558 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=0.588547 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_common in_sample_bce=0.002840 local_bce=0.002840 donor_bce=nan in_sample_local_ap=0.991436 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_native rows=2904 failures=20 prevalence=0.006887 ap=0.031520 auc=0.804438 f2_at20=0.083207 brier=0.006936 ece10=0.006819 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=0.950492 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_native in_sample_bce=0.002708 local_bce=0.002708 donor_bce=nan in_sample_local_ap=0.993382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator rows=2904 failures=20 prevalence=0.006887 ap=0.031520 auc=0.804438 f2_at20=0.083207 brier=0.006936 ece10=0.006819 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=1.642765 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator in_sample_bce=0.002708 local_bce=0.002708 donor_bce=nan in_sample_local_ap=0.993382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator rows=2904 failures=20 prevalence=0.006887 ap=0.031520 auc=0.804438 f2_at20=0.083207 brier=0.006936 ece10=0.006819 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=1.828058 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator in_sample_bce=0.002708 local_bce=0.002708 donor_bce=nan in_sample_local_ap=0.993382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=native_placebo_indicator rows=2904 failures=20 prevalence=0.006887 ap=0.031520 auc=0.804438 f2_at20=0.083207 brier=0.006936 ece10=0.006819 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=1.732160 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=native_placebo_indicator in_sample_bce=0.002708 local_bce=0.002708 donor_bce=nan in_sample_local_ap=0.993382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=native_placebo_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan rows=2904 failures=20 prevalence=0.006887 ap=0.031520 auc=0.804438 f2_at20=0.083207 brier=0.006936 ece10=0.006819 rho=0.000000 donor_rows=0 effective_n=17102.000000 seconds=1.766837 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.002708 local_bce=0.002708 donor_bce=nan in_sample_local_ap=0.993382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 75} positive_ess=75.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_han_quoc model=committee_reference ap=0.054434 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_han_quoc models=13 positive_rows=20 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_italia fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_italia fold=1 origin=6060 ap=0.253540 rows=1927 failures=53 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.040128 ap=0.293668 base_ap=0.253540 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=-0.028369 ap=0.225172 base_ap=0.253540 donor_failures=78 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6060 delta_ap=0.049849 ap=0.303389 base_ap=0.253540 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_anh origin=6060 delta_ap=-0.003724 ap=0.249816 base_ap=0.253540 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=-0.032997 ap=0.220544 base_ap=0.253540 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_duc origin=6060 delta_ap=0.007514 ap=0.261054 base_ap=0.253540 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6060 delta_ap=0.013336 ap=0.266877 base_ap=0.253540 donor_failures=26 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nga origin=6060 delta_ap=0.086021 ap=0.339562 base_ap=0.253540 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6060 delta_ap=0.038215 ap=0.291756 base_ap=0.253540 donor_failures=69 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_phap origin=6060 delta_ap=0.018823 ap=0.272364 base_ap=0.253540 donor_failures=74 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=-0.000423 ap=0.253118 base_ap=0.253540 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6060 delta_ap=0.003253 ap=0.256793 base_ap=0.253540 donor_failures=41 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6060 delta_ap=0.016171 ap=0.269712 base_ap=0.253540 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_italia fold=1 origin=6061 ap=0.588367 rows=1680 failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.045065 ap=0.543302 base_ap=0.588367 donor_failures=194 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.021085 ap=0.567281 base_ap=0.588367 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.057941 ap=0.530425 base_ap=0.588367 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_anh origin=6061 delta_ap=-0.026190 ap=0.562177 base_ap=0.588367 donor_failures=52 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.012295 ap=0.576071 base_ap=0.588367 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_duc origin=6061 delta_ap=-0.042989 ap=0.545377 base_ap=0.588367 donor_failures=76 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.029184 ap=0.559182 base_ap=0.588367 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nga origin=6061 delta_ap=-0.010231 ap=0.578135 base_ap=0.588367 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6061 delta_ap=-0.104674 ap=0.483693 base_ap=0.588367 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_phap origin=6061 delta_ap=0.020232 ap=0.608598 base_ap=0.588367 donor_failures=68 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.018020 ap=0.606386 base_ap=0.588367 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6061 delta_ap=-0.013704 ap=0.574663 base_ap=0.588367 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.031007 ap=0.557359 base_ap=0.588367 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_italia fold=1 origin=6063 ap=0.108261 rows=2287 failures=35 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.000976 ap=0.107285 base_ap=0.108261 donor_failures=158 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.045521 ap=0.062740 base_ap=0.108261 donor_failures=95 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6063 delta_ap=0.018854 ap=0.127115 base_ap=0.108261 donor_failures=35 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_anh origin=6063 delta_ap=-0.044888 ap=0.063374 base_ap=0.108261 donor_failures=68 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.047258 ap=0.061003 base_ap=0.108261 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_duc origin=6063 delta_ap=0.000250 ap=0.108511 base_ap=0.108261 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.057540 ap=0.050721 base_ap=0.108261 donor_failures=23 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nga origin=6063 delta_ap=-0.049225 ap=0.059036 base_ap=0.108261 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6063 delta_ap=-0.033087 ap=0.075175 base_ap=0.108261 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_phap origin=6063 delta_ap=0.025887 ap=0.134148 base_ap=0.108261 donor_failures=57 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.015813 ap=0.092449 base_ap=0.108261 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.016782 ap=0.091479 base_ap=0.108261 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.040201 ap=0.068060 base_ap=0.108261 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_italia fold=1 origin=6064 ap=0.292166 rows=2017 failures=65 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.039126 ap=0.331292 base_ap=0.292166 donor_failures=113 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.025269 ap=0.317435 base_ap=0.292166 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6064 delta_ap=0.018972 ap=0.311137 base_ap=0.292166 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_anh origin=6064 delta_ap=-0.016370 ap=0.275796 base_ap=0.292166 donor_failures=59 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.042129 ap=0.334295 base_ap=0.292166 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_duc origin=6064 delta_ap=0.032040 ap=0.324206 base_ap=0.292166 donor_failures=48 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6064 delta_ap=0.046882 ap=0.339048 base_ap=0.292166 donor_failures=17 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nga origin=6064 delta_ap=0.086354 ap=0.378520 base_ap=0.292166 donor_failures=149 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6064 delta_ap=0.011886 ap=0.304052 base_ap=0.292166 donor_failures=41 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_phap origin=6064 delta_ap=-0.017958 ap=0.274208 base_ap=0.292166 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.056385 ap=0.348551 base_ap=0.292166 donor_failures=69 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.022955 ap=0.269211 base_ap=0.292166 donor_failures=19 [TRANSFER_EDGE] fold=1 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6064 delta_ap=0.024709 ap=0.316875 base_ap=0.292166 donor_failures=275 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_italia origin=6063 policy_sources=khoa_tieng_nga,khoa_tieng_phap,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.108261 policy_ap=0.069335 delta_ap=-0.038927 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_italia origin=6064 policy_sources=khoa_tieng_phap,khoa_quoc_te_hoc policy_donor_count=2 base_ap=0.292166 policy_ap=0.314184 delta_ap=0.022019 policy_donor_rows=4096 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_italia selected=[] reason=prequential_policy_delta_-0.008454_below_threshold replay_mean=-0.008454 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=hgb_local_common rows=2166 failures=111 prevalence=0.051247 ap=0.198599 auc=0.763493 f2_at20=0.398633 brier=0.049180 ece10=0.045050 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=0.521951 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=hgb_local_common in_sample_bce=0.031252 local_bce=0.031252 donor_bce=nan in_sample_local_ap=0.926227 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=hgb_local_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=hgb_local_native rows=2166 failures=111 prevalence=0.051247 ap=0.261609 auc=0.803687 f2_at20=0.415718 brier=0.048910 ece10=0.045296 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=1.096052 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=hgb_local_native in_sample_bce=0.030548 local_bce=0.030548 donor_bce=nan in_sample_local_ap=0.932553 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=hgb_local_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=hgb_pool_all rows=2166 failures=111 prevalence=0.051247 ap=0.236899 auc=0.799362 f2_at20=0.404328 brier=0.048702 ece10=0.044521 rho=0.250000 donor_rows=4095 effective_n=13192.087421 seconds=0.622837 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=hgb_pool_all in_sample_bce=0.035726 local_bce=0.036607 donor_bce=0.053766 in_sample_local_ap=0.895669 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=hgb_pool_all mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.020748015407221918, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03635847875420612, 'khoa_quoc_te_hoc': 0.01134565389397536, 'khoa_tieng_anh': 0.05216566241489944, 'khoa_tieng_bo_dao_nha': 0.004474571997461112, 'khoa_tieng_duc': 0.011828216431471772, 'khoa_tieng_han_quoc': 0.018587352514107345, 'khoa_tieng_nga': 0.0071819217292259, 'khoa_tieng_nhat': 0.02069693333681126, 'khoa_tieng_phap': 0.016590804357844033, 'khoa_tieng_tay_ban_nha': 0.008721991809479093, 'khoa_tieng_trung_quoc': 0.035628113832589924, 'khoa_viet_nam_hoc': 0.005672283520706715} source_failures={'khoa_tieng_italia': 326, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_nga': 10, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 25} positive_ess=377.695550 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=hgb_pool_equal rows=2166 failures=111 prevalence=0.051247 ap=0.271242 auc=0.839331 f2_at20=0.427107 brier=0.048290 ece10=0.043755 rho=0.250000 donor_rows=4095 effective_n=14688.727502 seconds=0.574841 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=hgb_pool_equal in_sample_bce=0.039596 local_bce=0.038213 donor_bce=0.043744 in_sample_local_ap=0.890027 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=hgb_pool_equal mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_italia': 326, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_nga': 10, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 25} positive_ess=422.584048 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=hgb_language_prior rows=2166 failures=111 prevalence=0.051247 ap=0.286762 auc=0.855071 f2_at20=0.444191 brier=0.048515 ece10=0.044657 rho=0.250000 donor_rows=4095 effective_n=14688.727502 seconds=0.562753 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=hgb_language_prior in_sample_bce=0.034478 local_bce=0.036757 donor_bce=0.027644 in_sample_local_ap=0.894722 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=hgb_language_prior mass={'khoa_tieng_italia': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_italia': 326, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 1} positive_ess=397.893228 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=hgb_random_donor rows=2166 failures=111 prevalence=0.051247 ap=0.197135 auc=0.821218 f2_at20=0.415718 brier=0.048093 ece10=0.040159 rho=0.250000 donor_rows=4096 effective_n=14689.531507 seconds=0.550671 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=hgb_random_donor in_sample_bce=0.029971 local_bce=0.035086 donor_bce=0.014625 in_sample_local_ap=0.906769 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=hgb_random_donor mass={'khoa_tieng_italia': 0.75, np.str_('khoa_tieng_nhat'): 0.25} source_failures={'khoa_tieng_italia': 326, np.str_('khoa_tieng_nhat'): 44} positive_ess=369.290130 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=suite32_selective_common rows=2166 failures=111 prevalence=0.051247 ap=0.198599 auc=0.763493 f2_at20=0.398633 brier=0.049180 ece10=0.045050 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=0.523620 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=suite32_selective_common in_sample_bce=0.031252 local_bce=0.031252 donor_bce=nan in_sample_local_ap=0.926227 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=suite32_selective_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=suite32_selective_native rows=2166 failures=111 prevalence=0.051247 ap=0.261609 auc=0.803687 f2_at20=0.415718 brier=0.048910 ece10=0.045296 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=1.051296 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=suite32_selective_native in_sample_bce=0.030548 local_bce=0.030548 donor_bce=nan in_sample_local_ap=0.932553 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=suite32_selective_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=native_cache_nan_indicator rows=2166 failures=111 prevalence=0.051247 ap=0.261609 auc=0.803687 f2_at20=0.415718 brier=0.048910 ece10=0.045296 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=1.685232 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=native_cache_nan_indicator in_sample_bce=0.030548 local_bce=0.030548 donor_bce=nan in_sample_local_ap=0.932553 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=native_cache_nan_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=native_naive_zero_indicator rows=2166 failures=111 prevalence=0.051247 ap=0.268723 auc=0.795134 f2_at20=0.398633 brier=0.048704 ece10=0.045438 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=1.648472 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=native_naive_zero_indicator in_sample_bce=0.030627 local_bce=0.030627 donor_bce=nan in_sample_local_ap=0.932786 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=native_naive_zero_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=native_placebo_indicator rows=2166 failures=111 prevalence=0.051247 ap=0.261609 auc=0.803687 f2_at20=0.415718 brier=0.048910 ece10=0.045296 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=1.857007 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=native_placebo_indicator in_sample_bce=0.030548 local_bce=0.030548 donor_bce=nan in_sample_local_ap=0.932553 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=native_placebo_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan rows=2166 failures=111 prevalence=0.051247 ap=0.261609 auc=0.803687 f2_at20=0.415718 brier=0.048910 ece10=0.045296 rho=0.000000 donor_rows=0 effective_n=10650.000000 seconds=1.693003 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan in_sample_bce=0.030548 local_bce=0.030548 donor_bce=nan in_sample_local_ap=0.932553 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 326} positive_ess=326.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_italia model=committee_reference ap=0.259779 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_italia models=13 positive_rows=111 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nga fold=1 origins=[6061, 6063, 6064, 6065] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nga fold=1 origin=6061 ap=0.514711 rows=1084 failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.039032 ap=0.475679 base_ap=0.514711 donor_failures=203 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.002572 ap=0.512138 base_ap=0.514711 donor_failures=105 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6061 delta_ap=0.005832 ap=0.520543 base_ap=0.514711 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_anh origin=6061 delta_ap=-0.019829 ap=0.494881 base_ap=0.514711 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.042915 ap=0.471795 base_ap=0.514711 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_duc origin=6061 delta_ap=-0.017759 ap=0.496951 base_ap=0.514711 donor_failures=72 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.021612 ap=0.493099 base_ap=0.514711 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_italia origin=6061 delta_ap=0.017829 ap=0.532540 base_ap=0.514711 donor_failures=161 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6061 delta_ap=0.008371 ap=0.523082 base_ap=0.514711 donor_failures=62 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_phap origin=6061 delta_ap=-0.027052 ap=0.487659 base_ap=0.514711 donor_failures=59 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.029909 ap=0.484802 base_ap=0.514711 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6061 delta_ap=-0.009668 ap=0.505043 base_ap=0.514711 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6061 delta_ap=0.022317 ap=0.537027 base_ap=0.514711 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_nga fold=1 origin=6063 ap=0.096820 rows=1460 failures=17 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.023081 ap=0.119902 base_ap=0.096820 donor_failures=153 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.024910 ap=0.121730 base_ap=0.096820 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6063 delta_ap=0.031866 ap=0.128686 base_ap=0.096820 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_anh origin=6063 delta_ap=-0.013947 ap=0.082874 base_ap=0.096820 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.000304 ap=0.096516 base_ap=0.096820 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_duc origin=6063 delta_ap=0.048517 ap=0.145338 base_ap=0.096820 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.001328 ap=0.095493 base_ap=0.096820 donor_failures=13 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_italia origin=6063 delta_ap=0.016774 ap=0.113594 base_ap=0.096820 donor_failures=139 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6063 delta_ap=0.011497 ap=0.108318 base_ap=0.096820 donor_failures=57 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_phap origin=6063 delta_ap=0.034569 ap=0.131390 base_ap=0.096820 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.015290 ap=0.112110 base_ap=0.096820 donor_failures=78 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.008091 ap=0.104911 base_ap=0.096820 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.010352 ap=0.086468 base_ap=0.096820 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_nga fold=1 origin=6064 ap=0.362097 rows=1355 failures=78 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.002909 ap=0.365006 base_ap=0.362097 donor_failures=120 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.023959 ap=0.386055 base_ap=0.362097 donor_failures=90 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.022226 ap=0.339871 base_ap=0.362097 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_anh origin=6064 delta_ap=-0.016237 ap=0.345860 base_ap=0.362097 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.040495 ap=0.321602 base_ap=0.362097 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_duc origin=6064 delta_ap=-0.033415 ap=0.328681 base_ap=0.362097 donor_failures=55 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.003123 ap=0.358974 base_ap=0.362097 donor_failures=25 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_italia origin=6064 delta_ap=-0.013472 ap=0.348625 base_ap=0.362097 donor_failures=125 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6064 delta_ap=-0.038140 ap=0.323957 base_ap=0.362097 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_phap origin=6064 delta_ap=0.029258 ap=0.391355 base_ap=0.362097 donor_failures=39 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.045437 ap=0.316660 base_ap=0.362097 donor_failures=73 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.000921 ap=0.363018 base_ap=0.362097 donor_failures=24 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.020987 ap=0.341110 base_ap=0.362097 donor_failures=271 [SCREEN_BASE] target=khoa_tieng_nga fold=1 origin=6065 ap=0.305556 rows=45 failures=3 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=0.025794 ap=0.331349 base_ap=0.305556 donor_failures=114 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6065 delta_ap=0.081349 ap=0.386905 base_ap=0.305556 donor_failures=112 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6065 delta_ap=0.000000 ap=0.305556 base_ap=0.305556 donor_failures=38 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_anh origin=6065 delta_ap=0.005051 ap=0.310606 base_ap=0.305556 donor_failures=59 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=-0.020370 ap=0.285185 base_ap=0.305556 donor_failures=122 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_duc origin=6065 delta_ap=0.055556 ap=0.361111 base_ap=0.305556 donor_failures=67 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6065 delta_ap=0.154762 ap=0.460317 base_ap=0.305556 donor_failures=16 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_italia origin=6065 delta_ap=0.050000 ap=0.355556 base_ap=0.305556 donor_failures=121 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6065 delta_ap=0.081349 ap=0.386905 base_ap=0.305556 donor_failures=45 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_phap origin=6065 delta_ap=0.081349 ap=0.386905 base_ap=0.305556 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=0.137302 ap=0.442857 base_ap=0.305556 donor_failures=81 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6065 delta_ap=0.055556 ap=0.361111 base_ap=0.305556 donor_failures=19 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6065 delta_ap=-0.036075 ap=0.269481 base_ap=0.305556 donor_failures=269 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_nga origin=6064 policy_sources=khoa_quoc_te_hoc,khoa_tieng_italia,khoa_tieng_duc policy_donor_count=3 base_ap=0.362097 policy_ap=0.346769 delta_ap=-0.015328 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_nga origin=6065 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_phap,khoa_tieng_italia policy_donor_count=3 base_ap=0.305556 policy_ap=0.555556 delta_ap=0.250000 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_nga selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.117336 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=hgb_local_common rows=1588 failures=102 prevalence=0.064232 ap=0.326633 auc=0.725975 f2_at20=0.378788 brier=0.056822 ece10=0.049920 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=0.508714 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=hgb_local_common in_sample_bce=0.038502 local_bce=0.038502 donor_bce=nan in_sample_local_ap=0.938946 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=hgb_local_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=hgb_local_native rows=1588 failures=102 prevalence=0.064232 ap=0.312251 auc=0.712328 f2_at20=0.378788 brier=0.058010 ece10=0.051530 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=1.018296 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=hgb_local_native in_sample_bce=0.038555 local_bce=0.038555 donor_bce=nan in_sample_local_ap=0.940738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=hgb_local_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=hgb_pool_all rows=1588 failures=102 prevalence=0.064232 ap=0.289837 auc=0.734918 f2_at20=0.385675 brier=0.058443 ece10=0.050137 rho=0.250000 donor_rows=4095 effective_n=9295.770366 seconds=0.624768 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=hgb_pool_all in_sample_bce=0.041984 local_bce=0.044018 donor_bce=0.048634 in_sample_local_ap=0.919970 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=hgb_pool_all mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.02038972246906829, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.035730612140269326, 'khoa_quoc_te_hoc': 0.011149728279444939, 'khoa_tieng_anh': 0.0512648250051268, 'khoa_tieng_bo_dao_nha': 0.004397301592726776, 'khoa_tieng_duc': 0.011623957550071775, 'khoa_tieng_han_quoc': 0.018266371590676053, 'khoa_tieng_italia': 0.011375093991386972, 'khoa_tieng_nhat': 0.020339522523754188, 'khoa_tieng_phap': 0.01630430138765466, 'khoa_tieng_tay_ban_nha': 0.008571373641397225, 'khoa_tieng_trung_quoc': 0.035012859730671954, 'khoa_viet_nam_hoc': 0.005574330097751042} source_failures={'khoa_tieng_nga': 270, 'khoa_cong_nghe_thong_tin': 6, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 7, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 17} positive_ess=312.949575 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=hgb_pool_equal rows=1588 failures=102 prevalence=0.064232 ap=0.301620 auc=0.752009 f2_at20=0.399449 brier=0.058149 ece10=0.051423 rho=0.250000 donor_rows=4095 effective_n=9961.488162 seconds=0.552632 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=hgb_pool_equal in_sample_bce=0.043443 local_bce=0.043003 donor_bce=0.044764 in_sample_local_ap=0.925906 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=hgb_pool_equal mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nga': 270, 'khoa_cong_nghe_thong_tin': 6, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 7, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 17} positive_ess=327.726116 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=hgb_language_prior rows=1588 failures=102 prevalence=0.064232 ap=0.354770 auc=0.794540 f2_at20=0.399449 brier=0.056505 ece10=0.050101 rho=0.250000 donor_rows=4095 effective_n=9961.488162 seconds=0.539508 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=hgb_language_prior in_sample_bce=0.042071 local_bce=0.042894 donor_bce=0.039603 in_sample_local_ap=0.924329 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=hgb_language_prior mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nga': 270, 'khoa_tieng_anh': 14, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 11, 'khoa_tieng_nhat': 8, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 4} positive_ess=328.542251 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=hgb_random_donor rows=1588 failures=102 prevalence=0.064232 ap=0.301842 auc=0.751765 f2_at20=0.378788 brier=0.057841 ece10=0.052731 rho=0.250000 donor_rows=4096 effective_n=9961.857932 seconds=0.530952 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=hgb_random_donor in_sample_bce=0.034489 local_bce=0.039544 donor_bce=0.019321 in_sample_local_ap=0.937031 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=hgb_random_donor mass={'khoa_tieng_nga': 0.75, np.str_('khoa_quoc_te_hoc'): 0.25} source_failures={'khoa_tieng_nga': 270, np.str_('khoa_quoc_te_hoc'): 27} positive_ess=291.393175 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=suite32_selective_common rows=1588 failures=102 prevalence=0.064232 ap=0.326633 auc=0.725975 f2_at20=0.378788 brier=0.056822 ece10=0.049920 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=0.426945 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=suite32_selective_common in_sample_bce=0.038502 local_bce=0.038502 donor_bce=nan in_sample_local_ap=0.938946 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=suite32_selective_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=suite32_selective_native rows=1588 failures=102 prevalence=0.064232 ap=0.312251 auc=0.712328 f2_at20=0.378788 brier=0.058010 ece10=0.051530 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=0.973672 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=suite32_selective_native in_sample_bce=0.038555 local_bce=0.038555 donor_bce=nan in_sample_local_ap=0.940738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=suite32_selective_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=native_cache_nan_indicator rows=1588 failures=102 prevalence=0.064232 ap=0.312251 auc=0.712328 f2_at20=0.378788 brier=0.058010 ece10=0.051530 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=1.740041 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=native_cache_nan_indicator in_sample_bce=0.038555 local_bce=0.038555 donor_bce=nan in_sample_local_ap=0.940738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=native_cache_nan_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=native_naive_zero_indicator rows=1588 failures=102 prevalence=0.064232 ap=0.312251 auc=0.712328 f2_at20=0.378788 brier=0.058010 ece10=0.051530 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=1.738620 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=native_naive_zero_indicator in_sample_bce=0.038555 local_bce=0.038555 donor_bce=nan in_sample_local_ap=0.940738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=native_naive_zero_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=native_placebo_indicator rows=1588 failures=102 prevalence=0.064232 ap=0.312251 auc=0.712328 f2_at20=0.378788 brier=0.058010 ece10=0.051530 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=1.780592 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=native_placebo_indicator in_sample_bce=0.038555 local_bce=0.038555 donor_bce=nan in_sample_local_ap=0.940738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=native_placebo_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan rows=1588 failures=102 prevalence=0.064232 ap=0.312251 auc=0.712328 f2_at20=0.378788 brier=0.058010 ece10=0.051530 rho=0.000000 donor_rows=0 effective_n=6608.000000 seconds=1.760161 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan in_sample_bce=0.038555 local_bce=0.038555 donor_bce=nan in_sample_local_ap=0.940738 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 270} positive_ess=270.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nga model=committee_reference ap=0.347619 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_nga models=13 positive_rows=102 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nhat fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nhat fold=1 origin=6060 ap=0.332093 rows=3349 failures=37 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.073312 ap=0.405405 base_ap=0.332093 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.206648 ap=0.538741 base_ap=0.332093 donor_failures=77 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6060 delta_ap=0.157903 ap=0.489996 base_ap=0.332093 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6060 delta_ap=0.047213 ap=0.379306 base_ap=0.332093 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.098936 ap=0.431029 base_ap=0.332093 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6060 delta_ap=-0.006803 ap=0.325290 base_ap=0.332093 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6060 delta_ap=0.077817 ap=0.409910 base_ap=0.332093 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6060 delta_ap=-0.010021 ap=0.322072 base_ap=0.332093 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6060 delta_ap=0.129103 ap=0.461196 base_ap=0.332093 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6060 delta_ap=0.001750 ap=0.333843 base_ap=0.332093 donor_failures=73 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=-0.028533 ap=0.303560 base_ap=0.332093 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6060 delta_ap=0.046801 ap=0.378894 base_ap=0.332093 donor_failures=41 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6060 delta_ap=0.184096 ap=0.516189 base_ap=0.332093 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_nhat fold=1 origin=6061 ap=0.290348 rows=3201 failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.026975 ap=0.317323 base_ap=0.290348 donor_failures=185 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.027223 ap=0.263125 base_ap=0.290348 donor_failures=92 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6061 delta_ap=0.028917 ap=0.319265 base_ap=0.290348 donor_failures=34 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6061 delta_ap=0.020635 ap=0.310983 base_ap=0.290348 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.092997 ap=0.383344 base_ap=0.290348 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6061 delta_ap=0.044269 ap=0.334617 base_ap=0.290348 donor_failures=78 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6061 delta_ap=0.048064 ap=0.338412 base_ap=0.290348 donor_failures=21 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6061 delta_ap=0.099887 ap=0.390234 base_ap=0.290348 donor_failures=151 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6061 delta_ap=0.028852 ap=0.319200 base_ap=0.290348 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6061 delta_ap=0.017172 ap=0.307520 base_ap=0.290348 donor_failures=64 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.069121 ap=0.359469 base_ap=0.290348 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6061 delta_ap=-0.032164 ap=0.258184 base_ap=0.290348 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6061 delta_ap=0.056863 ap=0.347211 base_ap=0.290348 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_nhat fold=1 origin=6063 ap=0.043009 rows=3170 failures=7 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.031931 ap=0.011078 base_ap=0.043009 donor_failures=180 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.018743 ap=0.024266 base_ap=0.043009 donor_failures=90 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.008865 ap=0.034144 base_ap=0.043009 donor_failures=37 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6063 delta_ap=0.035354 ap=0.078363 base_ap=0.043009 donor_failures=78 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.016588 ap=0.026421 base_ap=0.043009 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6063 delta_ap=0.000530 ap=0.043540 base_ap=0.043009 donor_failures=74 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6063 delta_ap=0.004294 ap=0.047303 base_ap=0.043009 donor_failures=15 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6063 delta_ap=-0.013448 ap=0.029562 base_ap=0.043009 donor_failures=148 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6063 delta_ap=-0.025582 ap=0.017427 base_ap=0.043009 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6063 delta_ap=-0.016027 ap=0.026982 base_ap=0.043009 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.032177 ap=0.010832 base_ap=0.043009 donor_failures=84 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.031552 ap=0.011457 base_ap=0.043009 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.028422 ap=0.014587 base_ap=0.043009 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_nhat fold=1 origin=6064 ap=0.195407 rows=2945 failures=19 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.107873 ap=0.087534 base_ap=0.195407 donor_failures=126 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.034677 ap=0.160730 base_ap=0.195407 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.113424 ap=0.081983 base_ap=0.195407 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6064 delta_ap=-0.150197 ap=0.045210 base_ap=0.195407 donor_failures=44 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.061264 ap=0.256671 base_ap=0.195407 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6064 delta_ap=0.030555 ap=0.225961 base_ap=0.195407 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.132764 ap=0.062643 base_ap=0.195407 donor_failures=23 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6064 delta_ap=-0.117131 ap=0.078275 base_ap=0.195407 donor_failures=129 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6064 delta_ap=-0.061978 ap=0.133429 base_ap=0.195407 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6064 delta_ap=-0.082300 ap=0.113107 base_ap=0.195407 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.090607 ap=0.104800 base_ap=0.195407 donor_failures=77 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.101971 ap=0.093435 base_ap=0.195407 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.144006 ap=0.051401 base_ap=0.195407 donor_failures=271 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_nhat origin=6063 policy_sources=khoa_viet_nam_hoc,khoa_tieng_bo_dao_nha,khoa_quoc_te_hoc policy_donor_count=3 base_ap=0.043009 policy_ap=0.024002 delta_ap=-0.019008 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_nhat origin=6064 policy_sources=khoa_viet_nam_hoc,khoa_quoc_te_hoc,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.195407 policy_ap=0.054869 delta_ap=-0.140538 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_nhat selected=[] reason=prequential_policy_delta_-0.079773_below_threshold replay_mean=-0.079773 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=hgb_local_common rows=2840 failures=17 prevalence=0.005986 ap=0.423571 auc=0.904461 f2_at20=0.102201 brier=0.005799 ece10=0.004758 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=0.617511 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=hgb_local_common in_sample_bce=0.007393 local_bce=0.007393 donor_bce=nan in_sample_local_ap=0.966885 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=hgb_local_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=hgb_local_native rows=2840 failures=17 prevalence=0.005986 ap=0.159987 auc=0.891125 f2_at20=0.086478 brier=0.005834 ece10=0.004514 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=1.120819 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=hgb_local_native in_sample_bce=0.007149 local_bce=0.007149 donor_bce=nan in_sample_local_ap=0.970238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=hgb_local_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=hgb_pool_all rows=2840 failures=17 prevalence=0.005986 ap=0.411719 auc=0.924892 f2_at20=0.125786 brier=0.005625 ece10=0.004150 rho=0.250000 donor_rows=4095 effective_n=18764.623655 seconds=0.644007 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=hgb_pool_all in_sample_bce=0.014410 local_bce=0.011445 donor_bce=0.042754 in_sample_local_ap=0.911881 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=hgb_pool_all mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.02153373430372379, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03773535954229817, 'khoa_quoc_te_hoc': 0.011775309187877038, 'khoa_tieng_anh': 0.05414115481277269, 'khoa_tieng_bo_dao_nha': 0.004644022217309107, 'khoa_tieng_duc': 0.012276146172206705, 'khoa_tieng_han_quoc': 0.019291247986499963, 'khoa_tieng_italia': 0.012013319556556226, 'khoa_tieng_nga': 0.0074538981811946995, 'khoa_tieng_phap': 0.017219091364397258, 'khoa_tieng_tay_ban_nha': 0.00905229008839096, 'khoa_tieng_trung_quoc': 0.03697733599844786, 'khoa_viet_nam_hoc': 0.005887090588325535} source_failures={'khoa_tieng_nhat': 199, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 7, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 10, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 20} positive_ess=215.789021 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=hgb_pool_equal rows=2840 failures=17 prevalence=0.005986 ap=0.514617 auc=0.930258 f2_at20=0.125786 brier=0.005108 ece10=0.003776 rho=0.250000 donor_rows=4095 effective_n=22320.966045 seconds=0.642713 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=hgb_pool_equal in_sample_bce=0.017032 local_bce=0.012593 donor_bce=0.030349 in_sample_local_ap=0.893440 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=hgb_pool_equal mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nhat': 199, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 7, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 10, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 20} positive_ess=286.229539 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=hgb_language_prior rows=2840 failures=17 prevalence=0.005986 ap=0.511597 auc=0.945083 f2_at20=0.133648 brier=0.005234 ece10=0.004059 rho=0.250000 donor_rows=4095 effective_n=22320.966045 seconds=0.625148 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=hgb_language_prior in_sample_bce=0.013729 local_bce=0.011359 donor_bce=0.020840 in_sample_local_ap=0.907995 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=hgb_language_prior mass={'khoa_tieng_nhat': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nhat': 199, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 17, 'khoa_tieng_nga': 16, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 1} positive_ess=260.258604 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=hgb_random_donor rows=2840 failures=17 prevalence=0.005986 ap=0.310181 auc=0.935686 f2_at20=0.125786 brier=0.005692 ece10=0.004179 rho=0.250000 donor_rows=4096 effective_n=22322.822688 seconds=0.642827 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=hgb_random_donor in_sample_bce=0.018039 local_bce=0.011938 donor_bce=0.036345 in_sample_local_ap=0.899379 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=hgb_random_donor mass={'khoa_tieng_nhat': 0.75, np.str_('khoa_tieng_italia'): 0.25} source_failures={'khoa_tieng_nhat': 199, np.str_('khoa_tieng_italia'): 129} positive_ess=312.753370 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_common rows=2840 failures=17 prevalence=0.005986 ap=0.423571 auc=0.904461 f2_at20=0.102201 brier=0.005799 ece10=0.004758 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=0.610246 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_common in_sample_bce=0.007393 local_bce=0.007393 donor_bce=nan in_sample_local_ap=0.966885 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_native rows=2840 failures=17 prevalence=0.005986 ap=0.159987 auc=0.891125 f2_at20=0.086478 brier=0.005834 ece10=0.004514 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=1.118208 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_native in_sample_bce=0.007149 local_bce=0.007149 donor_bce=nan in_sample_local_ap=0.970238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=native_cache_nan_indicator rows=2840 failures=17 prevalence=0.005986 ap=0.159987 auc=0.891125 f2_at20=0.086478 brier=0.005834 ece10=0.004514 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=1.870567 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=native_cache_nan_indicator in_sample_bce=0.007149 local_bce=0.007149 donor_bce=nan in_sample_local_ap=0.970238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=native_cache_nan_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=native_naive_zero_indicator rows=2840 failures=17 prevalence=0.005986 ap=0.159987 auc=0.891125 f2_at20=0.086478 brier=0.005834 ece10=0.004514 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=1.981052 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=native_naive_zero_indicator in_sample_bce=0.007149 local_bce=0.007149 donor_bce=nan in_sample_local_ap=0.970238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=native_naive_zero_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=native_placebo_indicator rows=2840 failures=17 prevalence=0.005986 ap=0.159987 auc=0.891125 f2_at20=0.086478 brier=0.005834 ece10=0.004514 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=1.864453 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=native_placebo_indicator in_sample_bce=0.007149 local_bce=0.007149 donor_bce=nan in_sample_local_ap=0.970238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=native_placebo_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan rows=2840 failures=17 prevalence=0.005986 ap=0.159987 auc=0.891125 f2_at20=0.086478 brier=0.005834 ece10=0.004514 rho=0.000000 donor_rows=0 effective_n=19043.000000 seconds=1.873359 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan in_sample_bce=0.007149 local_bce=0.007149 donor_bce=nan in_sample_local_ap=0.970238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 199} positive_ess=199.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_nhat model=committee_reference ap=0.502424 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_nhat models=13 positive_rows=17 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_phap fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_phap fold=1 origin=6060 ap=0.109906 rows=2549 failures=26 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.005682 ap=0.115588 base_ap=0.109906 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=-0.051625 ap=0.058280 base_ap=0.109906 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6060 delta_ap=-0.022548 ap=0.087357 base_ap=0.109906 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_anh origin=6060 delta_ap=-0.005417 ap=0.104489 base_ap=0.109906 donor_failures=40 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.125684 ap=0.235590 base_ap=0.109906 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_duc origin=6060 delta_ap=-0.039535 ap=0.070371 base_ap=0.109906 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6060 delta_ap=0.067941 ap=0.177847 base_ap=0.109906 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_italia origin=6060 delta_ap=-0.016506 ap=0.093400 base_ap=0.109906 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nga origin=6060 delta_ap=0.046870 ap=0.156776 base_ap=0.109906 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6060 delta_ap=0.099922 ap=0.209827 base_ap=0.109906 donor_failures=74 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.394069 ap=0.503975 base_ap=0.109906 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6060 delta_ap=-0.062198 ap=0.047708 base_ap=0.109906 donor_failures=37 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6060 delta_ap=0.025581 ap=0.135487 base_ap=0.109906 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_phap fold=1 origin=6061 ap=0.525653 rows=2428 failures=17 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.015281 ap=0.510372 base_ap=0.525653 donor_failures=177 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.002961 ap=0.528614 base_ap=0.525653 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.026913 ap=0.498740 base_ap=0.525653 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_anh origin=6061 delta_ap=-0.013169 ap=0.512485 base_ap=0.525653 donor_failures=51 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.011823 ap=0.513830 base_ap=0.525653 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_duc origin=6061 delta_ap=-0.029422 ap=0.496231 base_ap=0.525653 donor_failures=78 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.041599 ap=0.484055 base_ap=0.525653 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_italia origin=6061 delta_ap=0.086138 ap=0.611791 base_ap=0.525653 donor_failures=167 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nga origin=6061 delta_ap=-0.072926 ap=0.452727 base_ap=0.525653 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6061 delta_ap=-0.030037 ap=0.495616 base_ap=0.525653 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.027748 ap=0.497905 base_ap=0.525653 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.042951 ap=0.568604 base_ap=0.525653 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.044589 ap=0.481064 base_ap=0.525653 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_phap fold=1 origin=6063 ap=0.095478 rows=2758 failures=12 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.164144 ap=0.259622 base_ap=0.095478 donor_failures=173 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.117090 ap=0.212568 base_ap=0.095478 donor_failures=93 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6063 delta_ap=0.017694 ap=0.113172 base_ap=0.095478 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_anh origin=6063 delta_ap=0.223976 ap=0.319454 base_ap=0.095478 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.221186 ap=0.316664 base_ap=0.095478 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_duc origin=6063 delta_ap=0.075919 ap=0.171396 base_ap=0.095478 donor_failures=56 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.019511 ap=0.075967 base_ap=0.095478 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_italia origin=6063 delta_ap=-0.007475 ap=0.088002 base_ap=0.095478 donor_failures=144 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nga origin=6063 delta_ap=-0.021864 ap=0.073614 base_ap=0.095478 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6063 delta_ap=0.190769 ap=0.286247 base_ap=0.095478 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.215147 ap=0.310625 base_ap=0.095478 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.027932 ap=0.123410 base_ap=0.095478 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6063 delta_ap=0.053991 ap=0.149469 base_ap=0.095478 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_phap fold=1 origin=6064 ap=0.461112 rows=2471 failures=32 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.073550 ap=0.387562 base_ap=0.461112 donor_failures=125 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.063987 ap=0.397125 base_ap=0.461112 donor_failures=98 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.069882 ap=0.391230 base_ap=0.461112 donor_failures=42 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_anh origin=6064 delta_ap=-0.081079 ap=0.380033 base_ap=0.461112 donor_failures=70 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.020996 ap=0.482108 base_ap=0.461112 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_duc origin=6064 delta_ap=-0.025625 ap=0.435487 base_ap=0.461112 donor_failures=44 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6064 delta_ap=0.038723 ap=0.499835 base_ap=0.461112 donor_failures=13 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_italia origin=6064 delta_ap=-0.072245 ap=0.388867 base_ap=0.461112 donor_failures=112 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nga origin=6064 delta_ap=-0.012501 ap=0.448611 base_ap=0.461112 donor_failures=148 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6064 delta_ap=-0.007582 ap=0.453530 base_ap=0.461112 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.055666 ap=0.405446 base_ap=0.461112 donor_failures=85 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.105923 ap=0.355189 base_ap=0.461112 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.046523 ap=0.414589 base_ap=0.461112 donor_failures=269 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_phap origin=6063 policy_sources=khoa_tieng_tay_ban_nha,khoa_tieng_bo_dao_nha,khoa_tieng_nhat policy_donor_count=3 base_ap=0.095478 policy_ap=0.343856 delta_ap=0.248378 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_phap origin=6064 policy_sources=khoa_tieng_tay_ban_nha,khoa_tieng_bo_dao_nha,khoa_tieng_nhat policy_donor_count=3 base_ap=0.461112 policy_ap=0.477477 delta_ap=0.016365 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_phap selected=['khoa_tieng_tay_ban_nha', 'khoa_tieng_bo_dao_nha', 'khoa_tieng_nhat'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.132372 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=hgb_local_common rows=2606 failures=42 prevalence=0.016117 ap=0.241290 auc=0.858318 f2_at20=0.217391 brier=0.015499 ece10=0.014665 rho=0.000000 donor_rows=0 effective_n=15265.000000 seconds=0.574829 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=hgb_local_common in_sample_bce=0.009162 local_bce=0.009162 donor_bce=nan in_sample_local_ap=0.958158 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=hgb_local_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 178} positive_ess=178.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=hgb_local_native rows=2606 failures=42 prevalence=0.016117 ap=0.236795 auc=0.893916 f2_at20=0.231884 brier=0.015482 ece10=0.014413 rho=0.000000 donor_rows=0 effective_n=15265.000000 seconds=1.250116 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=hgb_local_native in_sample_bce=0.008851 local_bce=0.008851 donor_bce=nan in_sample_local_ap=0.963462 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=hgb_local_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 178} positive_ess=178.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=hgb_pool_all rows=2606 failures=42 prevalence=0.016117 ap=0.278470 auc=0.934997 f2_at20=0.282609 brier=0.014984 ece10=0.014050 rho=0.250000 donor_rows=4095 effective_n=16578.539266 seconds=0.617423 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=hgb_pool_all in_sample_bce=0.014947 local_bce=0.013134 donor_bce=0.036473 in_sample_local_ap=0.918317 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=hgb_pool_all mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.02117281184701451, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03710288500357132, 'khoa_quoc_te_hoc': 0.01157794567160736, 'khoa_tieng_anh': 0.053233706140448166, 'khoa_tieng_bo_dao_nha': 0.00456618472363325, 'khoa_tieng_duc': 0.012070388231066471, 'khoa_tieng_han_quoc': 0.018967911378084974, 'khoa_tieng_italia': 0.011811966797836802, 'khoa_tieng_nga': 0.00732896493897705, 'khoa_tieng_nhat': 0.02112068391842312, 'khoa_tieng_tay_ban_nha': 0.008900566530764351, 'khoa_tieng_trung_quoc': 0.036357566535200773, 'khoa_viet_nam_hoc': 0.005788418283371856} source_failures={'khoa_tieng_phap': 178, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 6, 'khoa_tieng_nhat': 2, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 14} positive_ess=208.979582 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=hgb_pool_equal rows=2606 failures=42 prevalence=0.016117 ap=0.292773 auc=0.940314 f2_at20=0.289855 brier=0.013990 ece10=0.012568 rho=0.250000 donor_rows=4095 effective_n=19189.616270 seconds=0.524357 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=hgb_pool_equal in_sample_bce=0.016393 local_bce=0.013495 donor_bce=0.025086 in_sample_local_ap=0.913936 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=hgb_pool_equal mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_phap': 178, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 6, 'khoa_tieng_nhat': 2, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 14} positive_ess=260.121096 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=hgb_language_prior rows=2606 failures=42 prevalence=0.016117 ap=0.273459 auc=0.869605 f2_at20=0.195652 brier=0.014812 ece10=0.012870 rho=0.250000 donor_rows=4095 effective_n=19189.616270 seconds=0.527718 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=hgb_language_prior in_sample_bce=0.015896 local_bce=0.013646 donor_bce=0.022645 in_sample_local_ap=0.909965 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=hgb_language_prior mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_phap': 178, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 6, 'khoa_tieng_tay_ban_nha': 12, 'khoa_tieng_trung_quoc': 5} positive_ess=252.283549 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=hgb_random_donor rows=2606 failures=42 prevalence=0.016117 ap=0.323989 auc=0.910505 f2_at20=0.231884 brier=0.014259 ece10=0.012165 rho=0.250000 donor_rows=4095 effective_n=19189.616270 seconds=0.560642 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=hgb_random_donor in_sample_bce=0.023097 local_bce=0.015285 donor_bce=0.046533 in_sample_local_ap=0.889985 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=hgb_random_donor mass={'khoa_tieng_phap': 0.7500000000000001, np.str_('khoa_tieng_nga'): 0.08333333333333333, np.str_('khoa_tieng_italia'): 0.08333333333333333, np.str_('khoa_cong_nghe_thong_tin'): 0.08333333333333333} source_failures={'khoa_tieng_phap': 178, np.str_('khoa_tieng_nga'): 61, np.str_('khoa_tieng_italia'): 39, np.str_('khoa_cong_nghe_thong_tin'): 42} positive_ess=316.256041 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=suite32_selective_common rows=2606 failures=42 prevalence=0.016117 ap=0.257946 auc=0.884337 f2_at20=0.217391 brier=0.015320 ece10=0.013829 rho=0.250000 donor_rows=4095 effective_n=18705.564362 seconds=0.530825 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=suite32_selective_common in_sample_bce=0.016652 local_bce=0.013814 donor_bce=0.024450 in_sample_local_ap=0.906033 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=suite32_selective_common mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_tay_ban_nha': 0.11582398505210353, 'khoa_tieng_bo_dao_nha': 0.07842941981061026, 'khoa_tieng_nhat': 0.0557465951372862} source_failures={'khoa_tieng_phap': 178, 'khoa_tieng_tay_ban_nha': 37, 'khoa_tieng_bo_dao_nha': 45, 'khoa_tieng_nhat': 16} positive_ess=262.515174 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=suite32_selective_native rows=2606 failures=42 prevalence=0.016117 ap=0.264542 auc=0.910570 f2_at20=0.275362 brier=0.015118 ece10=0.013393 rho=0.250000 donor_rows=4095 effective_n=18705.564362 seconds=1.407134 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=suite32_selective_native in_sample_bce=0.016563 local_bce=0.013336 donor_bce=0.025512 in_sample_local_ap=0.911369 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=suite32_selective_native mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_tay_ban_nha': 0.11582398505210353, 'khoa_tieng_bo_dao_nha': 0.07842941981061026, 'khoa_tieng_nhat': 0.0557465951372862} source_failures={'khoa_tieng_phap': 178, 'khoa_tieng_tay_ban_nha': 37, 'khoa_tieng_bo_dao_nha': 45, 'khoa_tieng_nhat': 16} positive_ess=262.515174 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=native_cache_nan_indicator rows=2606 failures=42 prevalence=0.016117 ap=0.236795 auc=0.893916 f2_at20=0.231884 brier=0.015482 ece10=0.014413 rho=0.000000 donor_rows=0 effective_n=15265.000000 seconds=2.780773 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=native_cache_nan_indicator in_sample_bce=0.008851 local_bce=0.008851 donor_bce=nan in_sample_local_ap=0.963462 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=native_cache_nan_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 178} positive_ess=178.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=native_naive_zero_indicator rows=2606 failures=42 prevalence=0.016117 ap=0.247286 auc=0.900254 f2_at20=0.246377 brier=0.015523 ece10=0.014622 rho=0.000000 donor_rows=0 effective_n=15265.000000 seconds=2.738064 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=native_naive_zero_indicator in_sample_bce=0.008841 local_bce=0.008841 donor_bce=nan in_sample_local_ap=0.962658 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=native_naive_zero_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 178} positive_ess=178.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=native_placebo_indicator rows=2606 failures=42 prevalence=0.016117 ap=0.236795 auc=0.893916 f2_at20=0.231884 brier=0.015482 ece10=0.014413 rho=0.000000 donor_rows=0 effective_n=15265.000000 seconds=2.768396 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=native_placebo_indicator in_sample_bce=0.008851 local_bce=0.008851 donor_bce=nan in_sample_local_ap=0.963462 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=native_placebo_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 178} positive_ess=178.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan rows=2606 failures=42 prevalence=0.016117 ap=0.264542 auc=0.910570 f2_at20=0.275362 brier=0.015118 ece10=0.013393 rho=0.250000 donor_rows=4095 effective_n=18705.564362 seconds=3.445381 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan in_sample_bce=0.016563 local_bce=0.013336 donor_bce=0.025512 in_sample_local_ap=0.911369 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_tay_ban_nha': 0.11582398505210353, 'khoa_tieng_bo_dao_nha': 0.07842941981061026, 'khoa_tieng_nhat': 0.0557465951372862} source_failures={'khoa_tieng_phap': 178, 'khoa_tieng_tay_ban_nha': 37, 'khoa_tieng_bo_dao_nha': 45, 'khoa_tieng_nhat': 16} positive_ess=262.515174 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_phap model=committee_reference ap=0.227874 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_phap models=13 positive_rows=42 num_edges=52 num_replays=2 chosen=['khoa_tieng_tay_ban_nha', 'khoa_tieng_bo_dao_nha', 'khoa_tieng_nhat'] [SELECTION_ORIGINS] target=khoa_tieng_tay_ban_nha fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=1 origin=6060 ap=0.268471 rows=1295 failures=25 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.056687 ap=0.325158 base_ap=0.268471 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.098693 ap=0.367164 base_ap=0.268471 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6060 delta_ap=-0.040476 ap=0.227995 base_ap=0.268471 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6060 delta_ap=-0.092796 ap=0.175675 base_ap=0.268471 donor_failures=37 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=0.063622 ap=0.332093 base_ap=0.268471 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6060 delta_ap=0.078769 ap=0.347240 base_ap=0.268471 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6060 delta_ap=0.035196 ap=0.303668 base_ap=0.268471 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6060 delta_ap=-0.033560 ap=0.234911 base_ap=0.268471 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6060 delta_ap=0.061300 ap=0.329771 base_ap=0.268471 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6060 delta_ap=-0.050840 ap=0.217631 base_ap=0.268471 donor_failures=62 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6060 delta_ap=0.054067 ap=0.322538 base_ap=0.268471 donor_failures=73 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6060 delta_ap=-0.010555 ap=0.257916 base_ap=0.268471 donor_failures=34 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6060 delta_ap=0.048871 ap=0.317342 base_ap=0.268471 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=1 origin=6061 ap=0.346033 rows=1419 failures=20 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.096078 ap=0.442111 base_ap=0.346033 donor_failures=191 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.024532 ap=0.321501 base_ap=0.346033 donor_failures=84 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.039900 ap=0.306134 base_ap=0.346033 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6061 delta_ap=-0.027639 ap=0.318394 base_ap=0.346033 donor_failures=60 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.014939 ap=0.331094 base_ap=0.346033 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6061 delta_ap=-0.004151 ap=0.341883 base_ap=0.346033 donor_failures=77 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.002963 ap=0.343070 base_ap=0.346033 donor_failures=28 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6061 delta_ap=-0.006303 ap=0.339730 base_ap=0.346033 donor_failures=158 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6061 delta_ap=0.060475 ap=0.406508 base_ap=0.346033 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6061 delta_ap=0.057263 ap=0.403296 base_ap=0.346033 donor_failures=69 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6061 delta_ap=0.010645 ap=0.356678 base_ap=0.346033 donor_failures=62 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.090616 ap=0.436649 base_ap=0.346033 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.017930 ap=0.328103 base_ap=0.346033 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=1 origin=6063 ap=0.359683 rows=1560 failures=18 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.035478 ap=0.395160 base_ap=0.359683 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.109774 ap=0.469456 base_ap=0.359683 donor_failures=98 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6063 delta_ap=0.046427 ap=0.406110 base_ap=0.359683 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6063 delta_ap=0.049637 ap=0.409320 base_ap=0.359683 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.061504 ap=0.298179 base_ap=0.359683 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6063 delta_ap=-0.158590 ap=0.201093 base_ap=0.359683 donor_failures=70 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6063 delta_ap=0.044294 ap=0.403977 base_ap=0.359683 donor_failures=22 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6063 delta_ap=0.140048 ap=0.499730 base_ap=0.359683 donor_failures=153 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6063 delta_ap=-0.001060 ap=0.358622 base_ap=0.359683 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6063 delta_ap=0.015375 ap=0.375058 base_ap=0.359683 donor_failures=52 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6063 delta_ap=0.019975 ap=0.379658 base_ap=0.359683 donor_failures=57 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.102579 ap=0.462262 base_ap=0.359683 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6063 delta_ap=0.016363 ap=0.376045 base_ap=0.359683 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=1 origin=6064 ap=0.300379 rows=1388 failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.051831 ap=0.352210 base_ap=0.300379 donor_failures=118 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.070367 ap=0.370746 base_ap=0.300379 donor_failures=87 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=0.019508 ap=0.319887 base_ap=0.300379 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6064 delta_ap=0.062967 ap=0.363346 base_ap=0.300379 donor_failures=68 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.037395 ap=0.337774 base_ap=0.300379 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6064 delta_ap=0.016986 ap=0.317364 base_ap=0.300379 donor_failures=52 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=0.086764 ap=0.387143 base_ap=0.300379 donor_failures=17 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6064 delta_ap=-0.019941 ap=0.280437 base_ap=0.300379 donor_failures=136 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6064 delta_ap=0.051624 ap=0.352003 base_ap=0.300379 donor_failures=151 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6064 delta_ap=0.042650 ap=0.343029 base_ap=0.300379 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6064 delta_ap=0.022802 ap=0.323181 base_ap=0.300379 donor_failures=55 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.054673 ap=0.355052 base_ap=0.300379 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.070244 ap=0.370622 base_ap=0.300379 donor_failures=265 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_tay_ban_nha origin=6063 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nga,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.359683 policy_ap=0.354509 delta_ap=-0.005173 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_tay_ban_nha origin=6064 policy_sources=khoa_cong_nghe_thong_tin,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.300379 policy_ap=0.348853 delta_ap=0.048475 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_tay_ban_nha selected=['khoa_quan_tri_kinh_doanh_va_du_lich', 'khoa_cong_nghe_thong_tin', 'khoa_tieng_trung_quoc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.021651 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common rows=1401 failures=63 prevalence=0.044968 ap=0.245857 auc=0.813124 f2_at20=0.393996 brier=0.045376 ece10=0.044968 rho=0.000000 donor_rows=0 effective_n=8025.000000 seconds=0.512077 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common in_sample_bce=0.012930 local_bce=0.012930 donor_bce=nan in_sample_local_ap=0.981759 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 170} positive_ess=170.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native rows=1401 failures=63 prevalence=0.044968 ap=0.260794 auc=0.803432 f2_at20=0.403377 brier=0.045109 ece10=0.044785 rho=0.000000 donor_rows=0 effective_n=8025.000000 seconds=0.889555 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native in_sample_bce=0.012755 local_bce=0.012755 donor_bce=nan in_sample_local_ap=0.982840 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 170} positive_ess=170.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all rows=1401 failures=63 prevalence=0.044968 ap=0.217776 auc=0.852605 f2_at20=0.431520 brier=0.043847 ece10=0.041592 rho=0.250000 donor_rows=4095 effective_n=10782.427809 seconds=0.568011 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all in_sample_bce=0.019432 local_bce=0.017538 donor_bce=0.043866 in_sample_local_ap=0.958290 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.020513911634364508, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.035948239177810155, 'khoa_quoc_te_hoc': 0.011217638740237356, 'khoa_tieng_anh': 0.05157706740254549, 'khoa_tieng_bo_dao_nha': 0.004424084557290659, 'khoa_tieng_duc': 0.011694756433566733, 'khoa_tieng_han_quoc': 0.018377627908376208, 'khoa_tieng_italia': 0.011444377103508749, 'khoa_tieng_nga': 0.007100886751172377, 'khoa_tieng_nhat': 0.020463405932593156, 'khoa_tieng_phap': 0.016403607181695874, 'khoa_tieng_trung_quoc': 0.03522611510141975, 'khoa_viet_nam_hoc': 0.005608282075418983} source_failures={'khoa_tieng_tay_ban_nha': 170, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 6, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 3, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 19} positive_ess=216.593824 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal rows=1401 failures=63 prevalence=0.044968 ap=0.199324 auc=0.848453 f2_at20=0.403377 brier=0.043595 ece10=0.040506 rho=0.250000 donor_rows=4095 effective_n=11715.641711 seconds=0.508132 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal in_sample_bce=0.022228 local_bce=0.018258 donor_bce=0.034137 in_sample_local_ap=0.952932 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_tay_ban_nha': 170, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 6, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 3, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 19} positive_ess=243.822937 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior rows=1401 failures=63 prevalence=0.044968 ap=0.251571 auc=0.872482 f2_at20=0.440901 brier=0.043561 ece10=0.041832 rho=0.250000 donor_rows=4095 effective_n=11715.641711 seconds=0.482533 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior in_sample_bce=0.021090 local_bce=0.018246 donor_bce=0.029620 in_sample_local_ap=0.955129 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_tay_ban_nha': 170, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 6, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 7, 'khoa_tieng_phap': 6, 'khoa_tieng_trung_quoc': 3} positive_ess=238.325405 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor rows=1401 failures=63 prevalence=0.044968 ap=0.241751 auc=0.856745 f2_at20=0.431520 brier=0.043551 ece10=0.041953 rho=0.250000 donor_rows=4095 effective_n=11715.641711 seconds=0.506421 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor in_sample_bce=0.023390 local_bce=0.018469 donor_bce=0.038152 in_sample_local_ap=0.957199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor mass={'khoa_tieng_tay_ban_nha': 0.75, np.str_('khoa_tieng_nga'): 0.08333333333333333, np.str_('khoa_tieng_anh'): 0.08333333333333333, np.str_('khoa_quoc_te_hoc'): 0.08333333333333333} source_failures={'khoa_tieng_tay_ban_nha': 170, np.str_('khoa_tieng_nga'): 56, np.str_('khoa_tieng_anh'): 29, np.str_('khoa_quoc_te_hoc'): 13} positive_ess=258.542410 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common rows=1401 failures=63 prevalence=0.044968 ap=0.243276 auc=0.843067 f2_at20=0.422139 brier=0.044254 ece10=0.043836 rho=0.250000 donor_rows=4095 effective_n=11713.690897 seconds=0.553089 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common in_sample_bce=0.022337 local_bce=0.017737 donor_bce=0.035861 in_sample_local_ap=0.959979 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.08688853511000326, 'khoa_cong_nghe_thong_tin': 0.08202727028676778, 'khoa_tieng_trung_quoc': 0.08108419460322894} source_failures={'khoa_tieng_tay_ban_nha': 170, 'khoa_quan_tri_kinh_doanh_va_du_lich': 38, 'khoa_cong_nghe_thong_tin': 43, 'khoa_tieng_trung_quoc': 6} positive_ess=248.690780 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native rows=1401 failures=63 prevalence=0.044968 ap=0.240801 auc=0.829899 f2_at20=0.412758 brier=0.044610 ece10=0.043850 rho=0.250000 donor_rows=4095 effective_n=11713.690897 seconds=1.018211 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native in_sample_bce=0.022194 local_bce=0.017489 donor_bce=0.036030 in_sample_local_ap=0.961365 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.08688853511000326, 'khoa_cong_nghe_thong_tin': 0.08202727028676778, 'khoa_tieng_trung_quoc': 0.08108419460322894} source_failures={'khoa_tieng_tay_ban_nha': 170, 'khoa_quan_tri_kinh_doanh_va_du_lich': 38, 'khoa_cong_nghe_thong_tin': 43, 'khoa_tieng_trung_quoc': 6} positive_ess=248.690780 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator rows=1401 failures=63 prevalence=0.044968 ap=0.260794 auc=0.803432 f2_at20=0.403377 brier=0.045109 ece10=0.044785 rho=0.000000 donor_rows=0 effective_n=8025.000000 seconds=1.493503 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator in_sample_bce=0.012755 local_bce=0.012755 donor_bce=nan in_sample_local_ap=0.982840 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 170} positive_ess=170.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator rows=1401 failures=63 prevalence=0.044968 ap=0.260794 auc=0.803432 f2_at20=0.403377 brier=0.045109 ece10=0.044785 rho=0.000000 donor_rows=0 effective_n=8025.000000 seconds=1.576373 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator in_sample_bce=0.012755 local_bce=0.012755 donor_bce=nan in_sample_local_ap=0.982840 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 170} positive_ess=170.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator rows=1401 failures=63 prevalence=0.044968 ap=0.260794 auc=0.803432 f2_at20=0.403377 brier=0.045109 ece10=0.044785 rho=0.000000 donor_rows=0 effective_n=8025.000000 seconds=1.592378 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator in_sample_bce=0.012755 local_bce=0.012755 donor_bce=nan in_sample_local_ap=0.982840 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 170} positive_ess=170.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan rows=1401 failures=63 prevalence=0.044968 ap=0.240801 auc=0.829899 f2_at20=0.412758 brier=0.044610 ece10=0.043850 rho=0.250000 donor_rows=4095 effective_n=11713.690897 seconds=1.828386 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan in_sample_bce=0.022194 local_bce=0.017489 donor_bce=0.036030 in_sample_local_ap=0.961365 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.08688853511000326, 'khoa_cong_nghe_thong_tin': 0.08202727028676778, 'khoa_tieng_trung_quoc': 0.08108419460322894} source_failures={'khoa_tieng_tay_ban_nha': 170, 'khoa_quan_tri_kinh_doanh_va_du_lich': 38, 'khoa_cong_nghe_thong_tin': 43, 'khoa_tieng_trung_quoc': 6} positive_ess=248.690780 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_tay_ban_nha model=committee_reference ap=0.168906 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_tay_ban_nha models=13 positive_rows=63 num_edges=52 num_replays=2 chosen=['khoa_quan_tri_kinh_doanh_va_du_lich', 'khoa_cong_nghe_thong_tin', 'khoa_tieng_trung_quoc'] [SELECTION_ORIGINS] target=khoa_tieng_trung_quoc fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=1 origin=6060 ap=0.049501 rows=6574 failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.102176 ap=0.151678 base_ap=0.049501 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.105810 ap=0.155311 base_ap=0.049501 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6060 delta_ap=0.069176 ap=0.118678 base_ap=0.049501 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6060 delta_ap=0.127503 ap=0.177004 base_ap=0.049501 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=-0.009398 ap=0.040103 base_ap=0.049501 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6060 delta_ap=0.053493 ap=0.102994 base_ap=0.049501 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6060 delta_ap=0.005523 ap=0.055024 base_ap=0.049501 donor_failures=23 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6060 delta_ap=0.036771 ap=0.086272 base_ap=0.049501 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6060 delta_ap=0.097710 ap=0.147212 base_ap=0.049501 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6060 delta_ap=0.119606 ap=0.169108 base_ap=0.049501 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6060 delta_ap=0.000679 ap=0.050180 base_ap=0.049501 donor_failures=72 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.110937 ap=0.160438 base_ap=0.049501 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6060 delta_ap=0.018234 ap=0.067735 base_ap=0.049501 donor_failures=121 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=1 origin=6061 ap=0.609861 rows=4785 failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.046094 ap=0.563767 base_ap=0.609861 donor_failures=186 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.046834 ap=0.563027 base_ap=0.609861 donor_failures=96 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.073740 ap=0.536121 base_ap=0.609861 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6061 delta_ap=0.025689 ap=0.635550 base_ap=0.609861 donor_failures=57 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.006602 ap=0.603259 base_ap=0.609861 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6061 delta_ap=-0.052598 ap=0.557263 base_ap=0.609861 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.131473 ap=0.478387 base_ap=0.609861 donor_failures=31 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6061 delta_ap=-0.188196 ap=0.421665 base_ap=0.609861 donor_failures=157 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6061 delta_ap=-0.039240 ap=0.570621 base_ap=0.609861 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6061 delta_ap=-0.110838 ap=0.499022 base_ap=0.609861 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6061 delta_ap=-0.118137 ap=0.491724 base_ap=0.609861 donor_failures=64 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.003099 ap=0.612960 base_ap=0.609861 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6061 delta_ap=0.014459 ap=0.624320 base_ap=0.609861 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=1 origin=6063 ap=0.043199 rows=7413 failures=13 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.083401 ap=0.126600 base_ap=0.043199 donor_failures=168 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.044296 ap=0.087495 base_ap=0.043199 donor_failures=84 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.053481 ap=0.096680 base_ap=0.043199 donor_failures=35 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6063 delta_ap=0.116456 ap=0.159654 base_ap=0.043199 donor_failures=70 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.017553 ap=0.025645 base_ap=0.043199 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6063 delta_ap=0.064096 ap=0.107295 base_ap=0.043199 donor_failures=67 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.247357 ap=0.290556 base_ap=0.043199 donor_failures=17 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6063 delta_ap=0.185827 ap=0.229026 base_ap=0.043199 donor_failures=137 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6063 delta_ap=0.052658 ap=0.095857 base_ap=0.043199 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6063 delta_ap=-0.007806 ap=0.035393 base_ap=0.043199 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6063 delta_ap=0.059423 ap=0.102622 base_ap=0.043199 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.079473 ap=0.122672 base_ap=0.043199 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.113382 ap=0.156581 base_ap=0.043199 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=1 origin=6064 ap=0.273638 rows=5347 failures=29 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.117696 ap=0.155942 base_ap=0.273638 donor_failures=135 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.085334 ap=0.188304 base_ap=0.273638 donor_failures=90 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.061592 ap=0.212046 base_ap=0.273638 donor_failures=36 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6064 delta_ap=-0.010685 ap=0.262953 base_ap=0.273638 donor_failures=74 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.076517 ap=0.197121 base_ap=0.273638 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6064 delta_ap=-0.080530 ap=0.193108 base_ap=0.273638 donor_failures=52 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.028510 ap=0.245127 base_ap=0.273638 donor_failures=16 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6064 delta_ap=-0.045723 ap=0.227915 base_ap=0.273638 donor_failures=118 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.055390 ap=0.218248 base_ap=0.273638 donor_failures=150 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.082025 ap=0.355663 base_ap=0.273638 donor_failures=44 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.075460 ap=0.198178 base_ap=0.273638 donor_failures=39 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.002088 ap=0.271550 base_ap=0.273638 donor_failures=83 [TRANSFER_EDGE] fold=1 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.093192 ap=0.180446 base_ap=0.273638 donor_failures=270 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_trung_quoc origin=6063 policy_sources=khoa_tieng_anh,khoa_tieng_tay_ban_nha,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.043199 policy_ap=0.160908 delta_ap=0.117709 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_tieng_trung_quoc origin=6064 policy_sources=khoa_tieng_anh,khoa_tieng_tay_ban_nha,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.273638 policy_ap=0.291261 delta_ap=0.017623 policy_donor_rows=4095 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_tieng_trung_quoc selected=['khoa_tieng_anh', 'khoa_tieng_tay_ban_nha', 'khoa_tieng_han_quoc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.067666 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_local_common rows=7393 failures=35 prevalence=0.004734 ap=0.238954 auc=0.867835 f2_at20=0.086473 brier=0.004546 ece10=0.004151 rho=0.000000 donor_rows=0 effective_n=32781.000000 seconds=0.785226 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_local_common in_sample_bce=0.004904 local_bce=0.004904 donor_bce=nan in_sample_local_ap=0.955660 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_local_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 185} positive_ess=185.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_local_native rows=7393 failures=35 prevalence=0.004734 ap=0.185500 auc=0.811385 f2_at20=0.071032 brier=0.004568 ece10=0.004073 rho=0.000000 donor_rows=0 effective_n=32781.000000 seconds=1.708477 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_local_native in_sample_bce=0.004854 local_bce=0.004854 donor_bce=nan in_sample_local_ap=0.960489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_local_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 185} positive_ess=185.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_pool_all rows=7393 failures=35 prevalence=0.004734 ap=0.155886 auc=0.851934 f2_at20=0.086473 brier=0.004533 ece10=0.003476 rho=0.250000 donor_rows=4095 effective_n=24457.497548 seconds=0.751544 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_pool_all in_sample_bce=0.012326 local_bce=0.008886 donor_bce=0.038683 in_sample_local_ap=0.850391 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_pool_all mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.022956741754092288, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.040229014243040824, 'khoa_quoc_te_hoc': 0.012553453492455182, 'khoa_tieng_anh': 0.05771894887224555, 'khoa_tieng_bo_dao_nha': 0.004950911775882554, 'khoa_tieng_duc': 0.013087387140376447, 'khoa_tieng_han_quoc': 0.02056606587105743, 'khoa_tieng_italia': 0.012807192230543891, 'khoa_tieng_nga': 0.007946471949242633, 'khoa_tieng_nhat': 0.022900221750821344, 'khoa_tieng_phap': 0.018356975530446244, 'khoa_tieng_tay_ban_nha': 0.009650489920198565, 'khoa_viet_nam_hoc': 0.006276125469597048} source_failures={'khoa_tieng_trung_quoc': 185, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 18, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 10, 'khoa_viet_nam_hoc': 30} positive_ess=181.133103 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal rows=7393 failures=35 prevalence=0.004734 ap=0.197455 auc=0.882915 f2_at20=0.074120 brier=0.004596 ece10=0.003635 rho=0.250000 donor_rows=4095 effective_n=30843.401689 seconds=0.757288 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal in_sample_bce=0.014213 local_bce=0.009409 donor_bce=0.028627 in_sample_local_ap=0.832971 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_trung_quoc': 185, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 18, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 10, 'khoa_viet_nam_hoc': 30} positive_ess=242.820795 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_language_prior rows=7393 failures=35 prevalence=0.004734 ap=0.061694 auc=0.793150 f2_at20=0.067943 brier=0.004664 ece10=0.003774 rho=0.250000 donor_rows=4095 effective_n=30843.401689 seconds=0.767110 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_language_prior in_sample_bce=0.012584 local_bce=0.009230 donor_bce=0.022648 in_sample_local_ap=0.848103 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_language_prior mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776} source_failures={'khoa_tieng_trung_quoc': 185, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 21, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 8} positive_ess=218.880171 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_random_donor rows=7393 failures=35 prevalence=0.004734 ap=0.125290 auc=0.804318 f2_at20=0.067943 brier=0.004642 ece10=0.003689 rho=0.250000 donor_rows=4095 effective_n=30843.401689 seconds=0.825100 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_random_donor in_sample_bce=0.016465 local_bce=0.009870 donor_bce=0.036251 in_sample_local_ap=0.825441 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=hgb_random_donor mass={'khoa_tieng_trung_quoc': 0.75, np.str_('khoa_tieng_bo_dao_nha'): 0.08333333333333333, np.str_('khoa_cong_nghe_thong_tin'): 0.08333333333333333, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.08333333333333333} source_failures={'khoa_tieng_trung_quoc': 185, np.str_('khoa_tieng_bo_dao_nha'): 51, np.str_('khoa_cong_nghe_thong_tin'): 47, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 40} positive_ess=262.137798 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_common rows=7393 failures=35 prevalence=0.004734 ap=0.125318 auc=0.785429 f2_at20=0.067943 brier=0.004745 ece10=0.004282 rho=0.250000 donor_rows=4095 effective_n=28913.611443 seconds=0.802639 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_common in_sample_bce=0.007986 local_bce=0.007513 donor_bce=0.009458 in_sample_local_ap=0.880752 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_common mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.11916631391556239, 'khoa_tieng_tay_ban_nha': 0.08808594392046182, 'khoa_tieng_han_quoc': 0.042747742163975776} source_failures={'khoa_tieng_trung_quoc': 185, 'khoa_tieng_anh': 23, 'khoa_tieng_tay_ban_nha': 25, 'khoa_tieng_han_quoc': 8} positive_ess=171.005354 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_native rows=7393 failures=35 prevalence=0.004734 ap=0.083993 auc=0.810224 f2_at20=0.067943 brier=0.004723 ece10=0.004153 rho=0.250000 donor_rows=4095 effective_n=28913.611443 seconds=1.792718 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_native in_sample_bce=0.008076 local_bce=0.007443 donor_bce=0.009912 in_sample_local_ap=0.886452 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_native mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.11916631391556239, 'khoa_tieng_tay_ban_nha': 0.08808594392046182, 'khoa_tieng_han_quoc': 0.042747742163975776} source_failures={'khoa_tieng_trung_quoc': 185, 'khoa_tieng_anh': 23, 'khoa_tieng_tay_ban_nha': 25, 'khoa_tieng_han_quoc': 8} positive_ess=171.005354 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator rows=7393 failures=35 prevalence=0.004734 ap=0.185500 auc=0.811385 f2_at20=0.071032 brier=0.004568 ece10=0.004073 rho=0.000000 donor_rows=0 effective_n=32781.000000 seconds=3.350186 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator in_sample_bce=0.004854 local_bce=0.004854 donor_bce=nan in_sample_local_ap=0.960489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 185} positive_ess=185.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator rows=7393 failures=35 prevalence=0.004734 ap=0.185500 auc=0.811385 f2_at20=0.071032 brier=0.004568 ece10=0.004073 rho=0.000000 donor_rows=0 effective_n=32781.000000 seconds=3.185803 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator in_sample_bce=0.004854 local_bce=0.004854 donor_bce=nan in_sample_local_ap=0.960489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 185} positive_ess=185.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator rows=7393 failures=35 prevalence=0.004734 ap=0.185500 auc=0.811385 f2_at20=0.071032 brier=0.004568 ece10=0.004073 rho=0.000000 donor_rows=0 effective_n=32781.000000 seconds=3.332728 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator in_sample_bce=0.004854 local_bce=0.004854 donor_bce=nan in_sample_local_ap=0.960489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 185} positive_ess=185.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan rows=7393 failures=35 prevalence=0.004734 ap=0.083993 auc=0.810224 f2_at20=0.067943 brier=0.004723 ece10=0.004153 rho=0.250000 donor_rows=4095 effective_n=28913.611443 seconds=3.390417 [TRAINING_INSIGHT] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.008076 local_bce=0.007443 donor_bce=0.009912 in_sample_local_ap=0.886452 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.11916631391556239, 'khoa_tieng_tay_ban_nha': 0.08808594392046182, 'khoa_tieng_han_quoc': 0.042747742163975776} source_failures={'khoa_tieng_trung_quoc': 185, 'khoa_tieng_anh': 23, 'khoa_tieng_tay_ban_nha': 25, 'khoa_tieng_han_quoc': 8} positive_ess=171.005354 [FACULTY_RESULTS] fold=1 faculty=khoa_tieng_trung_quoc model=committee_reference ap=0.111159 [FACULTY_COMPLETE] fold=1 faculty=khoa_tieng_trung_quoc models=13 positive_rows=35 num_edges=52 num_replays=2 chosen=['khoa_tieng_anh', 'khoa_tieng_tay_ban_nha', 'khoa_tieng_han_quoc'] [SELECTION_ORIGINS] target=khoa_viet_nam_hoc fold=1 origins=[6060, 6061, 6063, 6064] availability=semester_proxy [SCREEN_BASE] target=khoa_viet_nam_hoc fold=1 origin=6060 ap=0.317331 rows=1192 failures=59 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6060 delta_ap=0.058630 ap=0.375961 base_ap=0.317331 donor_failures=224 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6060 delta_ap=0.028234 ap=0.345566 base_ap=0.317331 donor_failures=89 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6060 delta_ap=0.015808 ap=0.333140 base_ap=0.317331 donor_failures=33 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6060 delta_ap=0.045867 ap=0.363198 base_ap=0.317331 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6060 delta_ap=-0.057196 ap=0.260135 base_ap=0.317331 donor_failures=49 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6060 delta_ap=0.000621 ap=0.317952 base_ap=0.317331 donor_failures=46 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6060 delta_ap=0.012769 ap=0.330100 base_ap=0.317331 donor_failures=29 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6060 delta_ap=0.063972 ap=0.381304 base_ap=0.317331 donor_failures=127 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6060 delta_ap=0.036639 ap=0.353970 base_ap=0.317331 donor_failures=80 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6060 delta_ap=-0.019868 ap=0.297463 base_ap=0.317331 donor_failures=68 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6060 delta_ap=-0.047237 ap=0.270095 base_ap=0.317331 donor_failures=73 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6060 delta_ap=0.053926 ap=0.371257 base_ap=0.317331 donor_failures=61 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6060 delta_ap=0.025910 ap=0.343241 base_ap=0.317331 donor_failures=32 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=1 origin=6061 ap=0.720725 rows=866 failures=37 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.089940 ap=0.630785 base_ap=0.720725 donor_failures=188 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.020239 ap=0.700486 base_ap=0.720725 donor_failures=91 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.069362 ap=0.651363 base_ap=0.720725 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6061 delta_ap=-0.013747 ap=0.706978 base_ap=0.720725 donor_failures=47 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.096124 ap=0.624601 base_ap=0.720725 donor_failures=58 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6061 delta_ap=-0.004300 ap=0.716425 base_ap=0.720725 donor_failures=79 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.026291 ap=0.694434 base_ap=0.720725 donor_failures=27 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6061 delta_ap=-0.013727 ap=0.706998 base_ap=0.720725 donor_failures=156 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6061 delta_ap=-0.052145 ap=0.668580 base_ap=0.720725 donor_failures=142 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6061 delta_ap=-0.017012 ap=0.703714 base_ap=0.720725 donor_failures=66 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6061 delta_ap=-0.043657 ap=0.677068 base_ap=0.720725 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.057594 ap=0.663131 base_ap=0.720725 donor_failures=86 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6061 delta_ap=-0.044901 ap=0.675824 base_ap=0.720725 donor_failures=32 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=1 origin=6063 ap=0.549464 rows=993 failures=71 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.064512 ap=0.484951 base_ap=0.549464 donor_failures=170 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.073343 ap=0.476120 base_ap=0.549464 donor_failures=90 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.098072 ap=0.451391 base_ap=0.549464 donor_failures=32 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6063 delta_ap=-0.008206 ap=0.541258 base_ap=0.549464 donor_failures=65 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.092623 ap=0.456840 base_ap=0.549464 donor_failures=82 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6063 delta_ap=-0.048273 ap=0.501190 base_ap=0.549464 donor_failures=63 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.055480 ap=0.493984 base_ap=0.549464 donor_failures=21 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6063 delta_ap=-0.084770 ap=0.464694 base_ap=0.549464 donor_failures=149 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6063 delta_ap=-0.038298 ap=0.511165 base_ap=0.549464 donor_failures=172 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6063 delta_ap=-0.079409 ap=0.470055 base_ap=0.549464 donor_failures=50 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6063 delta_ap=-0.159203 ap=0.390261 base_ap=0.549464 donor_failures=54 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.072776 ap=0.476688 base_ap=0.549464 donor_failures=87 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.041374 ap=0.508090 base_ap=0.549464 donor_failures=32 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=1 origin=6064 ap=0.564570 rows=818 failures=49 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.024509 ap=0.540061 base_ap=0.564570 donor_failures=107 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.008250 ap=0.556320 base_ap=0.564570 donor_failures=96 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.015832 ap=0.548738 base_ap=0.564570 donor_failures=30 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6064 delta_ap=-0.014048 ap=0.550522 base_ap=0.564570 donor_failures=79 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.028974 ap=0.593544 base_ap=0.564570 donor_failures=97 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6064 delta_ap=-0.020557 ap=0.544013 base_ap=0.564570 donor_failures=54 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.006491 ap=0.558079 base_ap=0.564570 donor_failures=24 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6064 delta_ap=-0.031863 ap=0.532707 base_ap=0.564570 donor_failures=122 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6064 delta_ap=-0.019119 ap=0.545451 base_ap=0.564570 donor_failures=152 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6064 delta_ap=-0.014066 ap=0.550504 base_ap=0.564570 donor_failures=44 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6064 delta_ap=-0.005170 ap=0.559400 base_ap=0.564570 donor_failures=44 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.001814 ap=0.562756 base_ap=0.564570 donor_failures=81 [TRANSFER_EDGE] fold=1 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.031506 ap=0.533064 base_ap=0.564570 donor_failures=15 [POLICY_REPLAY] fold=1 faculty=khoa_viet_nam_hoc origin=6063 policy_sources=khoa_tieng_italia,khoa_tieng_anh,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.549464 policy_ap=0.467914 delta_ap=-0.081550 policy_donor_rows=4095 selected_using_origins=6060,6061 [POLICY_REPLAY] fold=1 faculty=khoa_viet_nam_hoc origin=6064 policy_sources= policy_donor_count=0 base_ap=0.564570 policy_ap=0.564570 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6060,6061,6063 [SELECTION_FINAL] fold=1 faculty=khoa_viet_nam_hoc selected=[] reason=no_stable_positive_directed_edge replay_mean=-0.040775 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_local_common rows=813 failures=30 prevalence=0.036900 ap=0.151746 auc=0.754555 f2_at20=0.300353 brier=0.034819 ece10=0.029492 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.405933 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=hgb_local_common in_sample_bce=0.053209 local_bce=0.053209 donor_bce=nan in_sample_local_ap=0.943237 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_local_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_local_native rows=813 failures=30 prevalence=0.036900 ap=0.177257 auc=0.746041 f2_at20=0.265018 brier=0.034586 ece10=0.028615 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.421134 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=hgb_local_native in_sample_bce=0.053286 local_bce=0.053286 donor_bce=nan in_sample_local_ap=0.942363 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_local_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_pool_all rows=813 failures=30 prevalence=0.036900 ap=0.182874 auc=0.767327 f2_at20=0.318021 brier=0.034818 ece10=0.030230 rho=0.250000 donor_rows=4095 effective_n=7690.789537 seconds=0.503714 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=hgb_pool_all in_sample_bce=0.054327 local_bce=0.059341 donor_bce=0.052559 in_sample_local_ap=0.925454 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_pool_all mass={'khoa_viet_nam_hoc': 0.7499999999999999, 'khoa_cong_nghe_thong_tin': 0.02026943806194867, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03551982773632105, 'khoa_quoc_te_hoc': 0.011083953060695765, 'khoa_tieng_anh': 0.05096240013930593, 'khoa_tieng_bo_dao_nha': 0.004371360738661219, 'khoa_tieng_duc': 0.011555384726463456, 'khoa_tieng_han_quoc': 0.018158613396304146, 'khoa_tieng_italia': 0.011307989280238519, 'khoa_tieng_nga': 0.007016262268902923, 'khoa_tieng_nhat': 0.02021953425949128, 'khoa_tieng_phap': 0.016208117968341876, 'khoa_tieng_tay_ban_nha': 0.008520808823841701, 'khoa_tieng_trung_quoc': 0.034806309539483465} source_failures={'khoa_viet_nam_hoc': 337, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 1, 'khoa_tieng_trung_quoc': 2} positive_ess=376.137918 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_pool_equal rows=813 failures=30 prevalence=0.036900 ap=0.171623 auc=0.798723 f2_at20=0.318021 brier=0.036268 ece10=0.033978 rho=0.250000 donor_rows=4095 effective_n=8127.320435 seconds=0.503044 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=hgb_pool_equal in_sample_bce=0.056913 local_bce=0.060077 donor_bce=0.047420 in_sample_local_ap=0.922097 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_pool_equal mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232} source_failures={'khoa_viet_nam_hoc': 337, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 1, 'khoa_tieng_trung_quoc': 2} positive_ess=388.840694 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_language_prior rows=813 failures=30 prevalence=0.036900 ap=0.151746 auc=0.754555 f2_at20=0.300353 brier=0.034819 ece10=0.029492 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.367758 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=hgb_language_prior in_sample_bce=0.053209 local_bce=0.053209 donor_bce=nan in_sample_local_ap=0.943237 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_language_prior mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_random_donor rows=813 failures=30 prevalence=0.036900 ap=0.204582 auc=0.772137 f2_at20=0.282686 brier=0.033603 ece10=0.029002 rho=0.250000 donor_rows=4096 effective_n=8127.566571 seconds=0.445016 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=hgb_random_donor in_sample_bce=0.064979 local_bce=0.062316 donor_bce=0.072968 in_sample_local_ap=0.918721 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=hgb_random_donor mass={'khoa_viet_nam_hoc': 0.75, np.str_('khoa_cong_nghe_thong_tin'): 0.25} source_failures={'khoa_viet_nam_hoc': 337, np.str_('khoa_cong_nghe_thong_tin'): 131} positive_ess=427.487261 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_common rows=813 failures=30 prevalence=0.036900 ap=0.151746 auc=0.754555 f2_at20=0.300353 brier=0.034819 ece10=0.029492 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.369854 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_common in_sample_bce=0.053209 local_bce=0.053209 donor_bce=nan in_sample_local_ap=0.943237 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_native rows=813 failures=30 prevalence=0.036900 ap=0.177257 auc=0.746041 f2_at20=0.265018 brier=0.034586 ece10=0.028615 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.415009 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_native in_sample_bce=0.053286 local_bce=0.053286 donor_bce=nan in_sample_local_ap=0.942363 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator rows=813 failures=30 prevalence=0.036900 ap=0.177257 auc=0.746041 f2_at20=0.265018 brier=0.034586 ece10=0.028615 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.677400 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator in_sample_bce=0.053286 local_bce=0.053286 donor_bce=nan in_sample_local_ap=0.942363 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator rows=813 failures=30 prevalence=0.036900 ap=0.177779 auc=0.746552 f2_at20=0.282686 brier=0.034558 ece10=0.028664 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.648468 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator in_sample_bce=0.053286 local_bce=0.053286 donor_bce=nan in_sample_local_ap=0.942363 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=native_placebo_indicator rows=813 failures=30 prevalence=0.036900 ap=0.177257 auc=0.746041 f2_at20=0.265018 brier=0.034586 ece10=0.028615 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.663887 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=native_placebo_indicator in_sample_bce=0.053286 local_bce=0.053286 donor_bce=nan in_sample_local_ap=0.942363 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=native_placebo_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan rows=813 failures=30 prevalence=0.036900 ap=0.177257 auc=0.746041 f2_at20=0.265018 brier=0.034586 ece10=0.028615 rho=0.000000 donor_rows=0 effective_n=5219.000000 seconds=0.632988 [TRAINING_INSIGHT] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.053286 local_bce=0.053286 donor_bce=nan in_sample_local_ap=0.942363 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=1 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 337} positive_ess=337.000000 [FACULTY_RESULTS] fold=1 faculty=khoa_viet_nam_hoc model=committee_reference ap=0.137280 [FACULTY_COMPLETE] fold=1 faculty=khoa_viet_nam_hoc models=13 positive_rows=30 num_edges=52 num_replays=2 chosen=[] [FOLD_START] fold=2 stage=development [DATA] fold=2 common_raw=136 course_features=17 common_total=153 faculties=14 history_rows=286472 target_rows=42122 target_failures=933 history_last=6066 target_first=6067 [OBSERVATION_AUDIT] fold=2 faculty=khoa_cong_nghe_thong_tin raw_columns=275 common_raw_columns=136 native_raw_columns=139 history_rows=24312 history_failures=765 target_rows=4598 target_failures=152 target_prevalence=0.033058 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.657038 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich raw_columns=301 common_raw_columns=136 native_raw_columns=165 history_rows=39125 history_failures=1134 target_rows=5530 target_failures=166 target_prevalence=0.030018 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.692619 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_quoc_te_hoc raw_columns=254 common_raw_columns=136 native_raw_columns=118 history_rows=12802 history_failures=129 target_rows=2095 target_failures=19 target_prevalence=0.009069 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.621566 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_anh raw_columns=511 common_raw_columns=136 native_raw_columns=375 history_rows=56161 history_failures=969 target_rows=8328 target_failures=122 target_prevalence=0.014649 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.810394 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_bo_dao_nha raw_columns=400 common_raw_columns=136 native_raw_columns=264 history_rows=4885 history_failures=153 target_rows=716 target_failures=30 target_prevalence=0.041899 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.751410 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_duc raw_columns=381 common_raw_columns=136 native_raw_columns=245 history_rows=12783 history_failures=218 target_rows=1893 target_failures=53 target_prevalence=0.027998 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.748564 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_han_quoc raw_columns=333 common_raw_columns=136 native_raw_columns=197 history_rows=20006 history_failures=95 target_rows=3337 target_failures=29 target_prevalence=0.008690 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.684017 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_italia raw_columns=420 common_raw_columns=136 native_raw_columns=284 history_rows=12816 history_failures=437 target_rows=2024 target_failures=71 target_prevalence=0.035079 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.743313 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_nga raw_columns=469 common_raw_columns=136 native_raw_columns=333 history_rows=8196 history_failures=372 target_rows=1567 target_failures=83 target_prevalence=0.052967 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.791533 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_nhat raw_columns=362 common_raw_columns=136 native_raw_columns=226 history_rows=21883 history_failures=216 target_rows=2448 target_failures=11 target_prevalence=0.004493 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.741612 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_phap raw_columns=495 common_raw_columns=136 native_raw_columns=359 history_rows=17871 history_failures=220 target_rows=2204 target_failures=40 target_prevalence=0.018149 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.787196 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_tay_ban_nha raw_columns=396 common_raw_columns=136 native_raw_columns=260 history_rows=9426 history_failures=233 target_rows=1242 target_failures=42 target_prevalence=0.033816 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.746678 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_tieng_trung_quoc raw_columns=436 common_raw_columns=136 native_raw_columns=300 history_rows=40174 history_failures=220 target_rows=5584 target_failures=76 target_prevalence=0.013610 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.758496 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=2 faculty=khoa_viet_nam_hoc raw_columns=181 common_raw_columns=136 native_raw_columns=45 history_rows=6032 history_failures=367 target_rows=556 target_failures=39 target_prevalence=0.070144 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.497718 history_latest_semester=6066 target_first_semester=6067 availability_provenance=semester_proxy; source release timestamps not verified [SELECTION_ORIGINS] target=khoa_cong_nghe_thong_tin fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=2 origin=6061 ap=0.397890 rows=2815 failures=77 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.007521 ap=0.405411 base_ap=0.397890 donor_failures=77 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6061 delta_ap=0.037298 ap=0.435188 base_ap=0.397890 donor_failures=33 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6061 delta_ap=0.011650 ap=0.409539 base_ap=0.397890 donor_failures=52 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.047833 ap=0.445723 base_ap=0.397890 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6061 delta_ap=0.018777 ap=0.416666 base_ap=0.397890 donor_failures=77 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6061 delta_ap=0.029744 ap=0.427634 base_ap=0.397890 donor_failures=26 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6061 delta_ap=0.045580 ap=0.443469 base_ap=0.397890 donor_failures=161 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6061 delta_ap=0.022130 ap=0.420020 base_ap=0.397890 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6061 delta_ap=0.030799 ap=0.428688 base_ap=0.397890 donor_failures=62 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6061 delta_ap=-0.003527 ap=0.394362 base_ap=0.397890 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.050214 ap=0.448104 base_ap=0.397890 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.006967 ap=0.404857 base_ap=0.397890 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6061 delta_ap=0.033341 ap=0.431231 base_ap=0.397890 donor_failures=180 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=2 origin=6063 ap=0.053488 rows=5277 failures=63 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.043830 ap=0.097318 base_ap=0.053488 donor_failures=95 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6063 delta_ap=0.003218 ap=0.056706 base_ap=0.053488 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6063 delta_ap=0.018470 ap=0.071959 base_ap=0.053488 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.002096 ap=0.051393 base_ap=0.053488 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6063 delta_ap=0.001999 ap=0.055487 base_ap=0.053488 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6063 delta_ap=0.008937 ap=0.062425 base_ap=0.053488 donor_failures=13 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6063 delta_ap=0.009666 ap=0.063154 base_ap=0.053488 donor_failures=146 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6063 delta_ap=0.020122 ap=0.073611 base_ap=0.053488 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6063 delta_ap=0.016516 ap=0.070004 base_ap=0.053488 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6063 delta_ap=0.011061 ap=0.064549 base_ap=0.053488 donor_failures=50 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.000837 ap=0.054325 base_ap=0.053488 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.005063 ap=0.058552 base_ap=0.053488 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.006507 ap=0.046981 base_ap=0.053488 donor_failures=217 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=2 origin=6064 ap=0.286568 rows=3992 failures=116 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.000238 ap=0.286330 base_ap=0.286568 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.024844 ap=0.261724 base_ap=0.286568 donor_failures=33 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6064 delta_ap=0.000880 ap=0.287448 base_ap=0.286568 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.022939 ap=0.263629 base_ap=0.286568 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6064 delta_ap=-0.003895 ap=0.282673 base_ap=0.286568 donor_failures=50 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.006541 ap=0.280027 base_ap=0.286568 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6064 delta_ap=-0.034929 ap=0.251639 base_ap=0.286568 donor_failures=128 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6064 delta_ap=-0.001230 ap=0.285338 base_ap=0.286568 donor_failures=138 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6064 delta_ap=-0.034156 ap=0.252411 base_ap=0.286568 donor_failures=41 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6064 delta_ap=-0.010811 ap=0.275757 base_ap=0.286568 donor_failures=48 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.049583 ap=0.236985 base_ap=0.286568 donor_failures=79 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.038115 ap=0.248453 base_ap=0.286568 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.080810 ap=0.205758 base_ap=0.286568 donor_failures=269 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=2 origin=6066 ap=0.245112 rows=5222 failures=189 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.014255 ap=0.259367 base_ap=0.245112 donor_failures=129 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6066 delta_ap=0.050901 ap=0.296013 base_ap=0.245112 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6066 delta_ap=0.002933 ap=0.248045 base_ap=0.245112 donor_failures=71 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.000606 ap=0.245718 base_ap=0.245112 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6066 delta_ap=-0.021674 ap=0.223438 base_ap=0.245112 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6066 delta_ap=0.054261 ap=0.299373 base_ap=0.245112 donor_failures=18 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6066 delta_ap=0.015258 ap=0.260370 base_ap=0.245112 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6066 delta_ap=0.038037 ap=0.283149 base_ap=0.245112 donor_failures=156 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6066 delta_ap=-0.065629 ap=0.179483 base_ap=0.245112 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6066 delta_ap=0.016506 ap=0.261618 base_ap=0.245112 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.024102 ap=0.269214 base_ap=0.245112 donor_failures=78 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.013061 ap=0.258172 base_ap=0.245112 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6066 delta_ap=0.050094 ap=0.295206 base_ap=0.245112 donor_failures=257 [POLICY_REPLAY] fold=2 faculty=khoa_cong_nghe_thong_tin origin=6064 policy_sources=khoa_tieng_italia,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.286568 policy_ap=0.300336 delta_ap=0.013768 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_cong_nghe_thong_tin origin=6066 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_nga,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.245112 policy_ap=0.270204 delta_ap=0.025092 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_cong_nghe_thong_tin selected=['khoa_tieng_han_quoc', 'khoa_tieng_nga', 'khoa_quoc_te_hoc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.019430 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common rows=4598 failures=152 prevalence=0.033058 ap=0.366817 auc=0.877549 f2_at20=0.363220 brier=0.026001 ece10=0.009233 rho=0.000000 donor_rows=0 effective_n=24312.000000 seconds=0.635061 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common in_sample_bce=0.055273 local_bce=0.055273 donor_bce=nan in_sample_local_ap=0.805304 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 765} positive_ess=765.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native rows=4598 failures=152 prevalence=0.033058 ap=0.399018 auc=0.891874 f2_at20=0.386126 brier=0.025165 ece10=0.007711 rho=0.000000 donor_rows=0 effective_n=24312.000000 seconds=1.036548 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native in_sample_bce=0.054213 local_bce=0.054213 donor_bce=nan in_sample_local_ap=0.814881 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 765} positive_ess=765.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all rows=4598 failures=152 prevalence=0.033058 ap=0.349823 auc=0.864509 f2_at20=0.359948 brier=0.026175 ece10=0.010046 rho=0.250000 donor_rows=4095 effective_n=21356.475330 seconds=0.693778 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all in_sample_bce=0.054170 local_bce=0.060971 donor_bce=0.053747 in_sample_local_ap=0.762132 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03731023039365273, 'khoa_quoc_te_hoc': 0.012208193469636863, 'khoa_tieng_anh': 0.05355603448275862, 'khoa_tieng_bo_dao_nha': 0.004658414708574916, 'khoa_tieng_duc': 0.012190074763503204, 'khoa_tieng_han_quoc': 0.019078043942630456, 'khoa_tieng_italia': 0.012221544095209032, 'khoa_tieng_nga': 0.007815837656393043, 'khoa_tieng_nhat': 0.020867981385413487, 'khoa_tieng_phap': 0.01704207354287458, 'khoa_tieng_tay_ban_nha': 0.008988785474519378, 'khoa_tieng_trung_quoc': 0.03831057369545316, 'khoa_viet_nam_hoc': 0.005752212389380531} source_failures={'khoa_cong_nghe_thong_tin': 765, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 8, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 19} positive_ess=714.128369 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal rows=4598 failures=152 prevalence=0.033058 ap=0.359162 auc=0.867124 f2_at20=0.350131 brier=0.025852 ece10=0.007544 rho=0.250000 donor_rows=4095 effective_n=26042.183530 seconds=0.695593 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal in_sample_bce=0.058169 local_bce=0.062229 donor_bce=0.045990 in_sample_local_ap=0.754720 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal mass={'khoa_cong_nghe_thong_tin': 0.7499999999999999, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_cong_nghe_thong_tin': 765, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 8, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 19} positive_ess=794.853470 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior rows=4598 failures=152 prevalence=0.033058 ap=0.366817 auc=0.877549 f2_at20=0.363220 brier=0.026001 ece10=0.009233 rho=0.000000 donor_rows=0 effective_n=24312.000000 seconds=0.652227 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior in_sample_bce=0.055273 local_bce=0.055273 donor_bce=nan in_sample_local_ap=0.805304 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 765} positive_ess=765.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor rows=4598 failures=152 prevalence=0.033058 ap=0.385705 auc=0.878121 f2_at20=0.373037 brier=0.025587 ece10=0.008416 rho=0.250000 donor_rows=4095 effective_n=26042.183530 seconds=0.740410 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor in_sample_bce=0.055017 local_bce=0.060339 donor_bce=0.039051 in_sample_local_ap=0.770453 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor mass={'khoa_cong_nghe_thong_tin': 0.7499999999999999, np.str_('khoa_tieng_bo_dao_nha'): 0.08333333333333333, np.str_('khoa_tieng_phap'): 0.08333333333333333, np.str_('khoa_tieng_duc'): 0.08333333333333333} source_failures={'khoa_cong_nghe_thong_tin': 765, np.str_('khoa_tieng_bo_dao_nha'): 35, np.str_('khoa_tieng_phap'): 17, np.str_('khoa_tieng_duc'): 20} positive_ess=786.578190 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common rows=4598 failures=152 prevalence=0.033058 ap=0.405384 auc=0.874804 f2_at20=0.356675 brier=0.025429 ece10=0.007886 rho=0.250000 donor_rows=4095 effective_n=25926.789867 seconds=0.733048 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common in_sample_bce=0.056891 local_bce=0.061742 donor_bce=0.042724 in_sample_local_ap=0.760729 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_tieng_han_quoc': 0.09309063863342315, 'khoa_tieng_nga': 0.08518068909534625, 'khoa_quoc_te_hoc': 0.07172867227123057} source_failures={'khoa_cong_nghe_thong_tin': 765, 'khoa_tieng_han_quoc': 6, 'khoa_tieng_nga': 67, 'khoa_quoc_te_hoc': 15} positive_ess=793.493485 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native rows=4598 failures=152 prevalence=0.033058 ap=0.423374 auc=0.888116 f2_at20=0.376309 brier=0.025056 ece10=0.007925 rho=0.250000 donor_rows=4095 effective_n=25926.789867 seconds=1.179562 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native in_sample_bce=0.055805 local_bce=0.060282 donor_bce=0.042608 in_sample_local_ap=0.770216 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_tieng_han_quoc': 0.09309063863342315, 'khoa_tieng_nga': 0.08518068909534625, 'khoa_quoc_te_hoc': 0.07172867227123057} source_failures={'khoa_cong_nghe_thong_tin': 765, 'khoa_tieng_han_quoc': 6, 'khoa_tieng_nga': 67, 'khoa_quoc_te_hoc': 15} positive_ess=793.493485 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator rows=4598 failures=152 prevalence=0.033058 ap=0.399018 auc=0.891874 f2_at20=0.386126 brier=0.025165 ece10=0.007711 rho=0.000000 donor_rows=0 effective_n=24312.000000 seconds=1.802799 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator in_sample_bce=0.054213 local_bce=0.054213 donor_bce=nan in_sample_local_ap=0.814881 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 765} positive_ess=765.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator rows=4598 failures=152 prevalence=0.033058 ap=0.389809 auc=0.885424 f2_at20=0.379581 brier=0.025494 ece10=0.009182 rho=0.000000 donor_rows=0 effective_n=24312.000000 seconds=1.871017 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator in_sample_bce=0.054215 local_bce=0.054215 donor_bce=nan in_sample_local_ap=0.814045 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 765} positive_ess=765.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator rows=4598 failures=152 prevalence=0.033058 ap=0.399018 auc=0.891874 f2_at20=0.386126 brier=0.025165 ece10=0.007711 rho=0.000000 donor_rows=0 effective_n=24312.000000 seconds=1.819041 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator in_sample_bce=0.054213 local_bce=0.054213 donor_bce=nan in_sample_local_ap=0.814881 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 765} positive_ess=765.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan rows=4598 failures=152 prevalence=0.033058 ap=0.423374 auc=0.888116 f2_at20=0.376309 brier=0.025056 ece10=0.007925 rho=0.250000 donor_rows=4095 effective_n=25926.789867 seconds=2.119936 [TRAINING_INSIGHT] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan in_sample_bce=0.055805 local_bce=0.060282 donor_bce=0.042608 in_sample_local_ap=0.770216 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_tieng_han_quoc': 0.09309063863342315, 'khoa_tieng_nga': 0.08518068909534625, 'khoa_quoc_te_hoc': 0.07172867227123057} source_failures={'khoa_cong_nghe_thong_tin': 765, 'khoa_tieng_han_quoc': 6, 'khoa_tieng_nga': 67, 'khoa_quoc_te_hoc': 15} positive_ess=793.493485 [FACULTY_RESULTS] fold=2 faculty=khoa_cong_nghe_thong_tin model=committee_reference ap=0.472860 [FACULTY_COMPLETE] fold=2 faculty=khoa_cong_nghe_thong_tin models=13 positive_rows=152 num_edges=52 num_replays=2 chosen=['khoa_tieng_han_quoc', 'khoa_tieng_nga', 'khoa_quoc_te_hoc'] [SELECTION_ORIGINS] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=2 origins=[6063, 6064, 6065, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=2 origin=6063 ap=0.317434 rows=6115 failures=157 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.020440 ap=0.337874 base_ap=0.317434 donor_failures=169 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.071885 ap=0.245549 base_ap=0.317434 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6063 delta_ap=-0.105883 ap=0.211551 base_ap=0.317434 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.013658 ap=0.331092 base_ap=0.317434 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6063 delta_ap=-0.012279 ap=0.305155 base_ap=0.317434 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.047016 ap=0.270418 base_ap=0.317434 donor_failures=20 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6063 delta_ap=-0.028741 ap=0.288692 base_ap=0.317434 donor_failures=139 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6063 delta_ap=0.017549 ap=0.334982 base_ap=0.317434 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6063 delta_ap=-0.031478 ap=0.285956 base_ap=0.317434 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6063 delta_ap=-0.030466 ap=0.286968 base_ap=0.317434 donor_failures=55 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.009557 ap=0.307877 base_ap=0.317434 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.031626 ap=0.285808 base_ap=0.317434 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.001566 ap=0.315868 base_ap=0.317434 donor_failures=217 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=2 origin=6064 ap=0.600178 rows=4784 failures=257 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.047958 ap=0.552220 base_ap=0.600178 donor_failures=115 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.045720 ap=0.554458 base_ap=0.600178 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6064 delta_ap=-0.030473 ap=0.569706 base_ap=0.600178 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.068503 ap=0.531676 base_ap=0.600178 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6064 delta_ap=-0.026993 ap=0.573186 base_ap=0.600178 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.034221 ap=0.565958 base_ap=0.600178 donor_failures=18 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6064 delta_ap=-0.027432 ap=0.572746 base_ap=0.600178 donor_failures=113 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6064 delta_ap=-0.011275 ap=0.588903 base_ap=0.600178 donor_failures=146 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6064 delta_ap=-0.041658 ap=0.558520 base_ap=0.600178 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6064 delta_ap=-0.023200 ap=0.576978 base_ap=0.600178 donor_failures=43 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.033871 ap=0.566308 base_ap=0.600178 donor_failures=76 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.043752 ap=0.556426 base_ap=0.600178 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.038713 ap=0.561466 base_ap=0.600178 donor_failures=275 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=2 origin=6065 ap=0.634407 rows=687 failures=21 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=-0.124875 ap=0.509532 base_ap=0.634407 donor_failures=114 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6065 delta_ap=-0.155870 ap=0.478537 base_ap=0.634407 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6065 delta_ap=-0.074295 ap=0.560112 base_ap=0.634407 donor_failures=84 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=-0.139648 ap=0.494759 base_ap=0.634407 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6065 delta_ap=-0.123478 ap=0.510928 base_ap=0.634407 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6065 delta_ap=-0.119734 ap=0.514673 base_ap=0.634407 donor_failures=18 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6065 delta_ap=-0.184149 ap=0.450257 base_ap=0.634407 donor_failures=131 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6065 delta_ap=-0.159754 ap=0.474652 base_ap=0.634407 donor_failures=152 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6065 delta_ap=-0.123019 ap=0.511388 base_ap=0.634407 donor_failures=45 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6065 delta_ap=-0.065021 ap=0.569385 base_ap=0.634407 donor_failures=53 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=-0.089240 ap=0.545167 base_ap=0.634407 donor_failures=79 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6065 delta_ap=-0.110328 ap=0.524078 base_ap=0.634407 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6065 delta_ap=-0.104701 ap=0.529705 base_ap=0.634407 donor_failures=261 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=2 origin=6066 ap=0.497606 rows=5672 failures=219 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.025313 ap=0.522919 base_ap=0.497606 donor_failures=130 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6066 delta_ap=0.014180 ap=0.511787 base_ap=0.497606 donor_failures=48 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6066 delta_ap=-0.005293 ap=0.492313 base_ap=0.497606 donor_failures=57 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.008758 ap=0.506365 base_ap=0.497606 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6066 delta_ap=0.077575 ap=0.575181 base_ap=0.497606 donor_failures=48 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6066 delta_ap=0.041651 ap=0.539257 base_ap=0.497606 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6066 delta_ap=0.029218 ap=0.526824 base_ap=0.497606 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6066 delta_ap=0.003471 ap=0.501077 base_ap=0.497606 donor_failures=162 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6066 delta_ap=-0.021238 ap=0.476368 base_ap=0.497606 donor_failures=33 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6066 delta_ap=0.028009 ap=0.525615 base_ap=0.497606 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.041286 ap=0.538893 base_ap=0.497606 donor_failures=93 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.009093 ap=0.506699 base_ap=0.497606 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6066 delta_ap=0.058634 ap=0.556240 base_ap=0.497606 donor_failures=266 [POLICY_REPLAY] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6065 policy_sources=khoa_tieng_nga policy_donor_count=1 base_ap=0.634407 policy_ap=0.474652 delta_ap=-0.159754 policy_donor_rows=4096 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 policy_sources= policy_donor_count=0 base_ap=0.497606 policy_ap=0.497606 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6063,6064,6065 [SELECTION_FINAL] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich selected=[] reason=no_stable_positive_directed_edge replay_mean=-0.079877 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common rows=5530 failures=166 prevalence=0.030018 ap=0.445227 auc=0.897112 f2_at20=0.353107 brier=0.021523 ece10=0.004841 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=0.804884 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common in_sample_bce=0.048559 local_bce=0.048559 donor_bce=nan in_sample_local_ap=0.800873 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native rows=5530 failures=166 prevalence=0.030018 ap=0.441179 auc=0.901066 f2_at20=0.370056 brier=0.021675 ece10=0.005940 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=1.434430 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native in_sample_bce=0.048261 local_bce=0.048261 donor_bce=nan in_sample_local_ap=0.802564 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all rows=5530 failures=166 prevalence=0.030018 ap=0.414574 auc=0.888720 f2_at20=0.353107 brier=0.022550 ece10=0.005772 rho=0.250000 donor_rows=4095 effective_n=26358.018815 seconds=0.814384 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all in_sample_bce=0.048755 local_bce=0.054816 donor_bce=0.050310 in_sample_local_ap=0.743177 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.024572766194859853, 'khoa_quoc_te_hoc': 0.012939311978718157, 'khoa_tieng_anh': 0.05676337291335654, 'khoa_tieng_bo_dao_nha': 0.004937395642558834, 'khoa_tieng_duc': 0.012920108188092033, 'khoa_tieng_han_quoc': 0.0202205808034866, 'khoa_tieng_italia': 0.012953462140232144, 'khoa_tieng_nga': 0.008283908840616624, 'khoa_tieng_nhat': 0.02211771317218321, 'khoa_tieng_phap': 0.01806268117260367, 'khoa_tieng_tay_ban_nha': 0.009527101602202574, 'khoa_tieng_trung_quoc': 0.04060489919020647, 'khoa_viet_nam_hoc': 0.006096698160883294} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 19, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 17} positive_ess=849.746160 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal rows=5530 failures=166 prevalence=0.030018 ap=0.421911 auc=0.891601 f2_at20=0.361582 brier=0.022047 ece10=0.002902 rho=0.250000 donor_rows=4095 effective_n=33738.747039 seconds=0.834621 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal in_sample_bce=0.051808 local_bce=0.056211 donor_bce=0.038597 in_sample_local_ap=0.734110 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 19, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 17} positive_ess=985.094917 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior rows=5530 failures=166 prevalence=0.030018 ap=0.445227 auc=0.897112 f2_at20=0.353107 brier=0.021523 ece10=0.004841 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=0.816265 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior in_sample_bce=0.048559 local_bce=0.048559 donor_bce=nan in_sample_local_ap=0.800873 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor rows=5530 failures=166 prevalence=0.030018 ap=0.420928 auc=0.900020 f2_at20=0.358757 brier=0.022042 ece10=0.005692 rho=0.250000 donor_rows=4096 effective_n=33742.989117 seconds=0.835200 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor in_sample_bce=0.048782 local_bce=0.053738 donor_bce=0.033914 in_sample_local_ap=0.749934 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.25} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134, np.str_('khoa_tieng_tay_ban_nha'): 115} positive_ess=978.531825 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common rows=5530 failures=166 prevalence=0.030018 ap=0.445227 auc=0.897112 f2_at20=0.353107 brier=0.021523 ece10=0.004841 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=0.809137 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common in_sample_bce=0.048559 local_bce=0.048559 donor_bce=nan in_sample_local_ap=0.800873 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native rows=5530 failures=166 prevalence=0.030018 ap=0.441179 auc=0.901066 f2_at20=0.370056 brier=0.021675 ece10=0.005940 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=1.472133 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native in_sample_bce=0.048261 local_bce=0.048261 donor_bce=nan in_sample_local_ap=0.802564 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator rows=5530 failures=166 prevalence=0.030018 ap=0.441179 auc=0.901066 f2_at20=0.370056 brier=0.021675 ece10=0.005940 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=2.688109 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator in_sample_bce=0.048261 local_bce=0.048261 donor_bce=nan in_sample_local_ap=0.802564 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator rows=5530 failures=166 prevalence=0.030018 ap=0.445616 auc=0.903387 f2_at20=0.372881 brier=0.021635 ece10=0.004772 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=2.761332 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator in_sample_bce=0.048368 local_bce=0.048368 donor_bce=nan in_sample_local_ap=0.801105 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator rows=5530 failures=166 prevalence=0.030018 ap=0.441179 auc=0.901066 f2_at20=0.370056 brier=0.021675 ece10=0.005940 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=2.759165 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator in_sample_bce=0.048261 local_bce=0.048261 donor_bce=nan in_sample_local_ap=0.802564 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan rows=5530 failures=166 prevalence=0.030018 ap=0.441179 auc=0.901066 f2_at20=0.370056 brier=0.021675 ece10=0.005940 rho=0.000000 donor_rows=0 effective_n=39125.000000 seconds=2.666940 [TRAINING_INSIGHT] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan in_sample_bce=0.048261 local_bce=0.048261 donor_bce=nan in_sample_local_ap=0.802564 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1134} positive_ess=1134.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=committee_reference ap=0.549834 [FACULTY_COMPLETE] fold=2 faculty=khoa_quan_tri_kinh_doanh_va_du_lich models=13 positive_rows=166 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quoc_te_hoc fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_quoc_te_hoc fold=2 origin=6061 ap=0.573303 rows=1641 failures=12 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.151970 ap=0.725273 base_ap=0.573303 donor_failures=178 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.082154 ap=0.655457 base_ap=0.573303 donor_failures=93 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6061 delta_ap=0.111364 ap=0.684667 base_ap=0.573303 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.001806 ap=0.571497 base_ap=0.573303 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6061 delta_ap=0.096182 ap=0.669485 base_ap=0.573303 donor_failures=80 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6061 delta_ap=0.070477 ap=0.643780 base_ap=0.573303 donor_failures=24 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6061 delta_ap=0.071803 ap=0.645106 base_ap=0.573303 donor_failures=163 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6061 delta_ap=0.047893 ap=0.621196 base_ap=0.573303 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6061 delta_ap=0.091255 ap=0.664558 base_ap=0.573303 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6061 delta_ap=0.066064 ap=0.639367 base_ap=0.573303 donor_failures=64 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.018758 ap=0.554545 base_ap=0.573303 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.099046 ap=0.672349 base_ap=0.573303 donor_failures=32 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.019629 ap=0.553674 base_ap=0.573303 donor_failures=180 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=2 origin=6063 ap=0.079597 rows=2231 failures=17 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.065935 ap=0.145532 base_ap=0.079597 donor_failures=151 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.022941 ap=0.102538 base_ap=0.079597 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6063 delta_ap=0.134245 ap=0.213842 base_ap=0.079597 donor_failures=72 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.086545 ap=0.166142 base_ap=0.079597 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6063 delta_ap=0.143486 ap=0.223083 base_ap=0.079597 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.031973 ap=0.111570 base_ap=0.079597 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6063 delta_ap=0.124026 ap=0.203623 base_ap=0.079597 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6063 delta_ap=0.099856 ap=0.179454 base_ap=0.079597 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6063 delta_ap=0.109079 ap=0.188677 base_ap=0.079597 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6063 delta_ap=0.006536 ap=0.086133 base_ap=0.079597 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.178915 ap=0.258513 base_ap=0.079597 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.143722 ap=0.223320 base_ap=0.079597 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.035645 ap=0.115242 base_ap=0.079597 donor_failures=217 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=2 origin=6064 ap=0.317214 rows=1690 failures=30 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.054941 ap=0.372155 base_ap=0.317214 donor_failures=132 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.047574 ap=0.364787 base_ap=0.317214 donor_failures=99 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6064 delta_ap=0.078645 ap=0.395859 base_ap=0.317214 donor_failures=73 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.024600 ap=0.341814 base_ap=0.317214 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6064 delta_ap=0.018138 ap=0.335351 base_ap=0.317214 donor_failures=55 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.011746 ap=0.305467 base_ap=0.317214 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6064 delta_ap=0.030460 ap=0.347674 base_ap=0.317214 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6064 delta_ap=0.038184 ap=0.355397 base_ap=0.317214 donor_failures=151 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6064 delta_ap=0.018294 ap=0.335508 base_ap=0.317214 donor_failures=38 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6064 delta_ap=0.070579 ap=0.387793 base_ap=0.317214 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.012142 ap=0.305072 base_ap=0.317214 donor_failures=72 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.037378 ap=0.354592 base_ap=0.317214 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6064 delta_ap=0.054919 ap=0.372132 base_ap=0.317214 donor_failures=271 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=2 origin=6066 ap=0.034602 rows=2363 failures=31 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.014754 ap=0.049356 base_ap=0.034602 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.016142 ap=0.050744 base_ap=0.034602 donor_failures=96 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6066 delta_ap=0.010751 ap=0.045353 base_ap=0.034602 donor_failures=68 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.014648 ap=0.049250 base_ap=0.034602 donor_failures=121 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.050057 ap=0.084659 base_ap=0.034602 donor_failures=73 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.001400 ap=0.033202 base_ap=0.034602 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6066 delta_ap=0.009119 ap=0.043721 base_ap=0.034602 donor_failures=118 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.016818 ap=0.051420 base_ap=0.034602 donor_failures=164 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.003583 ap=0.031019 base_ap=0.034602 donor_failures=49 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6066 delta_ap=0.013167 ap=0.047769 base_ap=0.034602 donor_failures=43 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.002627 ap=0.031975 base_ap=0.034602 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.061318 ap=0.095920 base_ap=0.034602 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.023378 ap=0.057980 base_ap=0.034602 donor_failures=266 [POLICY_REPLAY] fold=2 faculty=khoa_quoc_te_hoc origin=6064 policy_sources=khoa_tieng_anh,khoa_tieng_trung_quoc,khoa_tieng_duc policy_donor_count=3 base_ap=0.317214 policy_ap=0.357525 delta_ap=0.040311 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_quoc_te_hoc origin=6066 policy_sources=khoa_tieng_anh,khoa_tieng_trung_quoc,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.034602 policy_ap=0.059844 delta_ap=0.025242 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_quoc_te_hoc selected=['khoa_tieng_trung_quoc', 'khoa_tieng_anh', 'khoa_tieng_duc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.032777 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_local_common rows=2095 failures=19 prevalence=0.009069 ap=0.396754 auc=0.912205 f2_at20=0.171717 brier=0.006856 ece10=0.003584 rho=0.000000 donor_rows=0 effective_n=12802.000000 seconds=0.524225 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=hgb_local_common in_sample_bce=0.007388 local_bce=0.007388 donor_bce=nan in_sample_local_ap=0.984005 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_local_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 129} positive_ess=129.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_local_native rows=2095 failures=19 prevalence=0.009069 ap=0.432153 auc=0.897145 f2_at20=0.151515 brier=0.006736 ece10=0.003056 rho=0.000000 donor_rows=0 effective_n=12802.000000 seconds=0.730852 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=hgb_local_native in_sample_bce=0.007305 local_bce=0.007305 donor_bce=nan in_sample_local_ap=0.984959 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_local_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 129} positive_ess=129.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_pool_all rows=2095 failures=19 prevalence=0.009069 ap=0.335683 auc=0.922270 f2_at20=0.151515 brier=0.007108 ece10=0.004568 rho=0.250000 donor_rows=4095 effective_n=14968.484990 seconds=0.586757 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=hgb_pool_all in_sample_bce=0.014618 local_bce=0.011746 donor_bce=0.042852 in_sample_local_ap=0.928943 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_pool_all mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02220923009463953, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03574103847699785, 'khoa_tieng_anh': 0.0513035773011291, 'khoa_tieng_bo_dao_nha': 0.004462491321664779, 'khoa_tieng_duc': 0.011677385171922388, 'khoa_tieng_han_quoc': 0.018275660466985786, 'khoa_tieng_italia': 0.011707530967954105, 'khoa_tieng_nga': 0.007487119523513721, 'khoa_tieng_nhat': 0.019990316804911024, 'khoa_tieng_phap': 0.01632531881463076, 'khoa_tieng_tay_ban_nha': 0.008610735557423173, 'khoa_tieng_trung_quoc': 0.03669930938721818, 'khoa_viet_nam_hoc': 0.00551028611100961} source_failures={'khoa_quoc_te_hoc': 129, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 17, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 20} positive_ess=189.402349 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_pool_equal rows=2095 failures=19 prevalence=0.009069 ap=0.286705 auc=0.898083 f2_at20=0.151515 brier=0.007557 ece10=0.004539 rho=0.250000 donor_rows=4095 effective_n=16891.617295 seconds=0.577825 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=hgb_pool_equal in_sample_bce=0.017576 local_bce=0.012788 donor_bce=0.031940 in_sample_local_ap=0.916656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_pool_equal mass={'khoa_quoc_te_hoc': 0.7499999999999999, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quoc_te_hoc': 129, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 17, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 20} positive_ess=240.897903 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_language_prior rows=2095 failures=19 prevalence=0.009069 ap=0.396754 auc=0.912205 f2_at20=0.171717 brier=0.006856 ece10=0.003584 rho=0.000000 donor_rows=0 effective_n=12802.000000 seconds=0.523788 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=hgb_language_prior in_sample_bce=0.007388 local_bce=0.007388 donor_bce=nan in_sample_local_ap=0.984005 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_language_prior mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 129} positive_ess=129.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_random_donor rows=2095 failures=19 prevalence=0.009069 ap=0.351767 auc=0.889134 f2_at20=0.151515 brier=0.007361 ece10=0.005118 rho=0.250000 donor_rows=4095 effective_n=16891.617295 seconds=0.569686 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=hgb_random_donor in_sample_bce=0.022394 local_bce=0.013403 donor_bce=0.049366 in_sample_local_ap=0.907659 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=hgb_random_donor mass={'khoa_quoc_te_hoc': 0.7499999999999999, np.str_('khoa_viet_nam_hoc'): 0.08333333333333333, np.str_('khoa_tieng_han_quoc'): 0.08333333333333333, np.str_('khoa_tieng_nga'): 0.08333333333333333} source_failures={'khoa_quoc_te_hoc': 129, np.str_('khoa_viet_nam_hoc'): 84, np.str_('khoa_tieng_han_quoc'): 10, np.str_('khoa_tieng_nga'): 58} positive_ess=280.881907 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_common rows=2095 failures=19 prevalence=0.009069 ap=0.420686 auc=0.928050 f2_at20=0.181818 brier=0.006886 ece10=0.002408 rho=0.250000 donor_rows=4095 effective_n=16883.048413 seconds=0.601452 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_common in_sample_bce=0.012710 local_bce=0.011030 donor_bce=0.017900 in_sample_local_ap=0.939799 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_common mass={'khoa_quoc_te_hoc': 0.7500000000000001, 'khoa_tieng_trung_quoc': 0.0867247110983875, 'khoa_tieng_anh': 0.08508453280726176, 'khoa_tieng_duc': 0.07819075609435074} source_failures={'khoa_quoc_te_hoc': 129, 'khoa_tieng_trung_quoc': 8, 'khoa_tieng_anh': 30, 'khoa_tieng_duc': 23} positive_ess=189.835356 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_native rows=2095 failures=19 prevalence=0.009069 ap=0.438126 auc=0.924196 f2_at20=0.181818 brier=0.006624 ece10=0.002694 rho=0.250000 donor_rows=4095 effective_n=16883.048413 seconds=0.915896 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_native in_sample_bce=0.012567 local_bce=0.010922 donor_bce=0.017671 in_sample_local_ap=0.942980 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_native mass={'khoa_quoc_te_hoc': 0.7500000000000001, 'khoa_tieng_trung_quoc': 0.0867247110983875, 'khoa_tieng_anh': 0.08508453280726176, 'khoa_tieng_duc': 0.07819075609435074} source_failures={'khoa_quoc_te_hoc': 129, 'khoa_tieng_trung_quoc': 8, 'khoa_tieng_anh': 30, 'khoa_tieng_duc': 23} positive_ess=189.835356 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator rows=2095 failures=19 prevalence=0.009069 ap=0.432153 auc=0.897145 f2_at20=0.151515 brier=0.006736 ece10=0.003056 rho=0.000000 donor_rows=0 effective_n=12802.000000 seconds=1.335598 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator in_sample_bce=0.007305 local_bce=0.007305 donor_bce=nan in_sample_local_ap=0.984959 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 129} positive_ess=129.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator rows=2095 failures=19 prevalence=0.009069 ap=0.432153 auc=0.897145 f2_at20=0.151515 brier=0.006736 ece10=0.003056 rho=0.000000 donor_rows=0 effective_n=12802.000000 seconds=1.324827 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator in_sample_bce=0.007305 local_bce=0.007305 donor_bce=nan in_sample_local_ap=0.984959 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 129} positive_ess=129.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=native_placebo_indicator rows=2095 failures=19 prevalence=0.009069 ap=0.432153 auc=0.897145 f2_at20=0.151515 brier=0.006736 ece10=0.003056 rho=0.000000 donor_rows=0 effective_n=12802.000000 seconds=1.314477 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=native_placebo_indicator in_sample_bce=0.007305 local_bce=0.007305 donor_bce=nan in_sample_local_ap=0.984959 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=native_placebo_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 129} positive_ess=129.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan rows=2095 failures=19 prevalence=0.009069 ap=0.438126 auc=0.924196 f2_at20=0.181818 brier=0.006624 ece10=0.002694 rho=0.250000 donor_rows=4095 effective_n=16883.048413 seconds=1.412227 [TRAINING_INSIGHT] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.012567 local_bce=0.010922 donor_bce=0.017671 in_sample_local_ap=0.942980 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan mass={'khoa_quoc_te_hoc': 0.7500000000000001, 'khoa_tieng_trung_quoc': 0.0867247110983875, 'khoa_tieng_anh': 0.08508453280726176, 'khoa_tieng_duc': 0.07819075609435074} source_failures={'khoa_quoc_te_hoc': 129, 'khoa_tieng_trung_quoc': 8, 'khoa_tieng_anh': 30, 'khoa_tieng_duc': 23} positive_ess=189.835356 [FACULTY_RESULTS] fold=2 faculty=khoa_quoc_te_hoc model=committee_reference ap=0.466005 [FACULTY_COMPLETE] fold=2 faculty=khoa_quoc_te_hoc models=13 positive_rows=19 num_edges=52 num_replays=2 chosen=['khoa_tieng_trung_quoc', 'khoa_tieng_anh', 'khoa_tieng_duc'] [SELECTION_ORIGINS] target=khoa_tieng_anh fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_anh fold=2 origin=6061 ap=0.446307 rows=8489 failures=245 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.036927 ap=0.483234 base_ap=0.446307 donor_failures=179 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.067304 ap=0.513610 base_ap=0.446307 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6061 delta_ap=0.101997 ap=0.548304 base_ap=0.446307 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.132201 ap=0.578508 base_ap=0.446307 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_duc origin=6061 delta_ap=0.051943 ap=0.498249 base_ap=0.446307 donor_failures=85 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6061 delta_ap=0.103221 ap=0.549528 base_ap=0.446307 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_italia origin=6061 delta_ap=0.171177 ap=0.617483 base_ap=0.446307 donor_failures=159 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nga origin=6061 delta_ap=0.042733 ap=0.489039 base_ap=0.446307 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6061 delta_ap=0.148204 ap=0.594511 base_ap=0.446307 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_phap origin=6061 delta_ap=0.058715 ap=0.505021 base_ap=0.446307 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.021010 ap=0.467316 base_ap=0.446307 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.082691 ap=0.528997 base_ap=0.446307 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.075261 ap=0.371045 base_ap=0.446307 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_anh fold=2 origin=6063 ap=0.233358 rows=9030 failures=67 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.102459 ap=0.335818 base_ap=0.233358 donor_failures=148 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.070555 ap=0.303913 base_ap=0.233358 donor_failures=76 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6063 delta_ap=0.179947 ap=0.413306 base_ap=0.233358 donor_failures=35 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.023005 ap=0.210353 base_ap=0.233358 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_duc origin=6063 delta_ap=0.010429 ap=0.243787 base_ap=0.233358 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.008011 ap=0.225347 base_ap=0.233358 donor_failures=13 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_italia origin=6063 delta_ap=0.106008 ap=0.339366 base_ap=0.233358 donor_failures=143 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nga origin=6063 delta_ap=0.013422 ap=0.246780 base_ap=0.233358 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6063 delta_ap=0.111381 ap=0.344739 base_ap=0.233358 donor_failures=64 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_phap origin=6063 delta_ap=-0.012431 ap=0.220927 base_ap=0.233358 donor_failures=57 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.084827 ap=0.148531 base_ap=0.233358 donor_failures=92 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.021158 ap=0.212200 base_ap=0.233358 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6063 delta_ap=0.092199 ap=0.325557 base_ap=0.233358 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_anh fold=2 origin=6064 ap=0.576004 rows=9726 failures=240 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.026854 ap=0.549150 base_ap=0.576004 donor_failures=133 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.056714 ap=0.519291 base_ap=0.576004 donor_failures=107 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.023309 ap=0.552696 base_ap=0.576004 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.038481 ap=0.537523 base_ap=0.576004 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_duc origin=6064 delta_ap=-0.015831 ap=0.560173 base_ap=0.576004 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.010534 ap=0.565471 base_ap=0.576004 donor_failures=20 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_italia origin=6064 delta_ap=-0.021194 ap=0.554810 base_ap=0.576004 donor_failures=118 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nga origin=6064 delta_ap=-0.026681 ap=0.549324 base_ap=0.576004 donor_failures=144 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6064 delta_ap=-0.021621 ap=0.554384 base_ap=0.576004 donor_failures=49 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_phap origin=6064 delta_ap=-0.006974 ap=0.569030 base_ap=0.576004 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.021550 ap=0.554454 base_ap=0.576004 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.028504 ap=0.547501 base_ap=0.576004 donor_failures=33 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.032637 ap=0.543367 base_ap=0.576004 donor_failures=267 [SCREEN_BASE] target=khoa_tieng_anh fold=2 origin=6066 ap=0.505803 rows=8164 failures=148 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.088250 ap=0.594052 base_ap=0.505803 donor_failures=136 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.074603 ap=0.580406 base_ap=0.505803 donor_failures=117 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6066 delta_ap=0.079382 ap=0.585185 base_ap=0.505803 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.004202 ap=0.510005 base_ap=0.505803 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_duc origin=6066 delta_ap=0.067351 ap=0.573153 base_ap=0.505803 donor_failures=64 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6066 delta_ap=0.058768 ap=0.564571 base_ap=0.505803 donor_failures=14 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_italia origin=6066 delta_ap=0.085486 ap=0.591289 base_ap=0.505803 donor_failures=133 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nga origin=6066 delta_ap=0.090098 ap=0.595901 base_ap=0.505803 donor_failures=157 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6066 delta_ap=-0.026462 ap=0.479341 base_ap=0.505803 donor_failures=52 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_phap origin=6066 delta_ap=0.019574 ap=0.525377 base_ap=0.505803 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.076382 ap=0.582185 base_ap=0.505803 donor_failures=84 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.075304 ap=0.581107 base_ap=0.505803 donor_failures=18 [TRANSFER_EDGE] fold=2 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6066 delta_ap=0.063482 ap=0.569285 base_ap=0.505803 donor_failures=264 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_anh origin=6064 policy_sources=khoa_quoc_te_hoc,khoa_tieng_italia,khoa_tieng_nhat policy_donor_count=3 base_ap=0.576004 policy_ap=0.533260 delta_ap=-0.042744 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_anh origin=6066 policy_sources=khoa_quoc_te_hoc,khoa_tieng_italia,khoa_tieng_nhat policy_donor_count=3 base_ap=0.505803 policy_ap=0.594153 delta_ap=0.088350 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_anh selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.022803 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=hgb_local_common rows=8328 failures=122 prevalence=0.014649 ap=0.698511 auc=0.966434 f2_at20=0.271588 brier=0.007810 ece10=0.008234 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=0.967432 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=hgb_local_common in_sample_bce=0.021639 local_bce=0.021639 donor_bce=nan in_sample_local_ap=0.907001 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=hgb_local_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=hgb_local_native rows=8328 failures=122 prevalence=0.014649 ap=0.697400 auc=0.969858 f2_at20=0.271588 brier=0.007413 ece10=0.006625 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=2.734703 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=hgb_local_native in_sample_bce=0.021238 local_bce=0.021238 donor_bce=nan in_sample_local_ap=0.910760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=hgb_local_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=hgb_pool_all rows=8328 failures=122 prevalence=0.014649 ap=0.649384 auc=0.956975 f2_at20=0.259981 brier=0.008099 ece10=0.007500 rho=0.250000 donor_rows=4095 effective_n=32133.080249 seconds=0.971786 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=hgb_pool_all in_sample_bce=0.029424 local_bce=0.029286 donor_bce=0.047259 in_sample_local_ap=0.823252 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=hgb_pool_all mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.026390402542648853, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.042469747428477145, 'khoa_quoc_te_hoc': 0.013896427005223371, 'khoa_tieng_bo_dao_nha': 0.005302612554328712, 'khoa_tieng_duc': 0.013875802718932226, 'khoa_tieng_han_quoc': 0.021716287975823995, 'khoa_tieng_italia': 0.01391162384775369, 'khoa_tieng_nga': 0.008896665812748849, 'khoa_tieng_nhat': 0.02375375036363873, 'khoa_tieng_phap': 0.019398769489950545, 'khoa_tieng_tay_ban_nha': 0.010231816977912474, 'khoa_tieng_trung_quoc': 0.04360842512949881, 'khoa_viet_nam_hoc': 0.006547668153062598} source_failures={'khoa_tieng_anh': 969, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 19} positive_ess=553.353262 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=hgb_pool_equal rows=8328 failures=122 prevalence=0.014649 ap=0.647387 auc=0.956567 f2_at20=0.264624 brier=0.007939 ece10=0.006543 rho=0.250000 donor_rows=4095 effective_n=39559.524383 seconds=1.044130 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=hgb_pool_equal in_sample_bce=0.031676 local_bce=0.029837 donor_bce=0.037193 in_sample_local_ap=0.823771 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=hgb_pool_equal mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_anh': 969, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 19} positive_ess=662.724105 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=hgb_language_prior rows=8328 failures=122 prevalence=0.014649 ap=0.692645 auc=0.961118 f2_at20=0.262303 brier=0.007315 ece10=0.007164 rho=0.250000 donor_rows=4095 effective_n=39559.524383 seconds=0.997574 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=hgb_language_prior in_sample_bce=0.028472 local_bce=0.028291 donor_bce=0.029015 in_sample_local_ap=0.841842 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=hgb_language_prior mass={'khoa_tieng_anh': 0.75, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_anh': 969, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 11, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 18, 'khoa_tieng_nga': 26, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 3} positive_ess=668.196161 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=hgb_random_donor rows=8328 failures=122 prevalence=0.014649 ap=0.684629 auc=0.962589 f2_at20=0.262303 brier=0.007501 ece10=0.006986 rho=0.250000 donor_rows=4096 effective_n=39565.356582 seconds=1.022134 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=hgb_random_donor in_sample_bce=0.031630 local_bce=0.029087 donor_bce=0.039261 in_sample_local_ap=0.824915 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=hgb_random_donor mass={'khoa_tieng_anh': 0.75, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.25} source_failures={'khoa_tieng_anh': 969, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 124} positive_ess=662.643370 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=suite32_selective_common rows=8328 failures=122 prevalence=0.014649 ap=0.698511 auc=0.966434 f2_at20=0.271588 brier=0.007810 ece10=0.008234 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=0.941632 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=suite32_selective_common in_sample_bce=0.021639 local_bce=0.021639 donor_bce=nan in_sample_local_ap=0.907001 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=suite32_selective_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=suite32_selective_native rows=8328 failures=122 prevalence=0.014649 ap=0.697400 auc=0.969858 f2_at20=0.271588 brier=0.007413 ece10=0.006625 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=2.686240 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=suite32_selective_native in_sample_bce=0.021238 local_bce=0.021238 donor_bce=nan in_sample_local_ap=0.910760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=suite32_selective_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=native_cache_nan_indicator rows=8328 failures=122 prevalence=0.014649 ap=0.697400 auc=0.969858 f2_at20=0.271588 brier=0.007413 ece10=0.006625 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=5.336425 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=native_cache_nan_indicator in_sample_bce=0.021238 local_bce=0.021238 donor_bce=nan in_sample_local_ap=0.910760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=native_cache_nan_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=native_naive_zero_indicator rows=8328 failures=122 prevalence=0.014649 ap=0.697400 auc=0.969858 f2_at20=0.271588 brier=0.007413 ece10=0.006625 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=5.501112 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=native_naive_zero_indicator in_sample_bce=0.021238 local_bce=0.021238 donor_bce=nan in_sample_local_ap=0.910760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=native_naive_zero_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=native_placebo_indicator rows=8328 failures=122 prevalence=0.014649 ap=0.697400 auc=0.969858 f2_at20=0.271588 brier=0.007413 ece10=0.006625 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=5.596952 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=native_placebo_indicator in_sample_bce=0.021238 local_bce=0.021238 donor_bce=nan in_sample_local_ap=0.910760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=native_placebo_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan rows=8328 failures=122 prevalence=0.014649 ap=0.697400 auc=0.969858 f2_at20=0.271588 brier=0.007413 ece10=0.006625 rho=0.000000 donor_rows=0 effective_n=56161.000000 seconds=5.478841 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan in_sample_bce=0.021238 local_bce=0.021238 donor_bce=nan in_sample_local_ap=0.910760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 969} positive_ess=969.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_anh model=committee_reference ap=0.749047 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_anh models=13 positive_rows=122 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_bo_dao_nha fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=2 origin=6061 ap=0.403081 rows=785 failures=24 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.039923 ap=0.443003 base_ap=0.403081 donor_failures=179 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.010468 ap=0.392612 base_ap=0.403081 donor_failures=98 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6061 delta_ap=0.024430 ap=0.427511 base_ap=0.403081 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6061 delta_ap=0.056599 ap=0.459680 base_ap=0.403081 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6061 delta_ap=0.007382 ap=0.410463 base_ap=0.403081 donor_failures=79 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.009815 ap=0.393265 base_ap=0.403081 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6061 delta_ap=0.004116 ap=0.407196 base_ap=0.403081 donor_failures=158 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6061 delta_ap=0.013245 ap=0.416326 base_ap=0.403081 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6061 delta_ap=-0.008506 ap=0.394575 base_ap=0.403081 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6061 delta_ap=0.009568 ap=0.412648 base_ap=0.403081 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.005088 ap=0.397993 base_ap=0.403081 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6061 delta_ap=-0.007745 ap=0.395336 base_ap=0.403081 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.033600 ap=0.369480 base_ap=0.403081 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=2 origin=6063 ap=0.087458 rows=856 failures=15 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.094683 ap=0.182140 base_ap=0.087458 donor_failures=151 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.013060 ap=0.100518 base_ap=0.087458 donor_failures=104 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6063 delta_ap=0.008139 ap=0.095597 base_ap=0.087458 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6063 delta_ap=0.038179 ap=0.125636 base_ap=0.087458 donor_failures=76 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6063 delta_ap=-0.006725 ap=0.080733 base_ap=0.087458 donor_failures=55 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6063 delta_ap=0.002938 ap=0.090396 base_ap=0.087458 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6063 delta_ap=-0.001597 ap=0.085861 base_ap=0.087458 donor_failures=145 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6063 delta_ap=0.065947 ap=0.153404 base_ap=0.087458 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6063 delta_ap=0.027982 ap=0.115440 base_ap=0.087458 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6063 delta_ap=-0.003351 ap=0.084107 base_ap=0.087458 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.057419 ap=0.144876 base_ap=0.087458 donor_failures=85 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.040598 ap=0.128055 base_ap=0.087458 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6063 delta_ap=0.009134 ap=0.096592 base_ap=0.087458 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=2 origin=6064 ap=0.178245 rows=780 failures=26 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.019684 ap=0.158561 base_ap=0.178245 donor_failures=119 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.038592 ap=0.216837 base_ap=0.178245 donor_failures=96 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.011893 ap=0.166352 base_ap=0.178245 donor_failures=38 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6064 delta_ap=-0.017448 ap=0.160797 base_ap=0.178245 donor_failures=59 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6064 delta_ap=0.044205 ap=0.222450 base_ap=0.178245 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.005256 ap=0.172989 base_ap=0.178245 donor_failures=15 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6064 delta_ap=-0.004804 ap=0.173441 base_ap=0.178245 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6064 delta_ap=-0.000512 ap=0.177733 base_ap=0.178245 donor_failures=155 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6064 delta_ap=-0.006502 ap=0.171743 base_ap=0.178245 donor_failures=55 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6064 delta_ap=-0.001359 ap=0.176886 base_ap=0.178245 donor_failures=43 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.012825 ap=0.165420 base_ap=0.178245 donor_failures=77 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.018162 ap=0.196407 base_ap=0.178245 donor_failures=23 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.012535 ap=0.190780 base_ap=0.178245 donor_failures=272 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=2 origin=6066 ap=0.114102 rows=768 failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.018513 ap=0.132615 base_ap=0.114102 donor_failures=119 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.042839 ap=0.156940 base_ap=0.114102 donor_failures=114 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.041550 ap=0.155652 base_ap=0.114102 donor_failures=43 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.010396 ap=0.103706 base_ap=0.114102 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.001151 ap=0.112951 base_ap=0.114102 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=0.001104 ap=0.115206 base_ap=0.114102 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6066 delta_ap=0.040152 ap=0.154253 base_ap=0.114102 donor_failures=126 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6066 delta_ap=0.079856 ap=0.193958 base_ap=0.114102 donor_failures=178 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6066 delta_ap=-0.005379 ap=0.108723 base_ap=0.114102 donor_failures=54 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6066 delta_ap=0.052535 ap=0.166637 base_ap=0.114102 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.049253 ap=0.163355 base_ap=0.114102 donor_failures=103 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.013427 ap=0.100675 base_ap=0.114102 donor_failures=26 [TRANSFER_EDGE] fold=2 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=0.008547 ap=0.122649 base_ap=0.114102 donor_failures=259 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_bo_dao_nha origin=6064 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_anh,khoa_tieng_nga policy_donor_count=3 base_ap=0.178245 policy_ap=0.188066 delta_ap=0.009821 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_bo_dao_nha origin=6066 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nga,khoa_tieng_anh policy_donor_count=3 base_ap=0.114102 policy_ap=0.167109 delta_ap=0.053007 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_bo_dao_nha selected=['khoa_tieng_nga', 'khoa_cong_nghe_thong_tin', 'khoa_tieng_tay_ban_nha'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.031414 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common rows=716 failures=30 prevalence=0.041899 ap=0.378796 auc=0.917347 f2_at20=0.492424 brier=0.034657 ece10=0.027718 rho=0.000000 donor_rows=0 effective_n=4885.000000 seconds=0.379556 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common in_sample_bce=0.025289 local_bce=0.025289 donor_bce=nan in_sample_local_ap=0.975984 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 153} positive_ess=153.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native rows=716 failures=30 prevalence=0.041899 ap=0.413763 auc=0.912342 f2_at20=0.473485 brier=0.033653 ece10=0.026808 rho=0.000000 donor_rows=0 effective_n=4885.000000 seconds=0.698933 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native in_sample_bce=0.024980 local_bce=0.024980 donor_bce=nan in_sample_local_ap=0.976897 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 153} positive_ess=153.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all rows=716 failures=30 prevalence=0.041899 ap=0.414576 auc=0.922789 f2_at20=0.492424 brier=0.032135 ece10=0.021026 rho=0.250000 donor_rows=4095 effective_n=7293.970593 seconds=0.512687 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all in_sample_bce=0.032137 local_bce=0.030889 donor_bce=0.050812 in_sample_local_ap=0.959285 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.021584803275719405, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03473615614357197, 'khoa_quoc_te_hoc': 0.011365936637699893, 'khoa_tieng_anh': 0.049861144157933426, 'khoa_tieng_duc': 0.011349067961234006, 'khoa_tieng_han_quoc': 0.017761828493502895, 'khoa_tieng_italia': 0.011378366188780022, 'khoa_tieng_nga': 0.007276614332337786, 'khoa_tieng_nhat': 0.019428276163317198, 'khoa_tieng_phap': 0.015866321953783378, 'khoa_tieng_tay_ban_nha': 0.008368639177234745, 'khoa_tieng_trung_quoc': 0.035667484649504415, 'khoa_viet_nam_hoc': 0.005355360865380859} source_failures={'khoa_tieng_bo_dao_nha': 153, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 6, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 4, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 14} positive_ess=193.023038 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal rows=716 failures=30 prevalence=0.041899 ap=0.415174 auc=0.914723 f2_at20=0.454545 brier=0.032128 ece10=0.020686 rho=0.250000 donor_rows=4095 effective_n=7668.068999 seconds=0.564459 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal in_sample_bce=0.035017 local_bce=0.032137 donor_bce=0.043658 in_sample_local_ap=0.954953 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_bo_dao_nha': 153, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 6, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 4, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 14} positive_ess=208.263542 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior rows=716 failures=30 prevalence=0.041899 ap=0.472549 auc=0.929689 f2_at20=0.492424 brier=0.030922 ece10=0.018052 rho=0.250000 donor_rows=4095 effective_n=7668.068999 seconds=0.524798 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior in_sample_bce=0.034857 local_bce=0.032901 donor_bce=0.040725 in_sample_local_ap=0.947238 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_bo_dao_nha': 153, 'khoa_tieng_anh': 4, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 21, 'khoa_tieng_nga': 22, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 2} positive_ess=204.120796 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor rows=716 failures=30 prevalence=0.041899 ap=0.442912 auc=0.919728 f2_at20=0.473485 brier=0.031686 ece10=0.021038 rho=0.250000 donor_rows=4095 effective_n=7668.068999 seconds=0.513807 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor in_sample_bce=0.031818 local_bce=0.030399 donor_bce=0.036074 in_sample_local_ap=0.954531 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor mass={'khoa_tieng_bo_dao_nha': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.08333333333333333, np.str_('khoa_quoc_te_hoc'): 0.08333333333333333, np.str_('khoa_tieng_anh'): 0.08333333333333333} source_failures={'khoa_tieng_bo_dao_nha': 153, np.str_('khoa_tieng_tay_ban_nha'): 18, np.str_('khoa_quoc_te_hoc'): 14, np.str_('khoa_tieng_anh'): 32} positive_ess=195.218958 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common rows=716 failures=30 prevalence=0.041899 ap=0.463570 auc=0.917881 f2_at20=0.454545 brier=0.030286 ece10=0.017960 rho=0.250000 donor_rows=4095 effective_n=7621.946604 seconds=0.495559 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common in_sample_bce=0.042384 local_bce=0.034503 donor_bce=0.062900 in_sample_local_ap=0.939995 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_nga': 0.10410008834622779, 'khoa_cong_nghe_thong_tin': 0.0876180079020712, 'khoa_tieng_tay_ban_nha': 0.05828190375170104} source_failures={'khoa_tieng_bo_dao_nha': 153, 'khoa_tieng_nga': 65, 'khoa_cong_nghe_thong_tin': 49, 'khoa_tieng_tay_ban_nha': 18} positive_ess=248.207666 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native rows=716 failures=30 prevalence=0.041899 ap=0.406494 auc=0.921234 f2_at20=0.473485 brier=0.032294 ece10=0.022599 rho=0.250000 donor_rows=4095 effective_n=7621.946604 seconds=0.878470 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native in_sample_bce=0.042345 local_bce=0.033642 donor_bce=0.064988 in_sample_local_ap=0.942312 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_nga': 0.10410008834622779, 'khoa_cong_nghe_thong_tin': 0.0876180079020712, 'khoa_tieng_tay_ban_nha': 0.05828190375170104} source_failures={'khoa_tieng_bo_dao_nha': 153, 'khoa_tieng_nga': 65, 'khoa_cong_nghe_thong_tin': 49, 'khoa_tieng_tay_ban_nha': 18} positive_ess=248.207666 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator rows=716 failures=30 prevalence=0.041899 ap=0.413763 auc=0.912342 f2_at20=0.473485 brier=0.033653 ece10=0.026808 rho=0.000000 donor_rows=0 effective_n=4885.000000 seconds=1.250405 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator in_sample_bce=0.024980 local_bce=0.024980 donor_bce=nan in_sample_local_ap=0.976897 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 153} positive_ess=153.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator rows=716 failures=30 prevalence=0.041899 ap=0.421722 auc=0.916035 f2_at20=0.473485 brier=0.032880 ece10=0.023319 rho=0.000000 donor_rows=0 effective_n=4885.000000 seconds=1.206381 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator in_sample_bce=0.025025 local_bce=0.025025 donor_bce=nan in_sample_local_ap=0.975059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 153} positive_ess=153.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator rows=716 failures=30 prevalence=0.041899 ap=0.413763 auc=0.912342 f2_at20=0.473485 brier=0.033653 ece10=0.026808 rho=0.000000 donor_rows=0 effective_n=4885.000000 seconds=1.237233 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator in_sample_bce=0.024980 local_bce=0.024980 donor_bce=nan in_sample_local_ap=0.976897 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 153} positive_ess=153.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan rows=716 failures=30 prevalence=0.041899 ap=0.406494 auc=0.921234 f2_at20=0.473485 brier=0.032294 ece10=0.022599 rho=0.250000 donor_rows=4095 effective_n=7621.946604 seconds=1.672395 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan in_sample_bce=0.042345 local_bce=0.033642 donor_bce=0.064988 in_sample_local_ap=0.942312 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_nga': 0.10410008834622779, 'khoa_cong_nghe_thong_tin': 0.0876180079020712, 'khoa_tieng_tay_ban_nha': 0.05828190375170104} source_failures={'khoa_tieng_bo_dao_nha': 153, 'khoa_tieng_nga': 65, 'khoa_cong_nghe_thong_tin': 49, 'khoa_tieng_tay_ban_nha': 18} positive_ess=248.207666 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_bo_dao_nha model=committee_reference ap=0.526647 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_bo_dao_nha models=13 positive_rows=30 num_edges=52 num_replays=2 chosen=['khoa_tieng_nga', 'khoa_cong_nghe_thong_tin', 'khoa_tieng_tay_ban_nha'] [SELECTION_ORIGINS] target=khoa_tieng_duc fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_duc fold=2 origin=6061 ap=0.125469 rows=1836 failures=10 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.041986 ap=0.083483 base_ap=0.125469 donor_failures=185 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.041957 ap=0.083512 base_ap=0.125469 donor_failures=109 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6061 delta_ap=0.010874 ap=0.136343 base_ap=0.125469 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_anh origin=6061 delta_ap=-0.051357 ap=0.074112 base_ap=0.125469 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.055638 ap=0.069831 base_ap=0.125469 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.012860 ap=0.112609 base_ap=0.125469 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_italia origin=6061 delta_ap=0.051681 ap=0.177150 base_ap=0.125469 donor_failures=159 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nga origin=6061 delta_ap=0.037611 ap=0.163080 base_ap=0.125469 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6061 delta_ap=-0.039550 ap=0.085919 base_ap=0.125469 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_phap origin=6061 delta_ap=0.100610 ap=0.226079 base_ap=0.125469 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.003227 ap=0.122242 base_ap=0.125469 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.064879 ap=0.190348 base_ap=0.125469 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.039801 ap=0.085668 base_ap=0.125469 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_duc fold=2 origin=6063 ap=0.304535 rows=2282 failures=19 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.002487 ap=0.302048 base_ap=0.304535 donor_failures=156 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.173304 ap=0.131231 base_ap=0.304535 donor_failures=80 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.128052 ap=0.432587 base_ap=0.304535 donor_failures=40 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_anh origin=6063 delta_ap=-0.071821 ap=0.232714 base_ap=0.304535 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.035992 ap=0.268543 base_ap=0.304535 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.098166 ap=0.402701 base_ap=0.304535 donor_failures=26 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_italia origin=6063 delta_ap=-0.138948 ap=0.165587 base_ap=0.304535 donor_failures=144 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nga origin=6063 delta_ap=0.099619 ap=0.404154 base_ap=0.304535 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6063 delta_ap=-0.129362 ap=0.175173 base_ap=0.304535 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_phap origin=6063 delta_ap=-0.011079 ap=0.293456 base_ap=0.304535 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.061187 ap=0.365722 base_ap=0.304535 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.015703 ap=0.320238 base_ap=0.304535 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.094711 ap=0.399246 base_ap=0.304535 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_duc fold=2 origin=6064 ap=0.375883 rows=1984 failures=53 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.039158 ap=0.336725 base_ap=0.375883 donor_failures=146 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.011398 ap=0.387281 base_ap=0.375883 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.051616 ap=0.324266 base_ap=0.375883 donor_failures=23 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_anh origin=6064 delta_ap=0.012741 ap=0.388624 base_ap=0.375883 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.019427 ap=0.395310 base_ap=0.375883 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.028421 ap=0.347461 base_ap=0.375883 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_italia origin=6064 delta_ap=0.000442 ap=0.376325 base_ap=0.375883 donor_failures=126 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nga origin=6064 delta_ap=-0.004491 ap=0.371392 base_ap=0.375883 donor_failures=149 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6064 delta_ap=0.020576 ap=0.396458 base_ap=0.375883 donor_failures=39 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_phap origin=6064 delta_ap=0.010243 ap=0.386126 base_ap=0.375883 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.034994 ap=0.340889 base_ap=0.375883 donor_failures=81 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.004203 ap=0.380085 base_ap=0.375883 donor_failures=24 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.006163 ap=0.369720 base_ap=0.375883 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_duc fold=2 origin=6066 ap=0.136736 rows=1900 failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.087888 ap=0.224623 base_ap=0.136736 donor_failures=127 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.059619 ap=0.196354 base_ap=0.136736 donor_failures=105 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6066 delta_ap=0.030957 ap=0.167693 base_ap=0.136736 donor_failures=39 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_anh origin=6066 delta_ap=0.035475 ap=0.172211 base_ap=0.136736 donor_failures=75 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.059078 ap=0.195814 base_ap=0.136736 donor_failures=121 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6066 delta_ap=0.031147 ap=0.167883 base_ap=0.136736 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_italia origin=6066 delta_ap=0.022385 ap=0.159121 base_ap=0.136736 donor_failures=128 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nga origin=6066 delta_ap=0.053525 ap=0.190261 base_ap=0.136736 donor_failures=164 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6066 delta_ap=0.081458 ap=0.218194 base_ap=0.136736 donor_failures=43 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_phap origin=6066 delta_ap=0.055633 ap=0.192369 base_ap=0.136736 donor_failures=48 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.066187 ap=0.202923 base_ap=0.136736 donor_failures=88 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.000442 ap=0.136294 base_ap=0.136736 donor_failures=23 [TRANSFER_EDGE] fold=2 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.024803 ap=0.161539 base_ap=0.136736 donor_failures=263 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_duc origin=6064 policy_sources=khoa_quoc_te_hoc,khoa_tieng_nga,khoa_tieng_phap policy_donor_count=3 base_ap=0.375883 policy_ap=0.390796 delta_ap=0.014913 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_duc origin=6066 policy_sources=khoa_tieng_nga,khoa_tieng_phap,khoa_quoc_te_hoc policy_donor_count=3 base_ap=0.136736 policy_ap=0.239534 delta_ap=0.102798 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_duc selected=['khoa_tieng_nga', 'khoa_tieng_phap', 'khoa_quoc_te_hoc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.058855 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=hgb_local_common rows=1893 failures=53 prevalence=0.027998 ap=0.344746 auc=0.746457 f2_at20=0.236887 brier=0.021649 ece10=0.016615 rho=0.000000 donor_rows=0 effective_n=12783.000000 seconds=0.528838 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=hgb_local_common in_sample_bce=0.016564 local_bce=0.016564 donor_bce=nan in_sample_local_ap=0.949535 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=hgb_local_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 218} positive_ess=218.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=hgb_local_native rows=1893 failures=53 prevalence=0.027998 ap=0.335240 auc=0.794483 f2_at20=0.296108 brier=0.021300 ece10=0.015280 rho=0.000000 donor_rows=0 effective_n=12783.000000 seconds=0.993353 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=hgb_local_native in_sample_bce=0.016264 local_bce=0.016264 donor_bce=nan in_sample_local_ap=0.955131 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=hgb_local_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 218} positive_ess=218.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=hgb_pool_all rows=1893 failures=53 prevalence=0.027998 ap=0.351117 auc=0.770283 f2_at20=0.253807 brier=0.021305 ece10=0.013192 rho=0.250000 donor_rows=4095 effective_n=14954.288761 seconds=0.596016 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=hgb_pool_all in_sample_bce=0.021449 local_bce=0.020933 donor_bce=0.041431 in_sample_local_ap=0.908307 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=hgb_pool_all mass={'khoa_tieng_duc': 0.7499999999999999, 'khoa_cong_nghe_thong_tin': 0.022207688288531875, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03573855726755551, 'khoa_quoc_te_hoc': 0.011693929971610112, 'khoa_tieng_anh': 0.05130001571126352, 'khoa_tieng_bo_dao_nha': 0.004462181527207889, 'khoa_tieng_han_quoc': 0.01827439173660615, 'khoa_tieng_italia': 0.011706718209354414, 'khoa_tieng_nga': 0.007486599753735079, 'khoa_tieng_nhat': 0.01998892903989565, 'khoa_tieng_phap': 0.01632418548060024, 'khoa_tieng_tay_ban_nha': 0.00861013778412724, 'khoa_tieng_trung_quoc': 0.03669676165282492, 'khoa_viet_nam_hoc': 0.005509903576687408} source_failures={'khoa_tieng_duc': 218, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 11, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 20} positive_ess=257.740228 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=hgb_pool_equal rows=1893 failures=53 prevalence=0.027998 ap=0.422458 auc=0.792432 f2_at20=0.270728 brier=0.020760 ece10=0.013669 rho=0.250000 donor_rows=4095 effective_n=16873.003747 seconds=0.583830 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=hgb_pool_equal in_sample_bce=0.025211 local_bce=0.022510 donor_bce=0.033312 in_sample_local_ap=0.891520 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=hgb_pool_equal mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_duc': 218, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 11, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 20} positive_ess=309.896233 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=hgb_language_prior rows=1893 failures=53 prevalence=0.027998 ap=0.396831 auc=0.769442 f2_at20=0.262267 brier=0.021436 ece10=0.016088 rho=0.250000 donor_rows=4095 effective_n=16873.003747 seconds=0.743194 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=hgb_language_prior in_sample_bce=0.023782 local_bce=0.021629 donor_bce=0.030242 in_sample_local_ap=0.902563 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=hgb_language_prior mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_duc': 218, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 19, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 3} positive_ess=291.911079 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=hgb_random_donor rows=1893 failures=53 prevalence=0.027998 ap=0.374769 auc=0.785531 f2_at20=0.279188 brier=0.021115 ece10=0.013355 rho=0.250000 donor_rows=4095 effective_n=16873.003747 seconds=2.882430 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=hgb_random_donor in_sample_bce=0.031023 local_bce=0.024092 donor_bce=0.051814 in_sample_local_ap=0.882172 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=hgb_random_donor mass={'khoa_tieng_duc': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.08333333333333333, np.str_('khoa_tieng_bo_dao_nha'): 0.08333333333333333, np.str_('khoa_cong_nghe_thong_tin'): 0.08333333333333333} source_failures={'khoa_tieng_duc': 218, np.str_('khoa_tieng_tay_ban_nha'): 33, np.str_('khoa_tieng_bo_dao_nha'): 55, np.str_('khoa_cong_nghe_thong_tin'): 49} positive_ess=354.866029 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=suite32_selective_common rows=1893 failures=53 prevalence=0.027998 ap=0.406299 auc=0.814474 f2_at20=0.279188 brier=0.021277 ece10=0.015569 rho=0.250000 donor_rows=4095 effective_n=16731.340418 seconds=4.762752 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=suite32_selective_common in_sample_bce=0.023740 local_bce=0.022052 donor_bce=0.026687 in_sample_local_ap=0.899670 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=suite32_selective_common mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_nga': 0.10124417245090643, 'khoa_tieng_phap': 0.08447161276822819, 'khoa_quoc_te_hoc': 0.06428421478086536} source_failures={'khoa_tieng_duc': 218, 'khoa_tieng_nga': 48, 'khoa_tieng_phap': 14, 'khoa_quoc_te_hoc': 10} positive_ess=286.953484 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=suite32_selective_native rows=1893 failures=53 prevalence=0.027998 ap=0.409474 auc=0.833209 f2_at20=0.279188 brier=0.021280 ece10=0.015235 rho=0.250000 donor_rows=4095 effective_n=16731.340418 seconds=1.136869 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=suite32_selective_native in_sample_bce=0.023350 local_bce=0.021530 donor_bce=0.026767 in_sample_local_ap=0.905174 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=suite32_selective_native mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_nga': 0.10124417245090643, 'khoa_tieng_phap': 0.08447161276822819, 'khoa_quoc_te_hoc': 0.06428421478086536} source_failures={'khoa_tieng_duc': 218, 'khoa_tieng_nga': 48, 'khoa_tieng_phap': 14, 'khoa_quoc_te_hoc': 10} positive_ess=286.953484 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=native_cache_nan_indicator rows=1893 failures=53 prevalence=0.027998 ap=0.335240 auc=0.794483 f2_at20=0.296108 brier=0.021300 ece10=0.015280 rho=0.000000 donor_rows=0 effective_n=12783.000000 seconds=1.818922 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=native_cache_nan_indicator in_sample_bce=0.016264 local_bce=0.016264 donor_bce=nan in_sample_local_ap=0.955131 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=native_cache_nan_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 218} positive_ess=218.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=native_naive_zero_indicator rows=1893 failures=53 prevalence=0.027998 ap=0.335240 auc=0.794483 f2_at20=0.296108 brier=0.021300 ece10=0.015280 rho=0.000000 donor_rows=0 effective_n=12783.000000 seconds=1.737298 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=native_naive_zero_indicator in_sample_bce=0.016264 local_bce=0.016264 donor_bce=nan in_sample_local_ap=0.955131 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=native_naive_zero_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 218} positive_ess=218.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=native_placebo_indicator rows=1893 failures=53 prevalence=0.027998 ap=0.335240 auc=0.794483 f2_at20=0.296108 brier=0.021300 ece10=0.015280 rho=0.000000 donor_rows=0 effective_n=12783.000000 seconds=1.747924 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=native_placebo_indicator in_sample_bce=0.016264 local_bce=0.016264 donor_bce=nan in_sample_local_ap=0.955131 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=native_placebo_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 218} positive_ess=218.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan rows=1893 failures=53 prevalence=0.027998 ap=0.409474 auc=0.833209 f2_at20=0.279188 brier=0.021280 ece10=0.015235 rho=0.250000 donor_rows=4095 effective_n=16731.340418 seconds=9.367521 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan in_sample_bce=0.023350 local_bce=0.021530 donor_bce=0.026767 in_sample_local_ap=0.905174 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_nga': 0.10124417245090643, 'khoa_tieng_phap': 0.08447161276822819, 'khoa_quoc_te_hoc': 0.06428421478086536} source_failures={'khoa_tieng_duc': 218, 'khoa_tieng_nga': 48, 'khoa_tieng_phap': 14, 'khoa_quoc_te_hoc': 10} positive_ess=286.953484 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_duc model=committee_reference ap=0.393037 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_duc models=13 positive_rows=53 num_edges=52 num_replays=2 chosen=['khoa_tieng_nga', 'khoa_tieng_phap', 'khoa_quoc_te_hoc'] [SELECTION_ORIGINS] target=khoa_tieng_han_quoc fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_han_quoc fold=2 origin=6061 ap=0.152175 rows=2789 failures=5 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.022967 ap=0.129208 base_ap=0.152175 donor_failures=202 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.005162 ap=0.157337 base_ap=0.152175 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6061 delta_ap=0.000594 ap=0.152768 base_ap=0.152175 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6061 delta_ap=0.085604 ap=0.237779 base_ap=0.152175 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.067695 ap=0.219870 base_ap=0.152175 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6061 delta_ap=0.190923 ap=0.343098 base_ap=0.152175 donor_failures=72 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6061 delta_ap=-0.020226 ap=0.131949 base_ap=0.152175 donor_failures=151 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6061 delta_ap=-0.096827 ap=0.055348 base_ap=0.152175 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6061 delta_ap=-0.026336 ap=0.125839 base_ap=0.152175 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6061 delta_ap=0.098565 ap=0.250740 base_ap=0.152175 donor_failures=59 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.090148 ap=0.062027 base_ap=0.152175 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.084447 ap=0.236622 base_ap=0.152175 donor_failures=26 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6061 delta_ap=0.021779 ap=0.173954 base_ap=0.152175 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=2 origin=6063 ap=0.009075 rows=3094 failures=9 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.006683 ap=0.015758 base_ap=0.009075 donor_failures=173 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.002227 ap=0.011302 base_ap=0.009075 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.012147 ap=0.021222 base_ap=0.009075 donor_failures=32 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6063 delta_ap=0.008239 ap=0.017314 base_ap=0.009075 donor_failures=81 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.008016 ap=0.017091 base_ap=0.009075 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6063 delta_ap=0.011259 ap=0.020334 base_ap=0.009075 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6063 delta_ap=0.001006 ap=0.010081 base_ap=0.009075 donor_failures=156 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6063 delta_ap=0.006727 ap=0.015802 base_ap=0.009075 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6063 delta_ap=0.018789 ap=0.027864 base_ap=0.009075 donor_failures=50 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6063 delta_ap=0.002892 ap=0.011967 base_ap=0.009075 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.002914 ap=0.011989 base_ap=0.009075 donor_failures=88 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.023013 ap=0.032088 base_ap=0.009075 donor_failures=35 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.005751 ap=0.014826 base_ap=0.009075 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=2 origin=6064 ap=0.060463 rows=3265 failures=12 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.014828 ap=0.045635 base_ap=0.060463 donor_failures=134 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.030460 ap=0.030003 base_ap=0.060463 donor_failures=104 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.028634 ap=0.031829 base_ap=0.060463 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6064 delta_ap=0.001036 ap=0.061499 base_ap=0.060463 donor_failures=59 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.015218 ap=0.045245 base_ap=0.060463 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6064 delta_ap=0.001437 ap=0.061900 base_ap=0.060463 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6064 delta_ap=-0.021469 ap=0.038994 base_ap=0.060463 donor_failures=121 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.008763 ap=0.051700 base_ap=0.060463 donor_failures=146 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.046901 ap=0.107364 base_ap=0.060463 donor_failures=48 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.032099 ap=0.028364 base_ap=0.060463 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.027406 ap=0.033057 base_ap=0.060463 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.097319 ap=0.157782 base_ap=0.060463 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.022084 ap=0.038379 base_ap=0.060463 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=2 origin=6066 ap=0.033493 rows=2904 failures=20 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.023830 ap=0.057323 base_ap=0.033493 donor_failures=126 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.009170 ap=0.042663 base_ap=0.033493 donor_failures=98 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.003099 ap=0.030394 base_ap=0.033493 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6066 delta_ap=0.004490 ap=0.037983 base_ap=0.033493 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.032422 ap=0.065915 base_ap=0.033493 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6066 delta_ap=0.017379 ap=0.050871 base_ap=0.033493 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6066 delta_ap=0.010343 ap=0.043835 base_ap=0.033493 donor_failures=137 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6066 delta_ap=0.023586 ap=0.057079 base_ap=0.033493 donor_failures=177 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.000301 ap=0.033191 base_ap=0.033493 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6066 delta_ap=0.006977 ap=0.040470 base_ap=0.033493 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.005075 ap=0.038568 base_ap=0.033493 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.010838 ap=0.044331 base_ap=0.033493 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.006402 ap=0.039895 base_ap=0.033493 donor_failures=263 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_han_quoc origin=6064 policy_sources=khoa_tieng_duc,khoa_tieng_trung_quoc,khoa_tieng_phap policy_donor_count=3 base_ap=0.060463 policy_ap=0.039928 delta_ap=-0.020536 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_han_quoc origin=6066 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_duc,khoa_tieng_anh policy_donor_count=3 base_ap=0.033493 policy_ap=0.070438 delta_ap=0.036946 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_han_quoc selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.008205 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_local_common rows=3337 failures=29 prevalence=0.008690 ap=0.334566 auc=0.920892 f2_at20=0.159439 brier=0.007442 ece10=0.005795 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=0.652640 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=hgb_local_common in_sample_bce=0.004012 local_bce=0.004012 donor_bce=nan in_sample_local_ap=0.976999 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_local_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_local_native rows=3337 failures=29 prevalence=0.008690 ap=0.315015 auc=0.918765 f2_at20=0.159439 brier=0.007520 ece10=0.006010 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=1.169054 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=hgb_local_native in_sample_bce=0.003886 local_bce=0.003886 donor_bce=nan in_sample_local_ap=0.981311 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_local_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_pool_all rows=3337 failures=29 prevalence=0.008690 ap=0.340899 auc=0.941438 f2_at20=0.165816 brier=0.007416 ece10=0.005565 rho=0.250000 donor_rows=4095 effective_n=19388.136796 seconds=0.731016 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=hgb_pool_all in_sample_bce=0.010308 local_bce=0.007352 donor_bce=0.034825 in_sample_local_ap=0.869259 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_pool_all mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.022809664272364954, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.036707309750587315, 'khoa_quoc_te_hoc': 0.012010913212192175, 'khoa_tieng_anh': 0.05269058716684305, 'khoa_tieng_bo_dao_nha': 0.0045831363100733305, 'khoa_tieng_duc': 0.011993087298191889, 'khoa_tieng_italia': 0.012024048096192385, 'khoa_tieng_nga': 0.007689536376123032, 'khoa_tieng_nhat': 0.020530761898328493, 'khoa_tieng_phap': 0.016766679426268267, 'khoa_tieng_tay_ban_nha': 0.008843529756141496, 'khoa_tieng_trung_quoc': 0.037691487844603064, 'khoa_viet_nam_hoc': 0.005659258592090548} source_failures={'khoa_tieng_han_quoc': 95, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 7, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 16} positive_ess=134.333664 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_pool_equal rows=3337 failures=29 prevalence=0.008690 ap=0.414839 auc=0.948526 f2_at20=0.172194 brier=0.007462 ece10=0.005169 rho=0.250000 donor_rows=4095 effective_n=23052.586483 seconds=0.698303 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=hgb_pool_equal in_sample_bce=0.012309 local_bce=0.007981 donor_bce=0.025294 in_sample_local_ap=0.852626 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_pool_equal mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_han_quoc': 95, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 7, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 16} positive_ess=181.666417 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_language_prior rows=3337 failures=29 prevalence=0.008690 ap=0.404351 auc=0.938066 f2_at20=0.165816 brier=0.007335 ece10=0.004804 rho=0.250000 donor_rows=4095 effective_n=23052.586483 seconds=0.695231 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=hgb_language_prior in_sample_bce=0.010869 local_bce=0.007997 donor_bce=0.019482 in_sample_local_ap=0.855046 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_language_prior mass={'khoa_tieng_han_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_han_quoc': 95, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 11, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 8, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 5} positive_ess=166.189086 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_random_donor rows=3337 failures=29 prevalence=0.008690 ap=0.363812 auc=0.938780 f2_at20=0.172194 brier=0.007430 ece10=0.005491 rho=0.250000 donor_rows=4096 effective_n=23054.566837 seconds=0.697541 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=hgb_random_donor in_sample_bce=0.007201 local_bce=0.006342 donor_bce=0.009775 in_sample_local_ap=0.911401 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=hgb_random_donor mass={'khoa_tieng_han_quoc': 0.75, np.str_('khoa_tieng_phap'): 0.25} source_failures={'khoa_tieng_han_quoc': 95, np.str_('khoa_tieng_phap'): 51} positive_ess=137.696446 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_common rows=3337 failures=29 prevalence=0.008690 ap=0.334566 auc=0.920892 f2_at20=0.159439 brier=0.007442 ece10=0.005795 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=0.623911 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_common in_sample_bce=0.004012 local_bce=0.004012 donor_bce=nan in_sample_local_ap=0.976999 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_native rows=3337 failures=29 prevalence=0.008690 ap=0.315015 auc=0.918765 f2_at20=0.159439 brier=0.007520 ece10=0.006010 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=1.083848 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_native in_sample_bce=0.003886 local_bce=0.003886 donor_bce=nan in_sample_local_ap=0.981311 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator rows=3337 failures=29 prevalence=0.008690 ap=0.315015 auc=0.918765 f2_at20=0.159439 brier=0.007520 ece10=0.006010 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=1.993064 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator in_sample_bce=0.003886 local_bce=0.003886 donor_bce=nan in_sample_local_ap=0.981311 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator rows=3337 failures=29 prevalence=0.008690 ap=0.315015 auc=0.918765 f2_at20=0.159439 brier=0.007520 ece10=0.006010 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=2.103342 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator in_sample_bce=0.003886 local_bce=0.003886 donor_bce=nan in_sample_local_ap=0.981311 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=native_placebo_indicator rows=3337 failures=29 prevalence=0.008690 ap=0.315015 auc=0.918765 f2_at20=0.159439 brier=0.007520 ece10=0.006010 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=2.045611 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=native_placebo_indicator in_sample_bce=0.003886 local_bce=0.003886 donor_bce=nan in_sample_local_ap=0.981311 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=native_placebo_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan rows=3337 failures=29 prevalence=0.008690 ap=0.315015 auc=0.918765 f2_at20=0.159439 brier=0.007520 ece10=0.006010 rho=0.000000 donor_rows=0 effective_n=20006.000000 seconds=2.291261 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.003886 local_bce=0.003886 donor_bce=nan in_sample_local_ap=0.981311 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 95} positive_ess=95.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_han_quoc model=committee_reference ap=0.379219 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_han_quoc models=13 positive_rows=29 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_italia fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_italia fold=2 origin=6061 ap=0.584043 rows=1680 failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.037616 ap=0.546427 base_ap=0.584043 donor_failures=194 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.012478 ap=0.571565 base_ap=0.584043 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.070899 ap=0.513144 base_ap=0.584043 donor_failures=32 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_anh origin=6061 delta_ap=-0.041278 ap=0.542765 base_ap=0.584043 donor_failures=52 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.050950 ap=0.533093 base_ap=0.584043 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_duc origin=6061 delta_ap=-0.037139 ap=0.546904 base_ap=0.584043 donor_failures=76 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.015189 ap=0.568854 base_ap=0.584043 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nga origin=6061 delta_ap=-0.017382 ap=0.566661 base_ap=0.584043 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6061 delta_ap=-0.120315 ap=0.463728 base_ap=0.584043 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_phap origin=6061 delta_ap=-0.013735 ap=0.570308 base_ap=0.584043 donor_failures=68 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.026217 ap=0.610260 base_ap=0.584043 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.010118 ap=0.594161 base_ap=0.584043 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.030818 ap=0.553225 base_ap=0.584043 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_italia fold=2 origin=6063 ap=0.106650 rows=2287 failures=35 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.010715 ap=0.117365 base_ap=0.106650 donor_failures=158 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.033399 ap=0.073251 base_ap=0.106650 donor_failures=95 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6063 delta_ap=0.018097 ap=0.124747 base_ap=0.106650 donor_failures=35 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_anh origin=6063 delta_ap=-0.037837 ap=0.068813 base_ap=0.106650 donor_failures=68 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.035377 ap=0.071273 base_ap=0.106650 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_duc origin=6063 delta_ap=0.011918 ap=0.118568 base_ap=0.106650 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.052944 ap=0.053706 base_ap=0.106650 donor_failures=23 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nga origin=6063 delta_ap=-0.047051 ap=0.059599 base_ap=0.106650 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6063 delta_ap=-0.019083 ap=0.087567 base_ap=0.106650 donor_failures=61 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_phap origin=6063 delta_ap=0.015349 ap=0.121999 base_ap=0.106650 donor_failures=57 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.012828 ap=0.093822 base_ap=0.106650 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.017072 ap=0.123722 base_ap=0.106650 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.035259 ap=0.071391 base_ap=0.106650 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_italia fold=2 origin=6064 ap=0.291860 rows=2017 failures=65 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.044965 ap=0.336825 base_ap=0.291860 donor_failures=113 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.039006 ap=0.330866 base_ap=0.291860 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6064 delta_ap=0.021144 ap=0.313005 base_ap=0.291860 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_anh origin=6064 delta_ap=-0.022874 ap=0.268986 base_ap=0.291860 donor_failures=59 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.044164 ap=0.336024 base_ap=0.291860 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_duc origin=6064 delta_ap=0.035774 ap=0.327634 base_ap=0.291860 donor_failures=48 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6064 delta_ap=0.025444 ap=0.317304 base_ap=0.291860 donor_failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nga origin=6064 delta_ap=0.075181 ap=0.367042 base_ap=0.291860 donor_failures=149 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6064 delta_ap=0.013585 ap=0.305445 base_ap=0.291860 donor_failures=41 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_phap origin=6064 delta_ap=-0.010962 ap=0.280899 base_ap=0.291860 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.058418 ap=0.350278 base_ap=0.291860 donor_failures=69 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.007904 ap=0.283957 base_ap=0.291860 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6064 delta_ap=0.042159 ap=0.334019 base_ap=0.291860 donor_failures=275 [SCREEN_BASE] target=khoa_tieng_italia fold=2 origin=6066 ap=0.198600 rows=2166 failures=111 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.051580 ap=0.250181 base_ap=0.198600 donor_failures=133 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.072065 ap=0.270666 base_ap=0.198600 donor_failures=101 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6066 delta_ap=0.103357 ap=0.301957 base_ap=0.198600 donor_failures=34 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_anh origin=6066 delta_ap=0.027732 ap=0.226332 base_ap=0.198600 donor_failures=73 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.067751 ap=0.266351 base_ap=0.198600 donor_failures=120 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_duc origin=6066 delta_ap=0.006863 ap=0.205464 base_ap=0.198600 donor_failures=62 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6066 delta_ap=0.074314 ap=0.272914 base_ap=0.198600 donor_failures=20 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nga origin=6066 delta_ap=0.030169 ap=0.228769 base_ap=0.198600 donor_failures=165 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6066 delta_ap=0.012842 ap=0.211443 base_ap=0.198600 donor_failures=44 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_phap origin=6066 delta_ap=0.100786 ap=0.299386 base_ap=0.198600 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.026024 ap=0.224625 base_ap=0.198600 donor_failures=94 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.077082 ap=0.275683 base_ap=0.198600 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6066 delta_ap=0.049683 ap=0.248283 base_ap=0.198600 donor_failures=256 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_italia origin=6064 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_tay_ban_nha policy_donor_count=2 base_ap=0.291860 policy_ap=0.283917 delta_ap=-0.007944 policy_donor_rows=4096 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_italia origin=6066 policy_sources=khoa_tieng_tay_ban_nha,khoa_tieng_trung_quoc,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.198600 policy_ap=0.265304 delta_ap=0.066704 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_italia selected=['khoa_tieng_tay_ban_nha', 'khoa_tieng_trung_quoc', 'khoa_tieng_phap'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.029380 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=hgb_local_common rows=2024 failures=71 prevalence=0.035079 ap=0.506706 auc=0.896115 f2_at20=0.406386 brier=0.023752 ece10=0.010117 rho=0.000000 donor_rows=0 effective_n=12816.000000 seconds=0.620077 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=hgb_local_common in_sample_bce=0.039907 local_bce=0.039907 donor_bce=nan in_sample_local_ap=0.902202 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=hgb_local_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 437} positive_ess=437.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=hgb_local_native rows=2024 failures=71 prevalence=0.035079 ap=0.471553 auc=0.879831 f2_at20=0.399129 brier=0.024599 ece10=0.009492 rho=0.000000 donor_rows=0 effective_n=12816.000000 seconds=1.114763 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=hgb_local_native in_sample_bce=0.039282 local_bce=0.039282 donor_bce=nan in_sample_local_ap=0.907256 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=hgb_local_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 437} positive_ess=437.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=hgb_pool_all rows=2024 failures=71 prevalence=0.035079 ap=0.557818 auc=0.902844 f2_at20=0.420900 brier=0.022642 ece10=0.007262 rho=0.250000 donor_rows=4095 effective_n=14978.935058 seconds=0.602808 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=hgb_pool_all in_sample_bce=0.041541 local_bce=0.045436 donor_bce=0.046868 in_sample_local_ap=0.873236 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=hgb_pool_all mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.022210366299295466, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03574286695705557, 'khoa_quoc_te_hoc': 0.011695340135060076, 'khoa_tieng_anh': 0.05130620194696992, 'khoa_tieng_bo_dao_nha': 0.004462719618791475, 'khoa_tieng_duc': 0.011677982576665595, 'khoa_tieng_han_quoc': 0.018276595433683165, 'khoa_tieng_nga': 0.007487502557955974, 'khoa_tieng_nhat': 0.01999133949191686, 'khoa_tieng_phap': 0.01632615400356652, 'khoa_tieng_tay_ban_nha': 0.008611176075072353, 'khoa_tieng_trung_quoc': 0.0367011868915719, 'khoa_viet_nam_hoc': 0.0055105680123951235} source_failures={'khoa_tieng_italia': 437, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 1, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 18} positive_ess=475.928327 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=hgb_pool_equal rows=2024 failures=71 prevalence=0.035079 ap=0.480616 auc=0.877163 f2_at20=0.399129 brier=0.024291 ece10=0.006658 rho=0.250000 donor_rows=4095 effective_n=16905.323428 seconds=0.639297 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=hgb_pool_equal in_sample_bce=0.044778 local_bce=0.046102 donor_bce=0.040806 in_sample_local_ap=0.872735 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=hgb_pool_equal mass={'khoa_tieng_italia': 0.7499999999999999, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_italia': 437, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 1, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 18} positive_ess=526.862643 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=hgb_language_prior rows=2024 failures=71 prevalence=0.035079 ap=0.567218 auc=0.915190 f2_at20=0.435414 brier=0.022060 ece10=0.007770 rho=0.250000 donor_rows=4095 effective_n=16905.323428 seconds=0.617939 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=hgb_language_prior in_sample_bce=0.042038 local_bce=0.044287 donor_bce=0.035289 in_sample_local_ap=0.878721 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=hgb_language_prior mass={'khoa_tieng_italia': 0.7499999999999999, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_italia': 437, 'khoa_tieng_anh': 12, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 1, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_nga': 19, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 19, 'khoa_tieng_trung_quoc': 4} positive_ess=519.871496 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=hgb_random_donor rows=2024 failures=71 prevalence=0.035079 ap=0.454319 auc=0.886711 f2_at20=0.406386 brier=0.024994 ece10=0.005287 rho=0.250000 donor_rows=4095 effective_n=16905.323428 seconds=0.605550 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=hgb_random_donor in_sample_bce=0.044412 local_bce=0.045044 donor_bce=0.042517 in_sample_local_ap=0.880184 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=hgb_random_donor mass={'khoa_tieng_italia': 0.7499999999999999, np.str_('khoa_tieng_han_quoc'): 0.08333333333333333, np.str_('khoa_tieng_nga'): 0.08333333333333333, np.str_('khoa_tieng_duc'): 0.08333333333333333} source_failures={'khoa_tieng_italia': 437, np.str_('khoa_tieng_han_quoc'): 6, np.str_('khoa_tieng_nga'): 58, np.str_('khoa_tieng_duc'): 23} positive_ess=523.866406 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=suite32_selective_common rows=2024 failures=71 prevalence=0.035079 ap=0.506267 auc=0.883639 f2_at20=0.384615 brier=0.023807 ece10=0.007445 rho=0.250000 donor_rows=4095 effective_n=16901.725467 seconds=0.605354 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=suite32_selective_common in_sample_bce=0.039208 local_bce=0.043860 donor_bce=0.025220 in_sample_local_ap=0.879644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=suite32_selective_common mass={'khoa_tieng_italia': 0.75, 'khoa_tieng_tay_ban_nha': 0.08562482807621064, 'khoa_tieng_trung_quoc': 0.08434535622682122, 'khoa_tieng_phap': 0.08002981569696813} source_failures={'khoa_tieng_italia': 437, 'khoa_tieng_tay_ban_nha': 37, 'khoa_tieng_trung_quoc': 9, 'khoa_tieng_phap': 14} positive_ess=496.803588 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=suite32_selective_native rows=2024 failures=71 prevalence=0.035079 ap=0.492492 auc=0.890800 f2_at20=0.399129 brier=0.024190 ece10=0.007437 rho=0.250000 donor_rows=4095 effective_n=16901.725467 seconds=1.202513 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=suite32_selective_native in_sample_bce=0.038958 local_bce=0.043179 donor_bce=0.026243 in_sample_local_ap=0.886575 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=suite32_selective_native mass={'khoa_tieng_italia': 0.75, 'khoa_tieng_tay_ban_nha': 0.08562482807621064, 'khoa_tieng_trung_quoc': 0.08434535622682122, 'khoa_tieng_phap': 0.08002981569696813} source_failures={'khoa_tieng_italia': 437, 'khoa_tieng_tay_ban_nha': 37, 'khoa_tieng_trung_quoc': 9, 'khoa_tieng_phap': 14} positive_ess=496.803588 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=native_cache_nan_indicator rows=2024 failures=71 prevalence=0.035079 ap=0.471553 auc=0.879831 f2_at20=0.399129 brier=0.024599 ece10=0.009492 rho=0.000000 donor_rows=0 effective_n=12816.000000 seconds=1.914815 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=native_cache_nan_indicator in_sample_bce=0.039282 local_bce=0.039282 donor_bce=nan in_sample_local_ap=0.907256 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=native_cache_nan_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 437} positive_ess=437.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=native_naive_zero_indicator rows=2024 failures=71 prevalence=0.035079 ap=0.456884 auc=0.881367 f2_at20=0.384615 brier=0.024557 ece10=0.010306 rho=0.000000 donor_rows=0 effective_n=12816.000000 seconds=2.003051 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=native_naive_zero_indicator in_sample_bce=0.039073 local_bce=0.039073 donor_bce=nan in_sample_local_ap=0.908789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=native_naive_zero_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 437} positive_ess=437.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=native_placebo_indicator rows=2024 failures=71 prevalence=0.035079 ap=0.471553 auc=0.879831 f2_at20=0.399129 brier=0.024599 ece10=0.009492 rho=0.000000 donor_rows=0 effective_n=12816.000000 seconds=1.987963 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=native_placebo_indicator in_sample_bce=0.039282 local_bce=0.039282 donor_bce=nan in_sample_local_ap=0.907256 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=native_placebo_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 437} positive_ess=437.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan rows=2024 failures=71 prevalence=0.035079 ap=0.492492 auc=0.890800 f2_at20=0.399129 brier=0.024190 ece10=0.007437 rho=0.250000 donor_rows=4095 effective_n=16901.725467 seconds=2.286239 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan in_sample_bce=0.038958 local_bce=0.043179 donor_bce=0.026243 in_sample_local_ap=0.886575 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan mass={'khoa_tieng_italia': 0.75, 'khoa_tieng_tay_ban_nha': 0.08562482807621064, 'khoa_tieng_trung_quoc': 0.08434535622682122, 'khoa_tieng_phap': 0.08002981569696813} source_failures={'khoa_tieng_italia': 437, 'khoa_tieng_tay_ban_nha': 37, 'khoa_tieng_trung_quoc': 9, 'khoa_tieng_phap': 14} positive_ess=496.803588 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_italia model=committee_reference ap=0.683943 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_italia models=13 positive_rows=71 num_edges=52 num_replays=2 chosen=['khoa_tieng_tay_ban_nha', 'khoa_tieng_trung_quoc', 'khoa_tieng_phap'] [SELECTION_ORIGINS] target=khoa_tieng_nga fold=2 origins=[6063, 6064, 6065, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nga fold=2 origin=6063 ap=0.095409 rows=1460 failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.012665 ap=0.108074 base_ap=0.095409 donor_failures=153 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.019307 ap=0.114716 base_ap=0.095409 donor_failures=80 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6063 delta_ap=0.028778 ap=0.124187 base_ap=0.095409 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_anh origin=6063 delta_ap=0.042381 ap=0.137791 base_ap=0.095409 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.001296 ap=0.094113 base_ap=0.095409 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_duc origin=6063 delta_ap=0.007524 ap=0.102934 base_ap=0.095409 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6063 delta_ap=0.022670 ap=0.118079 base_ap=0.095409 donor_failures=13 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_italia origin=6063 delta_ap=0.012657 ap=0.108066 base_ap=0.095409 donor_failures=139 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6063 delta_ap=0.007794 ap=0.103203 base_ap=0.095409 donor_failures=57 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_phap origin=6063 delta_ap=0.018345 ap=0.113754 base_ap=0.095409 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.013498 ap=0.108907 base_ap=0.095409 donor_failures=78 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.013549 ap=0.108958 base_ap=0.095409 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6063 delta_ap=0.009141 ap=0.104550 base_ap=0.095409 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_nga fold=2 origin=6064 ap=0.364646 rows=1355 failures=78 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.004040 ap=0.360606 base_ap=0.364646 donor_failures=120 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.004842 ap=0.369488 base_ap=0.364646 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.018123 ap=0.346524 base_ap=0.364646 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_anh origin=6064 delta_ap=-0.009355 ap=0.355291 base_ap=0.364646 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.040190 ap=0.324456 base_ap=0.364646 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_duc origin=6064 delta_ap=-0.028097 ap=0.336549 base_ap=0.364646 donor_failures=55 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.009008 ap=0.355638 base_ap=0.364646 donor_failures=25 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_italia origin=6064 delta_ap=-0.024092 ap=0.340554 base_ap=0.364646 donor_failures=125 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6064 delta_ap=-0.048796 ap=0.315851 base_ap=0.364646 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_phap origin=6064 delta_ap=0.017948 ap=0.382594 base_ap=0.364646 donor_failures=39 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.049064 ap=0.315582 base_ap=0.364646 donor_failures=73 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.019091 ap=0.383737 base_ap=0.364646 donor_failures=24 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.010600 ap=0.354047 base_ap=0.364646 donor_failures=271 [SCREEN_BASE] target=khoa_tieng_nga fold=2 origin=6065 ap=0.317460 rows=45 failures=3 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=0.000000 ap=0.317460 base_ap=0.317460 donor_failures=114 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6065 delta_ap=0.069444 ap=0.386905 base_ap=0.317460 donor_failures=112 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6065 delta_ap=-0.011905 ap=0.305556 base_ap=0.317460 donor_failures=38 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_anh origin=6065 delta_ap=0.182540 ap=0.500000 base_ap=0.317460 donor_failures=59 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=-0.023016 ap=0.294444 base_ap=0.317460 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_duc origin=6065 delta_ap=-0.032275 ap=0.285185 base_ap=0.317460 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6065 delta_ap=0.222222 ap=0.539683 base_ap=0.317460 donor_failures=16 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_italia origin=6065 delta_ap=-0.013889 ap=0.303571 base_ap=0.317460 donor_failures=121 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6065 delta_ap=0.069444 ap=0.386905 base_ap=0.317460 donor_failures=45 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_phap origin=6065 delta_ap=0.042063 ap=0.359524 base_ap=0.317460 donor_failures=49 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=0.044444 ap=0.361905 base_ap=0.317460 donor_failures=81 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6065 delta_ap=0.000000 ap=0.317460 base_ap=0.317460 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6065 delta_ap=0.215873 ap=0.533333 base_ap=0.317460 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_nga fold=2 origin=6066 ap=0.305138 rows=1588 failures=102 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.000231 ap=0.304907 base_ap=0.305138 donor_failures=116 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.031558 ap=0.336695 base_ap=0.305138 donor_failures=108 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.000654 ap=0.304484 base_ap=0.305138 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_anh origin=6066 delta_ap=0.051745 ap=0.356882 base_ap=0.305138 donor_failures=69 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.013277 ap=0.318415 base_ap=0.305138 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_duc origin=6066 delta_ap=0.013928 ap=0.319065 base_ap=0.305138 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6066 delta_ap=0.001581 ap=0.306719 base_ap=0.305138 donor_failures=18 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_italia origin=6066 delta_ap=-0.020780 ap=0.284358 base_ap=0.305138 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6066 delta_ap=0.000512 ap=0.305649 base_ap=0.305138 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_phap origin=6066 delta_ap=0.050892 ap=0.356030 base_ap=0.305138 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.021146 ap=0.283991 base_ap=0.305138 donor_failures=89 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.012035 ap=0.317172 base_ap=0.305138 donor_failures=26 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6066 delta_ap=0.012903 ap=0.318041 base_ap=0.305138 donor_failures=262 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_nga origin=6065 policy_sources=khoa_tieng_phap,khoa_tieng_anh,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.317460 policy_ap=0.317460 delta_ap=0.000000 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_nga origin=6066 policy_sources=khoa_tieng_han_quoc,khoa_tieng_anh,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.305138 policy_ap=0.297969 delta_ap=-0.007168 policy_donor_rows=4095 selected_using_origins=6063,6064,6065 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_nga selected=[] reason=prequential_policy_delta_-0.003584_below_threshold replay_mean=-0.003584 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=hgb_local_common rows=1567 failures=83 prevalence=0.052967 ap=0.807127 auc=0.976204 f2_at20=0.626935 brier=0.026613 ece10=0.025544 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=0.465416 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=hgb_local_common in_sample_bce=0.049351 local_bce=0.049351 donor_bce=nan in_sample_local_ap=0.918168 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=hgb_local_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=hgb_local_native rows=1567 failures=83 prevalence=0.052967 ap=0.811390 auc=0.976557 f2_at20=0.626935 brier=0.025894 ece10=0.023639 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=1.039358 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=hgb_local_native in_sample_bce=0.049189 local_bce=0.049189 donor_bce=nan in_sample_local_ap=0.917139 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=hgb_local_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=hgb_pool_all rows=1567 failures=83 prevalence=0.052967 ap=0.742893 auc=0.967501 f2_at20=0.619195 brier=0.027750 ece10=0.023447 rho=0.250000 donor_rows=4095 effective_n=10989.414462 seconds=0.541873 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=hgb_pool_all in_sample_bce=0.050289 local_bce=0.054289 donor_bce=0.053433 in_sample_local_ap=0.899300 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=hgb_pool_all mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.02184162486164815, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03514945593583349, 'khoa_quoc_te_hoc': 0.011501171498799753, 'khoa_tieng_anh': 0.050454404979229255, 'khoa_tieng_bo_dao_nha': 0.004388628555822277, 'khoa_tieng_duc': 0.011484102114447527, 'khoa_tieng_han_quoc': 0.0179731633342437, 'khoa_tieng_italia': 0.011513748939901394, 'khoa_tieng_nhat': 0.01965943883051359, 'khoa_tieng_phap': 0.016055103566243584, 'khoa_tieng_tay_ban_nha': 0.008468211416004255, 'khoa_tieng_trung_quoc': 0.03609186562980638, 'khoa_viet_nam_hoc': 0.005419080337506648} source_failures={'khoa_tieng_nga': 372, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 9, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 16, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 15} positive_ess=426.970593 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=hgb_pool_equal rows=1567 failures=83 prevalence=0.052967 ap=0.755003 auc=0.972806 f2_at20=0.626935 brier=0.025639 ece10=0.024188 rho=0.250000 donor_rows=4095 effective_n=11919.866818 seconds=0.591225 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=hgb_pool_equal in_sample_bce=0.053960 local_bce=0.055898 donor_bce=0.048145 in_sample_local_ap=0.891530 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=hgb_pool_equal mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nga': 372, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 9, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 16, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 15} positive_ess=453.908399 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=hgb_language_prior rows=1567 failures=83 prevalence=0.052967 ap=0.713696 auc=0.974337 f2_at20=0.626935 brier=0.026918 ece10=0.025968 rho=0.250000 donor_rows=4095 effective_n=11919.866818 seconds=0.528016 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=hgb_language_prior in_sample_bce=0.052890 local_bce=0.055413 donor_bce=0.045324 in_sample_local_ap=0.895619 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=hgb_language_prior mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nga': 372, 'khoa_tieng_anh': 12, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 13, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 9, 'khoa_tieng_tay_ban_nha': 14, 'khoa_tieng_trung_quoc': 1} positive_ess=441.189115 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=hgb_random_donor rows=1567 failures=83 prevalence=0.052967 ap=0.760967 auc=0.977056 f2_at20=0.626935 brier=0.027937 ece10=0.027731 rho=0.250000 donor_rows=4096 effective_n=11920.396272 seconds=0.515194 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=hgb_random_donor in_sample_bce=0.050219 local_bce=0.054797 donor_bce=0.036483 in_sample_local_ap=0.896050 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=hgb_random_donor mass={'khoa_tieng_nga': 0.75, np.str_('khoa_tieng_duc'): 0.25} source_failures={'khoa_tieng_nga': 372, np.str_('khoa_tieng_duc'): 60} positive_ess=425.791814 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=suite32_selective_common rows=1567 failures=83 prevalence=0.052967 ap=0.807127 auc=0.976204 f2_at20=0.626935 brier=0.026613 ece10=0.025544 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=0.455593 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=suite32_selective_common in_sample_bce=0.049351 local_bce=0.049351 donor_bce=nan in_sample_local_ap=0.918168 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=suite32_selective_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=suite32_selective_native rows=1567 failures=83 prevalence=0.052967 ap=0.811390 auc=0.976557 f2_at20=0.626935 brier=0.025894 ece10=0.023639 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=1.063802 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=suite32_selective_native in_sample_bce=0.049189 local_bce=0.049189 donor_bce=nan in_sample_local_ap=0.917139 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=suite32_selective_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=native_cache_nan_indicator rows=1567 failures=83 prevalence=0.052967 ap=0.811390 auc=0.976557 f2_at20=0.626935 brier=0.025894 ece10=0.023639 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=2.014522 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=native_cache_nan_indicator in_sample_bce=0.049189 local_bce=0.049189 donor_bce=nan in_sample_local_ap=0.917139 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=native_cache_nan_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=native_naive_zero_indicator rows=1567 failures=83 prevalence=0.052967 ap=0.821504 auc=0.977081 f2_at20=0.626935 brier=0.025866 ece10=0.028090 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=1.719869 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=native_naive_zero_indicator in_sample_bce=0.049112 local_bce=0.049112 donor_bce=nan in_sample_local_ap=0.918825 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=native_naive_zero_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=native_placebo_indicator rows=1567 failures=83 prevalence=0.052967 ap=0.811390 auc=0.976557 f2_at20=0.626935 brier=0.025894 ece10=0.023639 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=1.801300 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=native_placebo_indicator in_sample_bce=0.049189 local_bce=0.049189 donor_bce=nan in_sample_local_ap=0.917139 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=native_placebo_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan rows=1567 failures=83 prevalence=0.052967 ap=0.811390 auc=0.976557 f2_at20=0.626935 brier=0.025894 ece10=0.023639 rho=0.000000 donor_rows=0 effective_n=8196.000000 seconds=1.681427 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan in_sample_bce=0.049189 local_bce=0.049189 donor_bce=nan in_sample_local_ap=0.917139 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 372} positive_ess=372.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nga model=committee_reference ap=0.759518 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_nga models=13 positive_rows=83 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nhat fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nhat fold=2 origin=6061 ap=0.275054 rows=3201 failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.032910 ap=0.307963 base_ap=0.275054 donor_failures=185 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.015239 ap=0.290293 base_ap=0.275054 donor_failures=92 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6061 delta_ap=0.045370 ap=0.320423 base_ap=0.275054 donor_failures=34 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6061 delta_ap=0.028035 ap=0.303089 base_ap=0.275054 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.138139 ap=0.413193 base_ap=0.275054 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6061 delta_ap=0.063141 ap=0.338195 base_ap=0.275054 donor_failures=78 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6061 delta_ap=0.048494 ap=0.323548 base_ap=0.275054 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6061 delta_ap=0.092369 ap=0.367422 base_ap=0.275054 donor_failures=151 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6061 delta_ap=0.056284 ap=0.331338 base_ap=0.275054 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6061 delta_ap=0.042701 ap=0.317755 base_ap=0.275054 donor_failures=64 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.127818 ap=0.402872 base_ap=0.275054 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.003191 ap=0.278245 base_ap=0.275054 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6061 delta_ap=0.069709 ap=0.344762 base_ap=0.275054 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_nhat fold=2 origin=6063 ap=0.080777 rows=3170 failures=7 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.069744 ap=0.011034 base_ap=0.080777 donor_failures=180 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.058410 ap=0.022368 base_ap=0.080777 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.049888 ap=0.030889 base_ap=0.080777 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6063 delta_ap=0.070317 ap=0.151094 base_ap=0.080777 donor_failures=78 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.058833 ap=0.021944 base_ap=0.080777 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6063 delta_ap=-0.044160 ap=0.036618 base_ap=0.080777 donor_failures=74 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6063 delta_ap=0.081091 ap=0.161868 base_ap=0.080777 donor_failures=15 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6063 delta_ap=-0.040536 ap=0.040242 base_ap=0.080777 donor_failures=148 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6063 delta_ap=-0.060292 ap=0.020485 base_ap=0.080777 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6063 delta_ap=-0.058530 ap=0.022247 base_ap=0.080777 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.069904 ap=0.010873 base_ap=0.080777 donor_failures=84 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.069914 ap=0.010863 base_ap=0.080777 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.067563 ap=0.013214 base_ap=0.080777 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_nhat fold=2 origin=6064 ap=0.189414 rows=2945 failures=19 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.071807 ap=0.117608 base_ap=0.189414 donor_failures=126 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.051348 ap=0.138067 base_ap=0.189414 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.106333 ap=0.083082 base_ap=0.189414 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6064 delta_ap=-0.141366 ap=0.048048 base_ap=0.189414 donor_failures=44 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.039793 ap=0.229208 base_ap=0.189414 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6064 delta_ap=0.035707 ap=0.225121 base_ap=0.189414 donor_failures=61 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.126070 ap=0.063345 base_ap=0.189414 donor_failures=23 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6064 delta_ap=-0.101368 ap=0.088046 base_ap=0.189414 donor_failures=129 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6064 delta_ap=-0.029892 ap=0.159522 base_ap=0.189414 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6064 delta_ap=0.030678 ap=0.220092 base_ap=0.189414 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.113748 ap=0.075666 base_ap=0.189414 donor_failures=77 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.037004 ap=0.152410 base_ap=0.189414 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.140843 ap=0.048572 base_ap=0.189414 donor_failures=271 [SCREEN_BASE] target=khoa_tieng_nhat fold=2 origin=6066 ap=0.398419 rows=2840 failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.129318 ap=0.527738 base_ap=0.398419 donor_failures=109 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.156310 ap=0.554730 base_ap=0.398419 donor_failures=104 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.267574 ap=0.130845 base_ap=0.398419 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6066 delta_ap=-0.225421 ap=0.172999 base_ap=0.398419 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.187520 ap=0.210899 base_ap=0.398419 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6066 delta_ap=-0.135059 ap=0.263361 base_ap=0.398419 donor_failures=72 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6066 delta_ap=0.040541 ap=0.438961 base_ap=0.398419 donor_failures=11 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6066 delta_ap=-0.227427 ap=0.170993 base_ap=0.398419 donor_failures=129 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6066 delta_ap=-0.268197 ap=0.130223 base_ap=0.398419 donor_failures=161 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6066 delta_ap=-0.031202 ap=0.367218 base_ap=0.398419 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.101794 ap=0.500213 base_ap=0.398419 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.135988 ap=0.534407 base_ap=0.398419 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6066 delta_ap=0.113258 ap=0.511677 base_ap=0.398419 donor_failures=270 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_nhat origin=6064 policy_sources=khoa_tieng_han_quoc,khoa_tieng_anh,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.189414 policy_ap=0.129764 delta_ap=-0.059651 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_nhat origin=6066 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_duc,khoa_tieng_phap policy_donor_count=3 base_ap=0.398419 policy_ap=0.558196 delta_ap=0.159776 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_nhat selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.050063 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=hgb_local_common rows=2448 failures=11 prevalence=0.004493 ap=0.481320 auc=0.979781 f2_at20=0.102996 brier=0.003046 ece10=0.002890 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=0.663712 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=hgb_local_common in_sample_bce=0.007517 local_bce=0.007517 donor_bce=nan in_sample_local_ap=0.959350 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=hgb_local_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=hgb_local_native rows=2448 failures=11 prevalence=0.004493 ap=0.367445 auc=0.978532 f2_at20=0.102996 brier=0.003503 ece10=0.002436 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=1.201725 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=hgb_local_native in_sample_bce=0.007130 local_bce=0.007130 donor_bce=nan in_sample_local_ap=0.966698 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=hgb_local_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=hgb_pool_all rows=2448 failures=11 prevalence=0.004493 ap=0.502146 auc=0.979446 f2_at20=0.102996 brier=0.003111 ece10=0.002332 rho=0.250000 donor_rows=4095 effective_n=20293.251660 seconds=0.659573 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=hgb_pool_all in_sample_bce=0.012922 local_bce=0.011221 donor_bce=0.033153 in_sample_local_ap=0.900407 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=hgb_pool_all mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.02297147651640847, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03696771218758149, 'khoa_quoc_te_hoc': 0.012096118886272673, 'khoa_tieng_anh': 0.0530643753141665, 'khoa_tieng_bo_dao_nha': 0.004615649176647555, 'khoa_tieng_duc': 0.012078166514858895, 'khoa_tieng_han_quoc': 0.01890290223705445, 'khoa_tieng_italia': 0.012109346949419666, 'khoa_tieng_nga': 0.00774408611091164, 'khoa_tieng_phap': 0.016885622607137864, 'khoa_tieng_tay_ban_nha': 0.0089062659445404, 'khoa_tieng_trung_quoc': 0.03795887206195269, 'khoa_viet_nam_hoc': 0.005699405493047708} source_failures={'khoa_tieng_nhat': 216, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 15, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 18} positive_ess=221.562933 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=hgb_pool_equal rows=2448 failures=11 prevalence=0.004493 ap=0.474110 auc=0.962043 f2_at20=0.102996 brier=0.003135 ece10=0.001799 rho=0.250000 donor_rows=4095 effective_n=24409.652355 seconds=0.667001 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=hgb_pool_equal in_sample_bce=0.014540 local_bce=0.011684 donor_bce=0.023109 in_sample_local_ap=0.894429 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=hgb_pool_equal mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nhat': 216, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 5, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 5, 'khoa_tieng_nga': 15, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 18} positive_ess=280.571220 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=hgb_language_prior rows=2448 failures=11 prevalence=0.004493 ap=0.358901 auc=0.988772 f2_at20=0.102996 brier=0.003561 ece10=0.002580 rho=0.250000 donor_rows=4095 effective_n=24409.652355 seconds=0.726388 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=hgb_language_prior in_sample_bce=0.014211 local_bce=0.011564 donor_bce=0.022152 in_sample_local_ap=0.898339 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=hgb_language_prior mass={'khoa_tieng_nhat': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nhat': 216, 'khoa_tieng_anh': 17, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 19, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 13, 'khoa_tieng_trung_quoc': 2} positive_ess=280.571220 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=hgb_random_donor rows=2448 failures=11 prevalence=0.004493 ap=0.330812 auc=0.987615 f2_at20=0.102996 brier=0.003847 ece10=0.003510 rho=0.250000 donor_rows=4096 effective_n=24411.872742 seconds=0.666696 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=hgb_random_donor in_sample_bce=0.010492 local_bce=0.009999 donor_bce=0.011970 in_sample_local_ap=0.917811 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=hgb_random_donor mass={'khoa_tieng_nhat': 0.75, np.str_('khoa_tieng_phap'): 0.25} source_failures={'khoa_tieng_nhat': 216, np.str_('khoa_tieng_phap'): 51} positive_ess=249.218976 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_common rows=2448 failures=11 prevalence=0.004493 ap=0.481320 auc=0.979781 f2_at20=0.102996 brier=0.003046 ece10=0.002890 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=0.593130 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_common in_sample_bce=0.007517 local_bce=0.007517 donor_bce=nan in_sample_local_ap=0.959350 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_native rows=2448 failures=11 prevalence=0.004493 ap=0.367445 auc=0.978532 f2_at20=0.102996 brier=0.003503 ece10=0.002436 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=1.267230 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_native in_sample_bce=0.007130 local_bce=0.007130 donor_bce=nan in_sample_local_ap=0.966698 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=native_cache_nan_indicator rows=2448 failures=11 prevalence=0.004493 ap=0.367445 auc=0.978532 f2_at20=0.102996 brier=0.003503 ece10=0.002436 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=2.224641 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=native_cache_nan_indicator in_sample_bce=0.007130 local_bce=0.007130 donor_bce=nan in_sample_local_ap=0.966698 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=native_cache_nan_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=native_naive_zero_indicator rows=2448 failures=11 prevalence=0.004493 ap=0.367445 auc=0.978532 f2_at20=0.102996 brier=0.003503 ece10=0.002436 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=2.141524 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=native_naive_zero_indicator in_sample_bce=0.007130 local_bce=0.007130 donor_bce=nan in_sample_local_ap=0.966698 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=native_naive_zero_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=native_placebo_indicator rows=2448 failures=11 prevalence=0.004493 ap=0.367445 auc=0.978532 f2_at20=0.102996 brier=0.003503 ece10=0.002436 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=2.114956 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=native_placebo_indicator in_sample_bce=0.007130 local_bce=0.007130 donor_bce=nan in_sample_local_ap=0.966698 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=native_placebo_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan rows=2448 failures=11 prevalence=0.004493 ap=0.367445 auc=0.978532 f2_at20=0.102996 brier=0.003503 ece10=0.002436 rho=0.000000 donor_rows=0 effective_n=21883.000000 seconds=2.184413 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan in_sample_bce=0.007130 local_bce=0.007130 donor_bce=nan in_sample_local_ap=0.966698 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 216} positive_ess=216.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_nhat model=committee_reference ap=0.542198 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_nhat models=13 positive_rows=11 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_phap fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_phap fold=2 origin=6061 ap=0.520097 rows=2428 failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.003100 ap=0.516997 base_ap=0.520097 donor_failures=177 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.006551 ap=0.513546 base_ap=0.520097 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.012735 ap=0.507362 base_ap=0.520097 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_anh origin=6061 delta_ap=-0.006817 ap=0.513280 base_ap=0.520097 donor_failures=51 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.006043 ap=0.514055 base_ap=0.520097 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_duc origin=6061 delta_ap=-0.015210 ap=0.504887 base_ap=0.520097 donor_failures=78 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.039144 ap=0.480953 base_ap=0.520097 donor_failures=29 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_italia origin=6061 delta_ap=0.069410 ap=0.589507 base_ap=0.520097 donor_failures=167 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nga origin=6061 delta_ap=-0.115685 ap=0.404412 base_ap=0.520097 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6061 delta_ap=0.000010 ap=0.520107 base_ap=0.520097 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.013008 ap=0.507090 base_ap=0.520097 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.051037 ap=0.571135 base_ap=0.520097 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.031259 ap=0.488839 base_ap=0.520097 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_phap fold=2 origin=6063 ap=0.081698 rows=2758 failures=12 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.071483 ap=0.153181 base_ap=0.081698 donor_failures=173 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.157892 ap=0.239590 base_ap=0.081698 donor_failures=93 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6063 delta_ap=0.013094 ap=0.094792 base_ap=0.081698 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_anh origin=6063 delta_ap=0.227391 ap=0.309089 base_ap=0.081698 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.324985 ap=0.406683 base_ap=0.081698 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_duc origin=6063 delta_ap=0.113777 ap=0.195475 base_ap=0.081698 donor_failures=56 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.005986 ap=0.075712 base_ap=0.081698 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_italia origin=6063 delta_ap=-0.002054 ap=0.079644 base_ap=0.081698 donor_failures=144 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nga origin=6063 delta_ap=-0.015854 ap=0.065844 base_ap=0.081698 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6063 delta_ap=0.186896 ap=0.268594 base_ap=0.081698 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.225200 ap=0.306898 base_ap=0.081698 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.038750 ap=0.120448 base_ap=0.081698 donor_failures=33 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6063 delta_ap=0.242270 ap=0.323968 base_ap=0.081698 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_phap fold=2 origin=6064 ap=0.452481 rows=2471 failures=32 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.074726 ap=0.377755 base_ap=0.452481 donor_failures=125 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.042364 ap=0.410117 base_ap=0.452481 donor_failures=98 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.046070 ap=0.406411 base_ap=0.452481 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_anh origin=6064 delta_ap=-0.069346 ap=0.383135 base_ap=0.452481 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.026951 ap=0.479431 base_ap=0.452481 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_duc origin=6064 delta_ap=-0.024914 ap=0.427566 base_ap=0.452481 donor_failures=44 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6064 delta_ap=0.033070 ap=0.485551 base_ap=0.452481 donor_failures=13 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_italia origin=6064 delta_ap=-0.019532 ap=0.432949 base_ap=0.452481 donor_failures=112 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nga origin=6064 delta_ap=-0.008367 ap=0.444114 base_ap=0.452481 donor_failures=148 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6064 delta_ap=-0.005920 ap=0.446561 base_ap=0.452481 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.053736 ap=0.398745 base_ap=0.452481 donor_failures=85 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.069155 ap=0.383326 base_ap=0.452481 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.054751 ap=0.397730 base_ap=0.452481 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_phap fold=2 origin=6066 ap=0.258397 rows=2606 failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.004184 ap=0.262581 base_ap=0.258397 donor_failures=136 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.006823 ap=0.265221 base_ap=0.258397 donor_failures=109 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6066 delta_ap=0.031277 ap=0.289674 base_ap=0.258397 donor_failures=41 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_anh origin=6066 delta_ap=-0.025744 ap=0.232653 base_ap=0.258397 donor_failures=78 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.014263 ap=0.272660 base_ap=0.258397 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_duc origin=6066 delta_ap=-0.044727 ap=0.213670 base_ap=0.258397 donor_failures=62 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6066 delta_ap=0.022074 ap=0.280471 base_ap=0.258397 donor_failures=19 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_italia origin=6066 delta_ap=0.006628 ap=0.265025 base_ap=0.258397 donor_failures=117 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nga origin=6066 delta_ap=0.056649 ap=0.315046 base_ap=0.258397 donor_failures=175 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6066 delta_ap=-0.027356 ap=0.231041 base_ap=0.258397 donor_failures=42 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.009533 ap=0.267930 base_ap=0.258397 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.036491 ap=0.294889 base_ap=0.258397 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6066 delta_ap=0.026743 ap=0.285141 base_ap=0.258397 donor_failures=250 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_phap origin=6064 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_anh,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.452481 policy_ap=0.441042 delta_ap=-0.011439 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_phap origin=6066 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_nhat,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.258397 policy_ap=0.310567 delta_ap=0.052170 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_phap selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.020366 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=hgb_local_common rows=2204 failures=40 prevalence=0.018149 ap=0.489212 auc=0.922158 f2_at20=0.299501 brier=0.013608 ece10=0.012287 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=1.381461 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=hgb_local_common in_sample_bce=0.010621 local_bce=0.010621 donor_bce=nan in_sample_local_ap=0.949188 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=hgb_local_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=hgb_local_native rows=2204 failures=40 prevalence=0.018149 ap=0.538547 auc=0.915411 f2_at20=0.299501 brier=0.013076 ece10=0.010475 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=1.420647 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=hgb_local_native in_sample_bce=0.010647 local_bce=0.010647 donor_bce=nan in_sample_local_ap=0.952597 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=hgb_local_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=hgb_pool_all rows=2204 failures=40 prevalence=0.018149 ap=0.504091 auc=0.927686 f2_at20=0.307820 brier=0.013664 ece10=0.009868 rho=0.250000 donor_rows=4095 effective_n=18252.221714 seconds=0.651957 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=hgb_pool_all in_sample_bce=0.017567 local_bce=0.015662 donor_bce=0.044208 in_sample_local_ap=0.888492 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=hgb_pool_all mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.022628359537008425, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.036415538289135185, 'khoa_quoc_te_hoc': 0.011915443352779774, 'khoa_tieng_anh': 0.05227177114009255, 'khoa_tieng_bo_dao_nha': 0.004546706825365505, 'khoa_tieng_duc': 0.011897759129712845, 'khoa_tieng_han_quoc': 0.018620556140893, 'khoa_tieng_italia': 0.011928473832934352, 'khoa_tieng_nga': 0.0076284153819233735, 'khoa_tieng_nhat': 0.020367571230189018, 'khoa_tieng_tay_ban_nha': 0.00877323613836136, 'khoa_tieng_trung_quoc': 0.03739189355214612, 'khoa_viet_nam_hoc': 0.0056142754494584906} source_failures={'khoa_tieng_phap': 220, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 14, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 2, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 20} positive_ess=257.360128 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=hgb_pool_equal rows=2204 failures=40 prevalence=0.018149 ap=0.449170 auc=0.921424 f2_at20=0.307820 brier=0.013060 ece10=0.009196 rho=0.250000 donor_rows=4095 effective_n=21395.825019 seconds=0.605528 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=hgb_pool_equal in_sample_bce=0.020681 local_bce=0.016696 donor_bce=0.032638 in_sample_local_ap=0.878656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=hgb_pool_equal mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_phap': 220, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 14, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 2, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 20} positive_ess=319.900899 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=hgb_language_prior rows=2204 failures=40 prevalence=0.018149 ap=0.502272 auc=0.929153 f2_at20=0.307820 brier=0.012815 ece10=0.009911 rho=0.250000 donor_rows=4095 effective_n=21395.825019 seconds=0.697145 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=hgb_language_prior in_sample_bce=0.018968 local_bce=0.016298 donor_bce=0.026977 in_sample_local_ap=0.889108 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=hgb_language_prior mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_phap': 220, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 20, 'khoa_tieng_duc': 11, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 24, 'khoa_tieng_nhat': 5, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3} positive_ess=306.625091 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=hgb_random_donor rows=2204 failures=40 prevalence=0.018149 ap=0.466510 auc=0.934387 f2_at20=0.299501 brier=0.013606 ece10=0.010328 rho=0.250000 donor_rows=4096 effective_n=21397.530940 seconds=0.602025 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=hgb_random_donor in_sample_bce=0.023058 local_bce=0.017082 donor_bce=0.040986 in_sample_local_ap=0.876806 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=hgb_random_donor mass={'khoa_tieng_phap': 0.75, np.str_('khoa_cong_nghe_thong_tin'): 0.25} source_failures={'khoa_tieng_phap': 220, np.str_('khoa_cong_nghe_thong_tin'): 118} positive_ess=326.587943 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=suite32_selective_common rows=2204 failures=40 prevalence=0.018149 ap=0.489212 auc=0.922158 f2_at20=0.299501 brier=0.013608 ece10=0.012287 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=0.600534 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=suite32_selective_common in_sample_bce=0.010621 local_bce=0.010621 donor_bce=nan in_sample_local_ap=0.949188 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=suite32_selective_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=suite32_selective_native rows=2204 failures=40 prevalence=0.018149 ap=0.538547 auc=0.915411 f2_at20=0.299501 brier=0.013076 ece10=0.010475 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=1.401980 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=suite32_selective_native in_sample_bce=0.010647 local_bce=0.010647 donor_bce=nan in_sample_local_ap=0.952597 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=suite32_selective_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=native_cache_nan_indicator rows=2204 failures=40 prevalence=0.018149 ap=0.538547 auc=0.915411 f2_at20=0.299501 brier=0.013076 ece10=0.010475 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=3.184023 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=native_cache_nan_indicator in_sample_bce=0.010647 local_bce=0.010647 donor_bce=nan in_sample_local_ap=0.952597 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=native_cache_nan_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=native_naive_zero_indicator rows=2204 failures=40 prevalence=0.018149 ap=0.556839 auc=0.923175 f2_at20=0.299501 brier=0.013173 ece10=0.012219 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=3.108279 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=native_naive_zero_indicator in_sample_bce=0.010603 local_bce=0.010603 donor_bce=nan in_sample_local_ap=0.951136 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=native_naive_zero_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=native_placebo_indicator rows=2204 failures=40 prevalence=0.018149 ap=0.538547 auc=0.915411 f2_at20=0.299501 brier=0.013076 ece10=0.010475 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=4.121237 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=native_placebo_indicator in_sample_bce=0.010647 local_bce=0.010647 donor_bce=nan in_sample_local_ap=0.952597 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=native_placebo_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan rows=2204 failures=40 prevalence=0.018149 ap=0.538547 auc=0.915411 f2_at20=0.299501 brier=0.013076 ece10=0.010475 rho=0.000000 donor_rows=0 effective_n=17871.000000 seconds=3.116200 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan in_sample_bce=0.010647 local_bce=0.010647 donor_bce=nan in_sample_local_ap=0.952597 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_phap model=committee_reference ap=0.584629 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_phap models=13 positive_rows=40 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_tay_ban_nha fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=2 origin=6061 ap=0.349675 rows=1419 failures=20 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.048219 ap=0.397894 base_ap=0.349675 donor_failures=191 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.002530 ap=0.347146 base_ap=0.349675 donor_failures=84 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.001835 ap=0.347840 base_ap=0.349675 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6061 delta_ap=0.021013 ap=0.370688 base_ap=0.349675 donor_failures=60 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.009110 ap=0.340565 base_ap=0.349675 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6061 delta_ap=-0.033723 ap=0.315952 base_ap=0.349675 donor_failures=77 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.010165 ap=0.339510 base_ap=0.349675 donor_failures=28 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6061 delta_ap=0.019996 ap=0.369671 base_ap=0.349675 donor_failures=158 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6061 delta_ap=0.054221 ap=0.403897 base_ap=0.349675 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6061 delta_ap=0.051546 ap=0.401221 base_ap=0.349675 donor_failures=69 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6061 delta_ap=0.004629 ap=0.354304 base_ap=0.349675 donor_failures=62 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.069212 ap=0.418887 base_ap=0.349675 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6061 delta_ap=-0.004536 ap=0.345139 base_ap=0.349675 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=2 origin=6063 ap=0.381329 rows=1560 failures=18 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.095263 ap=0.476592 base_ap=0.381329 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.086025 ap=0.467354 base_ap=0.381329 donor_failures=98 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6063 delta_ap=0.032013 ap=0.413342 base_ap=0.381329 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6063 delta_ap=0.035773 ap=0.417102 base_ap=0.381329 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.066088 ap=0.315241 base_ap=0.381329 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6063 delta_ap=-0.214616 ap=0.166713 base_ap=0.381329 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.004438 ap=0.376891 base_ap=0.381329 donor_failures=22 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6063 delta_ap=0.115076 ap=0.496404 base_ap=0.381329 donor_failures=153 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6063 delta_ap=-0.022237 ap=0.359092 base_ap=0.381329 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6063 delta_ap=0.011842 ap=0.393170 base_ap=0.381329 donor_failures=52 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6063 delta_ap=-0.032804 ap=0.348525 base_ap=0.381329 donor_failures=57 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.075883 ap=0.457212 base_ap=0.381329 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.020329 ap=0.361000 base_ap=0.381329 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=2 origin=6064 ap=0.306637 rows=1388 failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.032667 ap=0.339305 base_ap=0.306637 donor_failures=118 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.053442 ap=0.360079 base_ap=0.306637 donor_failures=87 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=0.009544 ap=0.316181 base_ap=0.306637 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6064 delta_ap=0.059670 ap=0.366308 base_ap=0.306637 donor_failures=68 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.047616 ap=0.354253 base_ap=0.306637 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6064 delta_ap=0.008372 ap=0.315009 base_ap=0.306637 donor_failures=52 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=0.083283 ap=0.389921 base_ap=0.306637 donor_failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6064 delta_ap=-0.016455 ap=0.290182 base_ap=0.306637 donor_failures=136 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6064 delta_ap=0.053280 ap=0.359917 base_ap=0.306637 donor_failures=151 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6064 delta_ap=0.050196 ap=0.356833 base_ap=0.306637 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6064 delta_ap=0.046482 ap=0.353119 base_ap=0.306637 donor_failures=55 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.050838 ap=0.357476 base_ap=0.306637 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.068532 ap=0.375169 base_ap=0.306637 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=2 origin=6066 ap=0.245492 rows=1401 failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.001802 ap=0.247295 base_ap=0.245492 donor_failures=120 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.006345 ap=0.239147 base_ap=0.245492 donor_failures=125 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.019541 ap=0.265034 base_ap=0.245492 donor_failures=41 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.011837 ap=0.233656 base_ap=0.245492 donor_failures=77 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.044917 ap=0.200576 base_ap=0.245492 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.047330 ap=0.198162 base_ap=0.245492 donor_failures=64 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.009623 ap=0.235869 base_ap=0.245492 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6066 delta_ap=-0.021713 ap=0.223779 base_ap=0.245492 donor_failures=120 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6066 delta_ap=-0.045308 ap=0.200184 base_ap=0.245492 donor_failures=162 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6066 delta_ap=-0.000558 ap=0.244934 base_ap=0.245492 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6066 delta_ap=-0.003206 ap=0.242287 base_ap=0.245492 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.002547 ap=0.242946 base_ap=0.245492 donor_failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.008892 ap=0.236600 base_ap=0.245492 donor_failures=283 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_tay_ban_nha origin=6064 policy_sources=khoa_tieng_trung_quoc,khoa_cong_nghe_thong_tin,khoa_tieng_italia policy_donor_count=3 base_ap=0.306637 policy_ap=0.349745 delta_ap=0.043108 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_tay_ban_nha origin=6066 policy_sources=khoa_tieng_trung_quoc,khoa_cong_nghe_thong_tin,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.245492 policy_ap=0.245074 delta_ap=-0.000418 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_tay_ban_nha selected=['khoa_tieng_trung_quoc', 'khoa_cong_nghe_thong_tin', 'khoa_quan_tri_kinh_doanh_va_du_lich'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.021345 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common rows=1242 failures=42 prevalence=0.033816 ap=0.418102 auc=0.934018 f2_at20=0.431655 brier=0.025696 ece10=0.020305 rho=0.000000 donor_rows=0 effective_n=9426.000000 seconds=0.537048 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common in_sample_bce=0.018036 local_bce=0.018036 donor_bce=nan in_sample_local_ap=0.971666 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native rows=1242 failures=42 prevalence=0.033816 ap=0.355105 auc=0.914018 f2_at20=0.419664 brier=0.027800 ece10=0.019729 rho=0.000000 donor_rows=0 effective_n=9426.000000 seconds=0.918574 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native in_sample_bce=0.017958 local_bce=0.017958 donor_bce=nan in_sample_local_ap=0.970558 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all rows=1242 failures=42 prevalence=0.033816 ap=0.391042 auc=0.934107 f2_at20=0.479616 brier=0.025724 ece10=0.013819 rho=0.250000 donor_rows=4095 effective_n=12167.820228 seconds=0.564089 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all in_sample_bce=0.023714 local_bce=0.022638 donor_bce=0.046330 in_sample_local_ap=0.948899 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.021938595034759572, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03530550883246825, 'khoa_quoc_te_hoc': 0.011552233203150379, 'khoa_tieng_anh': 0.05067840719591692, 'khoa_tieng_bo_dao_nha': 0.0044081127321816595, 'khoa_tieng_duc': 0.011535088035921832, 'khoa_tieng_han_quoc': 0.018052958714437312, 'khoa_tieng_italia': 0.011564866484266152, 'khoa_tieng_nga': 0.007395883716061593, 'khoa_tieng_nhat': 0.019746720761173234, 'khoa_tieng_phap': 0.016126383344282177, 'khoa_tieng_trung_quoc': 0.036252102538928556, 'khoa_viet_nam_hoc': 0.005443139406452358} source_failures={'khoa_tieng_tay_ban_nha': 233, 'khoa_cong_nghe_thong_tin': 6, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 2, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 20} positive_ess=282.860095 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal rows=1242 failures=42 prevalence=0.033816 ap=0.444277 auc=0.934246 f2_at20=0.467626 brier=0.025331 ece10=0.014827 rho=0.250000 donor_rows=4095 effective_n=13344.385817 seconds=0.556905 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal in_sample_bce=0.027091 local_bce=0.024056 donor_bce=0.036195 in_sample_local_ap=0.944109 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_tay_ban_nha': 233, 'khoa_cong_nghe_thong_tin': 6, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 2, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 20} positive_ess=319.028686 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior rows=1242 failures=42 prevalence=0.033816 ap=0.378945 auc=0.931062 f2_at20=0.467626 brier=0.025880 ece10=0.015206 rho=0.250000 donor_rows=4095 effective_n=13344.385817 seconds=0.541828 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior in_sample_bce=0.025050 local_bce=0.023574 donor_bce=0.029481 in_sample_local_ap=0.947669 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_tay_ban_nha': 233, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_trung_quoc': 3} positive_ess=299.778570 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor rows=1242 failures=42 prevalence=0.033816 ap=0.457470 auc=0.919454 f2_at20=0.431655 brier=0.023831 ece10=0.011932 rho=0.250000 donor_rows=4095 effective_n=13344.385817 seconds=0.531396 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor in_sample_bce=0.031252 local_bce=0.024798 donor_bce=0.050613 in_sample_local_ap=0.941474 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor mass={'khoa_tieng_tay_ban_nha': 0.75, np.str_('khoa_viet_nam_hoc'): 0.08333333333333333, np.str_('khoa_tieng_italia'): 0.08333333333333333, np.str_('khoa_tieng_phap'): 0.08333333333333333} source_failures={'khoa_tieng_tay_ban_nha': 233, np.str_('khoa_viet_nam_hoc'): 80, np.str_('khoa_tieng_italia'): 43, np.str_('khoa_tieng_phap'): 14} positive_ess=364.488068 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common rows=1242 failures=42 prevalence=0.033816 ap=0.385250 auc=0.926161 f2_at20=0.467626 brier=0.026982 ece10=0.019514 rho=0.250000 donor_rows=4095 effective_n=13275.440129 seconds=0.562296 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common in_sample_bce=0.027099 local_bce=0.023467 donor_bce=0.040643 in_sample_local_ap=0.949213 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_trung_quoc': 0.09632146936246112, 'khoa_cong_nghe_thong_tin': 0.08863342258269452, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.06504510805484434} source_failures={'khoa_tieng_tay_ban_nha': 233, 'khoa_tieng_trung_quoc': 6, 'khoa_cong_nghe_thong_tin': 38, 'khoa_quan_tri_kinh_doanh_va_du_lich': 40} positive_ess=310.115328 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native rows=1242 failures=42 prevalence=0.033816 ap=0.422838 auc=0.912788 f2_at20=0.443645 brier=0.025348 ece10=0.017632 rho=0.250000 donor_rows=4095 effective_n=13275.440129 seconds=1.024187 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native in_sample_bce=0.027144 local_bce=0.022919 donor_bce=0.042393 in_sample_local_ap=0.950884 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_trung_quoc': 0.09632146936246112, 'khoa_cong_nghe_thong_tin': 0.08863342258269452, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.06504510805484434} source_failures={'khoa_tieng_tay_ban_nha': 233, 'khoa_tieng_trung_quoc': 6, 'khoa_cong_nghe_thong_tin': 38, 'khoa_quan_tri_kinh_doanh_va_du_lich': 40} positive_ess=310.115328 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator rows=1242 failures=42 prevalence=0.033816 ap=0.355105 auc=0.914018 f2_at20=0.419664 brier=0.027800 ece10=0.019729 rho=0.000000 donor_rows=0 effective_n=9426.000000 seconds=1.625291 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator in_sample_bce=0.017958 local_bce=0.017958 donor_bce=nan in_sample_local_ap=0.970558 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator rows=1242 failures=42 prevalence=0.033816 ap=0.352878 auc=0.908006 f2_at20=0.407674 brier=0.027309 ece10=0.018387 rho=0.000000 donor_rows=0 effective_n=9426.000000 seconds=1.731156 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator in_sample_bce=0.018118 local_bce=0.018118 donor_bce=nan in_sample_local_ap=0.970952 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator rows=1242 failures=42 prevalence=0.033816 ap=0.355105 auc=0.914018 f2_at20=0.419664 brier=0.027800 ece10=0.019729 rho=0.000000 donor_rows=0 effective_n=9426.000000 seconds=1.522928 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator in_sample_bce=0.017958 local_bce=0.017958 donor_bce=nan in_sample_local_ap=0.970558 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan rows=1242 failures=42 prevalence=0.033816 ap=0.422838 auc=0.912788 f2_at20=0.443645 brier=0.025348 ece10=0.017632 rho=0.250000 donor_rows=4095 effective_n=13275.440129 seconds=1.850058 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan in_sample_bce=0.027144 local_bce=0.022919 donor_bce=0.042393 in_sample_local_ap=0.950884 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_trung_quoc': 0.09632146936246112, 'khoa_cong_nghe_thong_tin': 0.08863342258269452, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.06504510805484434} source_failures={'khoa_tieng_tay_ban_nha': 233, 'khoa_tieng_trung_quoc': 6, 'khoa_cong_nghe_thong_tin': 38, 'khoa_quan_tri_kinh_doanh_va_du_lich': 40} positive_ess=310.115328 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_tay_ban_nha model=committee_reference ap=0.377620 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_tay_ban_nha models=13 positive_rows=42 num_edges=52 num_replays=2 chosen=['khoa_tieng_trung_quoc', 'khoa_cong_nghe_thong_tin', 'khoa_quan_tri_kinh_doanh_va_du_lich'] [SELECTION_ORIGINS] target=khoa_tieng_trung_quoc fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=2 origin=6061 ap=0.594771 rows=4785 failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=0.002973 ap=0.597743 base_ap=0.594771 donor_failures=186 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=-0.001423 ap=0.593348 base_ap=0.594771 donor_failures=96 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.097990 ap=0.496781 base_ap=0.594771 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6061 delta_ap=0.008095 ap=0.602865 base_ap=0.594771 donor_failures=57 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=0.001491 ap=0.596261 base_ap=0.594771 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6061 delta_ap=-0.056708 ap=0.538062 base_ap=0.594771 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6061 delta_ap=-0.112017 ap=0.482753 base_ap=0.594771 donor_failures=31 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6061 delta_ap=-0.155947 ap=0.438824 base_ap=0.594771 donor_failures=157 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6061 delta_ap=-0.060667 ap=0.534104 base_ap=0.594771 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6061 delta_ap=-0.106608 ap=0.488163 base_ap=0.594771 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6061 delta_ap=-0.104116 ap=0.490655 base_ap=0.594771 donor_failures=64 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=0.021247 ap=0.616018 base_ap=0.594771 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6061 delta_ap=0.039341 ap=0.634112 base_ap=0.594771 donor_failures=180 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=2 origin=6063 ap=0.038175 rows=7413 failures=13 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.077833 ap=0.116008 base_ap=0.038175 donor_failures=168 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.050609 ap=0.088784 base_ap=0.038175 donor_failures=84 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.069428 ap=0.107603 base_ap=0.038175 donor_failures=35 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6063 delta_ap=0.115352 ap=0.153527 base_ap=0.038175 donor_failures=70 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.010886 ap=0.027289 base_ap=0.038175 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6063 delta_ap=0.061655 ap=0.099830 base_ap=0.038175 donor_failures=67 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.101119 ap=0.139294 base_ap=0.038175 donor_failures=17 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6063 delta_ap=0.115647 ap=0.153822 base_ap=0.038175 donor_failures=137 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6063 delta_ap=0.050253 ap=0.088429 base_ap=0.038175 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6063 delta_ap=0.000927 ap=0.039103 base_ap=0.038175 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6063 delta_ap=0.085427 ap=0.123602 base_ap=0.038175 donor_failures=46 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.094668 ap=0.132843 base_ap=0.038175 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.084352 ap=0.122527 base_ap=0.038175 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=2 origin=6064 ap=0.270711 rows=5347 failures=29 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.132687 ap=0.138024 base_ap=0.270711 donor_failures=135 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.064149 ap=0.206562 base_ap=0.270711 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.061499 ap=0.209211 base_ap=0.270711 donor_failures=36 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6064 delta_ap=-0.006453 ap=0.264258 base_ap=0.270711 donor_failures=74 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.087928 ap=0.182783 base_ap=0.270711 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6064 delta_ap=-0.100145 ap=0.170565 base_ap=0.270711 donor_failures=52 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.021229 ap=0.249482 base_ap=0.270711 donor_failures=16 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6064 delta_ap=-0.046534 ap=0.224177 base_ap=0.270711 donor_failures=118 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.058408 ap=0.212303 base_ap=0.270711 donor_failures=150 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.076203 ap=0.346914 base_ap=0.270711 donor_failures=44 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.085540 ap=0.185171 base_ap=0.270711 donor_failures=39 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.022372 ap=0.293083 base_ap=0.270711 donor_failures=83 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.099259 ap=0.171452 base_ap=0.270711 donor_failures=270 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=2 origin=6066 ap=0.287191 rows=7393 failures=35 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.161726 ap=0.125465 base_ap=0.287191 donor_failures=133 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.226983 ap=0.060208 base_ap=0.287191 donor_failures=119 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.109685 ap=0.177505 base_ap=0.287191 donor_failures=33 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6066 delta_ap=-0.191097 ap=0.096093 base_ap=0.287191 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.143551 ap=0.143640 base_ap=0.287191 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6066 delta_ap=-0.124222 ap=0.162968 base_ap=0.287191 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.165554 ap=0.121637 base_ap=0.287191 donor_failures=12 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6066 delta_ap=-0.042648 ap=0.244542 base_ap=0.287191 donor_failures=125 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6066 delta_ap=-0.216075 ap=0.071116 base_ap=0.287191 donor_failures=171 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.222548 ap=0.064643 base_ap=0.287191 donor_failures=41 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6066 delta_ap=-0.131821 ap=0.155370 base_ap=0.287191 donor_failures=43 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.205630 ap=0.081561 base_ap=0.287191 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.161887 ap=0.125303 base_ap=0.287191 donor_failures=265 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_trung_quoc origin=6064 policy_sources=khoa_viet_nam_hoc,khoa_tieng_anh,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.270711 policy_ap=0.273201 delta_ap=0.002491 policy_donor_rows=4095 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_tieng_trung_quoc origin=6066 policy_sources=khoa_tieng_tay_ban_nha,khoa_tieng_anh,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.287191 policy_ap=0.209242 delta_ap=-0.077949 policy_donor_rows=4095 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_tieng_trung_quoc selected=[] reason=no_stable_positive_directed_edge replay_mean=-0.037729 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_local_common rows=5584 failures=76 prevalence=0.013610 ap=0.101745 auc=0.830138 f2_at20=0.175932 brier=0.013286 ece10=0.012020 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=0.818844 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_local_common in_sample_bce=0.005690 local_bce=0.005690 donor_bce=nan in_sample_local_ap=0.934230 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_local_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_local_native rows=5584 failures=76 prevalence=0.013610 ap=0.112376 auc=0.818749 f2_at20=0.172414 brier=0.013182 ece10=0.011880 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=2.178220 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_local_native in_sample_bce=0.005605 local_bce=0.005605 donor_bce=nan in_sample_local_ap=0.938249 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_local_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_pool_all rows=5584 failures=76 prevalence=0.013610 ap=0.111344 auc=0.807482 f2_at20=0.158339 brier=0.013080 ece10=0.010364 rho=0.250000 donor_rows=4095 effective_n=26671.808166 seconds=0.811512 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_pool_all in_sample_bce=0.012234 local_bce=0.009513 donor_bce=0.035160 in_sample_local_ap=0.831413 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_pool_all mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.024677423283989315, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03971307115770327, 'khoa_quoc_te_hoc': 0.012994421391972327, 'khoa_tieng_anh': 0.05700513199457568, 'khoa_tieng_bo_dao_nha': 0.004958424347741354, 'khoa_tieng_duc': 0.012975135811090631, 'khoa_tieng_han_quoc': 0.020306701637853333, 'khoa_tieng_italia': 0.013008631819990417, 'khoa_tieng_nga': 0.008319190574020089, 'khoa_tieng_nhat': 0.022211914022850367, 'khoa_tieng_phap': 0.018139611365094316, 'khoa_tieng_tay_ban_nha': 0.009567678178466736, 'khoa_viet_nam_hoc': 0.006122664414652169} source_failures={'khoa_tieng_trung_quoc': 220, 'khoa_cong_nghe_thong_tin': 14, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 7, 'khoa_viet_nam_hoc': 21} positive_ess=169.482582 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal rows=5584 failures=76 prevalence=0.013610 ap=0.070446 auc=0.849518 f2_at20=0.165376 brier=0.013410 ece10=0.010901 rho=0.250000 donor_rows=4095 effective_n=34171.552013 seconds=0.819036 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal in_sample_bce=0.013957 local_bce=0.010275 donor_bce=0.025004 in_sample_local_ap=0.809467 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_trung_quoc': 220, 'khoa_cong_nghe_thong_tin': 14, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 7, 'khoa_viet_nam_hoc': 21} positive_ess=238.074604 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_language_prior rows=5584 failures=76 prevalence=0.013610 ap=0.068743 auc=0.884059 f2_at20=0.228712 brier=0.014011 ece10=0.011088 rho=0.250000 donor_rows=4095 effective_n=34171.552013 seconds=0.806328 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_language_prior in_sample_bce=0.012303 local_bce=0.009990 donor_bce=0.019243 in_sample_local_ap=0.812186 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_language_prior mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776} source_failures={'khoa_tieng_trung_quoc': 220, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 20, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 7, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 12} positive_ess=225.352766 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_random_donor rows=5584 failures=76 prevalence=0.013610 ap=0.106732 auc=0.862054 f2_at20=0.204082 brier=0.013167 ece10=0.010948 rho=0.250000 donor_rows=4096 effective_n=34175.903632 seconds=0.806927 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_random_donor in_sample_bce=0.023322 local_bce=0.011032 donor_bce=0.060192 in_sample_local_ap=0.785979 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=hgb_random_donor mass={'khoa_tieng_trung_quoc': 0.75, np.str_('khoa_viet_nam_hoc'): 0.25} source_failures={'khoa_tieng_trung_quoc': 220, np.str_('khoa_viet_nam_hoc'): 254} positive_ess=375.945828 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_common rows=5584 failures=76 prevalence=0.013610 ap=0.101745 auc=0.830138 f2_at20=0.175932 brier=0.013286 ece10=0.012020 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=0.806618 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_common in_sample_bce=0.005690 local_bce=0.005690 donor_bce=nan in_sample_local_ap=0.934230 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_native rows=5584 failures=76 prevalence=0.013610 ap=0.112376 auc=0.818749 f2_at20=0.172414 brier=0.013182 ece10=0.011880 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=1.954145 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_native in_sample_bce=0.005605 local_bce=0.005605 donor_bce=nan in_sample_local_ap=0.938249 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator rows=5584 failures=76 prevalence=0.013610 ap=0.112376 auc=0.818749 f2_at20=0.172414 brier=0.013182 ece10=0.011880 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=3.573898 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator in_sample_bce=0.005605 local_bce=0.005605 donor_bce=nan in_sample_local_ap=0.938249 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator rows=5584 failures=76 prevalence=0.013610 ap=0.112352 auc=0.818747 f2_at20=0.172414 brier=0.013182 ece10=0.011879 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=3.753985 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator in_sample_bce=0.005605 local_bce=0.005605 donor_bce=nan in_sample_local_ap=0.938249 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator rows=5584 failures=76 prevalence=0.013610 ap=0.112376 auc=0.818749 f2_at20=0.172414 brier=0.013182 ece10=0.011880 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=3.852296 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator in_sample_bce=0.005605 local_bce=0.005605 donor_bce=nan in_sample_local_ap=0.938249 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan rows=5584 failures=76 prevalence=0.013610 ap=0.112376 auc=0.818749 f2_at20=0.172414 brier=0.013182 ece10=0.011880 rho=0.000000 donor_rows=0 effective_n=40174.000000 seconds=3.728726 [TRAINING_INSIGHT] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.005605 local_bce=0.005605 donor_bce=nan in_sample_local_ap=0.938249 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 220} positive_ess=220.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_tieng_trung_quoc model=committee_reference ap=0.091000 [FACULTY_COMPLETE] fold=2 faculty=khoa_tieng_trung_quoc models=13 positive_rows=76 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_viet_nam_hoc fold=2 origins=[6061, 6063, 6064, 6066] availability=semester_proxy [SCREEN_BASE] target=khoa_viet_nam_hoc fold=2 origin=6061 ap=0.690536 rows=866 failures=37 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6061 delta_ap=-0.006276 ap=0.684260 base_ap=0.690536 donor_failures=188 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6061 delta_ap=0.027643 ap=0.718179 base_ap=0.690536 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6061 delta_ap=-0.032250 ap=0.658286 base_ap=0.690536 donor_failures=32 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6061 delta_ap=-0.002852 ap=0.687685 base_ap=0.690536 donor_failures=47 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6061 delta_ap=-0.080391 ap=0.610145 base_ap=0.690536 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6061 delta_ap=0.014842 ap=0.705378 base_ap=0.690536 donor_failures=79 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6061 delta_ap=0.025682 ap=0.716219 base_ap=0.690536 donor_failures=27 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6061 delta_ap=0.006470 ap=0.697007 base_ap=0.690536 donor_failures=156 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6061 delta_ap=-0.020315 ap=0.670221 base_ap=0.690536 donor_failures=142 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6061 delta_ap=-0.008038 ap=0.682498 base_ap=0.690536 donor_failures=66 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6061 delta_ap=-0.017742 ap=0.672794 base_ap=0.690536 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6061 delta_ap=-0.023664 ap=0.666872 base_ap=0.690536 donor_failures=86 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6061 delta_ap=0.004744 ap=0.695280 base_ap=0.690536 donor_failures=32 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=2 origin=6063 ap=0.559746 rows=993 failures=71 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.050310 ap=0.509436 base_ap=0.559746 donor_failures=170 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.126454 ap=0.433293 base_ap=0.559746 donor_failures=90 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.117946 ap=0.441801 base_ap=0.559746 donor_failures=32 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6063 delta_ap=-0.018902 ap=0.540845 base_ap=0.559746 donor_failures=65 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.124623 ap=0.435124 base_ap=0.559746 donor_failures=82 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6063 delta_ap=-0.035244 ap=0.524502 base_ap=0.559746 donor_failures=63 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.080491 ap=0.479255 base_ap=0.559746 donor_failures=21 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6063 delta_ap=-0.097284 ap=0.462463 base_ap=0.559746 donor_failures=149 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6063 delta_ap=-0.042715 ap=0.517032 base_ap=0.559746 donor_failures=172 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6063 delta_ap=-0.112785 ap=0.446962 base_ap=0.559746 donor_failures=50 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6063 delta_ap=-0.138597 ap=0.421150 base_ap=0.559746 donor_failures=54 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.081657 ap=0.478090 base_ap=0.559746 donor_failures=87 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.061725 ap=0.498021 base_ap=0.559746 donor_failures=32 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=2 origin=6064 ap=0.577804 rows=818 failures=49 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.012084 ap=0.565720 base_ap=0.577804 donor_failures=107 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.021601 ap=0.556203 base_ap=0.577804 donor_failures=96 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.035917 ap=0.541887 base_ap=0.577804 donor_failures=30 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6064 delta_ap=-0.022142 ap=0.555662 base_ap=0.577804 donor_failures=79 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.033713 ap=0.611517 base_ap=0.577804 donor_failures=97 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6064 delta_ap=-0.038821 ap=0.538983 base_ap=0.577804 donor_failures=54 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.020396 ap=0.557408 base_ap=0.577804 donor_failures=24 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6064 delta_ap=-0.007987 ap=0.569817 base_ap=0.577804 donor_failures=122 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6064 delta_ap=-0.021933 ap=0.555871 base_ap=0.577804 donor_failures=152 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6064 delta_ap=-0.022917 ap=0.554887 base_ap=0.577804 donor_failures=44 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6064 delta_ap=-0.031064 ap=0.546740 base_ap=0.577804 donor_failures=44 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.006546 ap=0.584350 base_ap=0.577804 donor_failures=81 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.036255 ap=0.541549 base_ap=0.577804 donor_failures=15 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=2 origin=6066 ap=0.178075 rows=813 failures=30 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.031347 ap=0.209422 base_ap=0.178075 donor_failures=131 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.006102 ap=0.171974 base_ap=0.178075 donor_failures=104 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.051256 ap=0.126819 base_ap=0.178075 donor_failures=37 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6066 delta_ap=-0.009662 ap=0.168413 base_ap=0.178075 donor_failures=61 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.054432 ap=0.123643 base_ap=0.178075 donor_failures=123 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.022580 ap=0.200655 base_ap=0.178075 donor_failures=74 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.011232 ap=0.166843 base_ap=0.178075 donor_failures=23 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6066 delta_ap=-0.045356 ap=0.132719 base_ap=0.178075 donor_failures=113 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.011388 ap=0.189463 base_ap=0.178075 donor_failures=167 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6066 delta_ap=0.027002 ap=0.205077 base_ap=0.178075 donor_failures=39 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6066 delta_ap=-0.003410 ap=0.174665 base_ap=0.178075 donor_failures=58 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.043405 ap=0.134670 base_ap=0.178075 donor_failures=91 [TRANSFER_EDGE] fold=2 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.015441 ap=0.193516 base_ap=0.178075 donor_failures=16 [POLICY_REPLAY] fold=2 faculty=khoa_viet_nam_hoc origin=6064 policy_sources= policy_donor_count=0 base_ap=0.577804 policy_ap=0.577804 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6061,6063 [POLICY_REPLAY] fold=2 faculty=khoa_viet_nam_hoc origin=6066 policy_sources= policy_donor_count=0 base_ap=0.178075 policy_ap=0.178075 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6061,6063,6064 [SELECTION_FINAL] fold=2 faculty=khoa_viet_nam_hoc selected=[] reason=no_stable_positive_directed_edge replay_mean=0.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_local_common rows=556 failures=39 prevalence=0.070144 ap=0.451298 auc=0.785126 f2_at20=0.466418 brier=0.055842 ece10=0.052163 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.402960 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=hgb_local_common in_sample_bce=0.055067 local_bce=0.055067 donor_bce=nan in_sample_local_ap=0.935846 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_local_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_local_native rows=556 failures=39 prevalence=0.070144 ap=0.427956 auc=0.789466 f2_at20=0.466418 brier=0.056860 ece10=0.050136 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.420552 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=hgb_local_native in_sample_bce=0.055212 local_bce=0.055212 donor_bce=nan in_sample_local_ap=0.933789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_local_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_pool_all rows=556 failures=39 prevalence=0.070144 ap=0.347531 auc=0.760576 f2_at20=0.466418 brier=0.059031 ece10=0.051529 rho=0.250000 donor_rows=4095 effective_n=8669.108556 seconds=0.531613 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=hgb_pool_all in_sample_bce=0.058798 local_bce=0.062889 donor_bce=0.056673 in_sample_local_ap=0.904960 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_pool_all mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02167308515190415, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0348782270717444, 'khoa_quoc_te_hoc': 0.011412423334759664, 'khoa_tieng_anh': 0.05006507630865782, 'khoa_tieng_bo_dao_nha': 0.004354763942376266, 'khoa_tieng_duc': 0.01139548566538297, 'khoa_tieng_han_quoc': 0.017834474397375552, 'khoa_tieng_italia': 0.011424903722721438, 'khoa_tieng_nga': 0.0073063756953359, 'khoa_tieng_nhat': 0.0195077378405363, 'khoa_tieng_phap': 0.015931215233204964, 'khoa_tieng_tay_ban_nha': 0.008402866923406076, 'khoa_tieng_trung_quoc': 0.0358133647125945} source_failures={'khoa_viet_nam_hoc': 367, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 9, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 3, 'khoa_tieng_trung_quoc': 4} positive_ess=417.850233 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_pool_equal rows=556 failures=39 prevalence=0.070144 ap=0.394585 auc=0.772405 f2_at20=0.466418 brier=0.056847 ece10=0.047566 rho=0.250000 donor_rows=4095 effective_n=9215.301607 seconds=0.520153 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=hgb_pool_equal in_sample_bce=0.060756 local_bce=0.063015 donor_bce=0.053982 in_sample_local_ap=0.901466 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_pool_equal mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232} source_failures={'khoa_viet_nam_hoc': 367, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 9, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 3, 'khoa_tieng_trung_quoc': 4} positive_ess=428.841662 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_language_prior rows=556 failures=39 prevalence=0.070144 ap=0.451298 auc=0.785126 f2_at20=0.466418 brier=0.055842 ece10=0.052163 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.401800 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=hgb_language_prior in_sample_bce=0.055067 local_bce=0.055067 donor_bce=nan in_sample_local_ap=0.935846 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_language_prior mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_random_donor rows=556 failures=39 prevalence=0.070144 ap=0.421286 auc=0.763949 f2_at20=0.466418 brier=0.056845 ece10=0.050251 rho=0.250000 donor_rows=4096 effective_n=9215.618053 seconds=0.496227 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=hgb_random_donor in_sample_bce=0.070922 local_bce=0.065367 donor_bce=0.087586 in_sample_local_ap=0.897462 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=hgb_random_donor mass={'khoa_viet_nam_hoc': 0.75, np.str_('khoa_tieng_nga'): 0.25} source_failures={'khoa_viet_nam_hoc': 367, np.str_('khoa_tieng_nga'): 200} positive_ess=521.177753 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_common rows=556 failures=39 prevalence=0.070144 ap=0.451298 auc=0.785126 f2_at20=0.466418 brier=0.055842 ece10=0.052163 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.407513 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_common in_sample_bce=0.055067 local_bce=0.055067 donor_bce=nan in_sample_local_ap=0.935846 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_native rows=556 failures=39 prevalence=0.070144 ap=0.427956 auc=0.789466 f2_at20=0.466418 brier=0.056860 ece10=0.050136 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.433039 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_native in_sample_bce=0.055212 local_bce=0.055212 donor_bce=nan in_sample_local_ap=0.933789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator rows=556 failures=39 prevalence=0.070144 ap=0.427956 auc=0.789466 f2_at20=0.466418 brier=0.056860 ece10=0.050136 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.692903 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator in_sample_bce=0.055212 local_bce=0.055212 donor_bce=nan in_sample_local_ap=0.933789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator rows=556 failures=39 prevalence=0.070144 ap=0.427956 auc=0.789466 f2_at20=0.466418 brier=0.056860 ece10=0.050136 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.755996 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator in_sample_bce=0.055212 local_bce=0.055212 donor_bce=nan in_sample_local_ap=0.933789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=native_placebo_indicator rows=556 failures=39 prevalence=0.070144 ap=0.427956 auc=0.789466 f2_at20=0.466418 brier=0.056860 ece10=0.050136 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.681618 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=native_placebo_indicator in_sample_bce=0.055212 local_bce=0.055212 donor_bce=nan in_sample_local_ap=0.933789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=native_placebo_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan rows=556 failures=39 prevalence=0.070144 ap=0.427956 auc=0.789466 f2_at20=0.466418 brier=0.056860 ece10=0.050136 rho=0.000000 donor_rows=0 effective_n=6032.000000 seconds=0.703069 [TRAINING_INSIGHT] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.055212 local_bce=0.055212 donor_bce=nan in_sample_local_ap=0.933789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=2 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 367} positive_ess=367.000000 [FACULTY_RESULTS] fold=2 faculty=khoa_viet_nam_hoc model=committee_reference ap=0.482981 [FACULTY_COMPLETE] fold=2 faculty=khoa_viet_nam_hoc models=13 positive_rows=39 num_edges=52 num_replays=2 chosen=[] [FOLD_START] fold=3 stage=development [DATA] fold=3 common_raw=136 course_features=17 common_total=153 faculties=14 history_rows=329579 target_rows=47420 target_failures=1103 history_last=6068 target_first=6069 [OBSERVATION_AUDIT] fold=3 faculty=khoa_cong_nghe_thong_tin raw_columns=275 common_raw_columns=136 native_raw_columns=139 history_rows=28910 history_failures=917 target_rows=6277 target_failures=218 target_prevalence=0.034730 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.656104 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich raw_columns=301 common_raw_columns=136 native_raw_columns=165 history_rows=45639 history_failures=1334 target_rows=6491 target_failures=209 target_prevalence=0.032198 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.691904 history_latest_semester=6068 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_quoc_te_hoc raw_columns=254 common_raw_columns=136 native_raw_columns=118 history_rows=14897 history_failures=148 target_rows=2665 target_failures=26 target_prevalence=0.009756 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.621570 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_anh raw_columns=511 common_raw_columns=136 native_raw_columns=375 history_rows=64490 history_failures=1091 target_rows=7279 target_failures=101 target_prevalence=0.013876 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.808049 history_latest_semester=6068 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_bo_dao_nha raw_columns=400 common_raw_columns=136 native_raw_columns=264 history_rows=5601 history_failures=183 target_rows=1032 target_failures=50 target_prevalence=0.048450 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.751928 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_duc raw_columns=381 common_raw_columns=136 native_raw_columns=245 history_rows=14676 history_failures=271 target_rows=1909 target_failures=47 target_prevalence=0.024620 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.748122 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_han_quoc raw_columns=333 common_raw_columns=136 native_raw_columns=197 history_rows=23343 history_failures=124 target_rows=3235 target_failures=42 target_prevalence=0.012983 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.683090 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_italia raw_columns=420 common_raw_columns=136 native_raw_columns=284 history_rows=14840 history_failures=508 target_rows=2145 target_failures=68 target_prevalence=0.031702 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.742879 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_nga raw_columns=469 common_raw_columns=136 native_raw_columns=333 history_rows=9763 history_failures=455 target_rows=1996 target_failures=92 target_prevalence=0.046092 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.791992 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_nhat raw_columns=362 common_raw_columns=136 native_raw_columns=226 history_rows=24331 history_failures=227 target_rows=2588 target_failures=17 target_prevalence=0.006569 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.741412 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_phap raw_columns=495 common_raw_columns=136 native_raw_columns=359 history_rows=20075 history_failures=260 target_rows=2693 target_failures=43 target_prevalence=0.015967 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.787668 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_tay_ban_nha raw_columns=396 common_raw_columns=136 native_raw_columns=260 history_rows=10668 history_failures=275 target_rows=1417 target_failures=54 target_prevalence=0.038109 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.746694 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_tieng_trung_quoc raw_columns=436 common_raw_columns=136 native_raw_columns=300 history_rows=45758 history_failures=296 target_rows=6920 target_failures=103 target_prevalence=0.014884 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.759712 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=3 faculty=khoa_viet_nam_hoc raw_columns=181 common_raw_columns=136 native_raw_columns=45 history_rows=6588 history_failures=406 target_rows=773 target_failures=33 target_prevalence=0.042691 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.497264 history_latest_semester=6067 target_first_semester=6069 availability_provenance=semester_proxy; source release timestamps not verified [SELECTION_ORIGINS] target=khoa_cong_nghe_thong_tin fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=3 origin=6063 ap=0.058647 rows=5277 failures=63 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.048334 ap=0.106981 base_ap=0.058647 donor_failures=95 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.006423 ap=0.052224 base_ap=0.058647 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6063 delta_ap=0.008379 ap=0.067025 base_ap=0.058647 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.004224 ap=0.062870 base_ap=0.058647 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6063 delta_ap=-0.007157 ap=0.051489 base_ap=0.058647 donor_failures=70 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6063 delta_ap=0.000367 ap=0.059013 base_ap=0.058647 donor_failures=13 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6063 delta_ap=0.009159 ap=0.067805 base_ap=0.058647 donor_failures=146 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6063 delta_ap=0.010327 ap=0.068974 base_ap=0.058647 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6063 delta_ap=0.013178 ap=0.071825 base_ap=0.058647 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6063 delta_ap=0.020410 ap=0.079057 base_ap=0.058647 donor_failures=50 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.004504 ap=0.063150 base_ap=0.058647 donor_failures=83 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.000248 ap=0.058398 base_ap=0.058647 donor_failures=29 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.008562 ap=0.050085 base_ap=0.058647 donor_failures=217 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=3 origin=6064 ap=0.301541 rows=3992 failures=116 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.028106 ap=0.273435 base_ap=0.301541 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.027013 ap=0.274528 base_ap=0.301541 donor_failures=33 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6064 delta_ap=-0.020913 ap=0.280627 base_ap=0.301541 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.010683 ap=0.290857 base_ap=0.301541 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6064 delta_ap=-0.012585 ap=0.288956 base_ap=0.301541 donor_failures=50 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.034059 ap=0.267482 base_ap=0.301541 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6064 delta_ap=-0.060473 ap=0.241068 base_ap=0.301541 donor_failures=128 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6064 delta_ap=-0.011440 ap=0.290101 base_ap=0.301541 donor_failures=138 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6064 delta_ap=-0.045881 ap=0.255660 base_ap=0.301541 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6064 delta_ap=-0.027183 ap=0.274357 base_ap=0.301541 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.050050 ap=0.251491 base_ap=0.301541 donor_failures=79 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.043132 ap=0.258408 base_ap=0.301541 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.086082 ap=0.215458 base_ap=0.301541 donor_failures=269 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=3 origin=6066 ap=0.265097 rows=5222 failures=189 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.019763 ap=0.245334 base_ap=0.265097 donor_failures=129 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6066 delta_ap=0.034999 ap=0.300095 base_ap=0.265097 donor_failures=29 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6066 delta_ap=-0.055607 ap=0.209489 base_ap=0.265097 donor_failures=71 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.015938 ap=0.281034 base_ap=0.265097 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6066 delta_ap=-0.051442 ap=0.213654 base_ap=0.265097 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6066 delta_ap=0.027458 ap=0.292555 base_ap=0.265097 donor_failures=18 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6066 delta_ap=-0.011501 ap=0.253595 base_ap=0.265097 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6066 delta_ap=0.008829 ap=0.273926 base_ap=0.265097 donor_failures=156 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6066 delta_ap=-0.084712 ap=0.180384 base_ap=0.265097 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6066 delta_ap=0.002324 ap=0.267421 base_ap=0.265097 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.003853 ap=0.261244 base_ap=0.265097 donor_failures=78 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.006006 ap=0.271102 base_ap=0.265097 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6066 delta_ap=0.033477 ap=0.298574 base_ap=0.265097 donor_failures=257 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=3 origin=6067 ap=0.375590 rows=4598 failures=152 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.023606 ap=0.399196 base_ap=0.375590 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.023411 ap=0.352179 base_ap=0.375590 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6067 delta_ap=-0.050985 ap=0.324605 base_ap=0.375590 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.019764 ap=0.355826 base_ap=0.375590 donor_failures=136 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6067 delta_ap=0.017582 ap=0.393172 base_ap=0.375590 donor_failures=71 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6067 delta_ap=0.008220 ap=0.383810 base_ap=0.375590 donor_failures=15 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6067 delta_ap=-0.011157 ap=0.364433 base_ap=0.375590 donor_failures=143 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6067 delta_ap=0.015402 ap=0.390992 base_ap=0.375590 donor_failures=200 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6067 delta_ap=-0.030020 ap=0.345570 base_ap=0.375590 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6067 delta_ap=0.019229 ap=0.394819 base_ap=0.375590 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.002957 ap=0.372633 base_ap=0.375590 donor_failures=106 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.005474 ap=0.370116 base_ap=0.375590 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6067 delta_ap=0.009678 ap=0.385268 base_ap=0.375590 donor_failures=244 [POLICY_REPLAY] fold=3 faculty=khoa_cong_nghe_thong_tin origin=6066 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=1 base_ap=0.265097 policy_ap=0.245334 delta_ap=-0.019763 policy_donor_rows=4096 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_cong_nghe_thong_tin origin=6067 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_nga policy_donor_count=2 base_ap=0.375590 policy_ap=0.380487 delta_ap=0.004897 policy_donor_rows=4096 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_cong_nghe_thong_tin selected=[] reason=prequential_policy_delta_-0.007433_below_threshold replay_mean=-0.007433 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common rows=6277 failures=218 prevalence=0.034730 ap=0.252431 auc=0.863884 f2_at20=0.380639 brier=0.029523 ece10=0.007148 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=0.686856 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common in_sample_bce=0.058915 local_bce=0.058915 donor_bce=nan in_sample_local_ap=0.782562 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native rows=6277 failures=218 prevalence=0.034730 ap=0.244735 auc=0.866675 f2_at20=0.368891 brier=0.029440 ece10=0.005610 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=1.060341 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native in_sample_bce=0.057719 local_bce=0.057719 donor_bce=nan in_sample_local_ap=0.792003 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all rows=6277 failures=218 prevalence=0.034730 ap=0.268734 auc=0.871905 f2_at20=0.397086 brier=0.029081 ece10=0.005622 rho=0.250000 donor_rows=4095 effective_n=23149.257101 seconds=0.758729 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all in_sample_bce=0.057024 local_bce=0.064746 donor_bce=0.057112 in_sample_local_ap=0.737409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03794787623599374, 'khoa_quoc_te_hoc': 0.012386544672048, 'khoa_tieng_anh': 0.053622089407288416, 'khoa_tieng_bo_dao_nha': 0.004657114634365365, 'khoa_tieng_duc': 0.012202787783243367, 'khoa_tieng_han_quoc': 0.019409217445097434, 'khoa_tieng_italia': 0.012339150361360832, 'khoa_tieng_nga': 0.008117730793663463, 'khoa_tieng_nhat': 0.020230718830341672, 'khoa_tieng_phap': 0.01669194363236649, 'khoa_tieng_tay_ban_nha': 0.008870219410714107, 'khoa_tieng_trung_quoc': 0.03804682225304238, 'khoa_viet_nam_hoc': 0.005477784540474742} source_failures={'khoa_cong_nghe_thong_tin': 917, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 22, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 0, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 23} positive_ess=764.466627 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal rows=6277 failures=218 prevalence=0.034730 ap=0.284751 auc=0.875386 f2_at20=0.385338 brier=0.029034 ece10=0.010556 rho=0.250000 donor_rows=4095 effective_n=28802.299095 seconds=0.747165 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal in_sample_bce=0.061245 local_bce=0.065439 donor_bce=0.048665 in_sample_local_ap=0.731743 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal mass={'khoa_cong_nghe_thong_tin': 0.7500000000000001, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_cong_nghe_thong_tin': 917, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 22, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 0, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 23} positive_ess=904.014361 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior rows=6277 failures=218 prevalence=0.034730 ap=0.252431 auc=0.863884 f2_at20=0.380639 brier=0.029523 ece10=0.007148 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=0.950462 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior in_sample_bce=0.058915 local_bce=0.058915 donor_bce=nan in_sample_local_ap=0.782562 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor rows=6277 failures=218 prevalence=0.034730 ap=0.257885 auc=0.866583 f2_at20=0.385338 brier=0.030008 ece10=0.009642 rho=0.250000 donor_rows=4096 effective_n=28805.390580 seconds=0.788945 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor in_sample_bce=0.051189 local_bce=0.062363 donor_bce=0.017668 in_sample_local_ap=0.753438 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor mass={'khoa_cong_nghe_thong_tin': 0.75, np.str_('khoa_tieng_trung_quoc'): 0.25} source_failures={'khoa_cong_nghe_thong_tin': 917, np.str_('khoa_tieng_trung_quoc'): 31} positive_ess=900.217344 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common rows=6277 failures=218 prevalence=0.034730 ap=0.252431 auc=0.863884 f2_at20=0.380639 brier=0.029523 ece10=0.007148 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=0.728838 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common in_sample_bce=0.058915 local_bce=0.058915 donor_bce=nan in_sample_local_ap=0.782562 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native rows=6277 failures=218 prevalence=0.034730 ap=0.244735 auc=0.866675 f2_at20=0.368891 brier=0.029440 ece10=0.005610 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=1.303066 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native in_sample_bce=0.057719 local_bce=0.057719 donor_bce=nan in_sample_local_ap=0.792003 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator rows=6277 failures=218 prevalence=0.034730 ap=0.244735 auc=0.866675 f2_at20=0.368891 brier=0.029440 ece10=0.005610 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=2.171841 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator in_sample_bce=0.057719 local_bce=0.057719 donor_bce=nan in_sample_local_ap=0.792003 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator rows=6277 failures=218 prevalence=0.034730 ap=0.257488 auc=0.871325 f2_at20=0.378289 brier=0.029130 ece10=0.005555 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=1.884017 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator in_sample_bce=0.057823 local_bce=0.057823 donor_bce=nan in_sample_local_ap=0.791524 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator rows=6277 failures=218 prevalence=0.034730 ap=0.244735 auc=0.866675 f2_at20=0.368891 brier=0.029440 ece10=0.005610 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=2.058404 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator in_sample_bce=0.057719 local_bce=0.057719 donor_bce=nan in_sample_local_ap=0.792003 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan rows=6277 failures=218 prevalence=0.034730 ap=0.244735 auc=0.866675 f2_at20=0.368891 brier=0.029440 ece10=0.005610 rho=0.000000 donor_rows=0 effective_n=28910.000000 seconds=2.041946 [TRAINING_INSIGHT] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan in_sample_bce=0.057719 local_bce=0.057719 donor_bce=nan in_sample_local_ap=0.792003 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 917} positive_ess=917.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_cong_nghe_thong_tin model=committee_reference ap=0.273839 [FACULTY_COMPLETE] fold=3 faculty=khoa_cong_nghe_thong_tin models=13 positive_rows=218 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=3 origins=[6065, 6066, 6067, 6068] availability=semester_proxy [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=3 origin=6065 ap=0.615288 rows=687 failures=21 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=-0.106854 ap=0.508434 base_ap=0.615288 donor_failures=114 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6065 delta_ap=-0.105412 ap=0.509876 base_ap=0.615288 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6065 delta_ap=-0.058827 ap=0.556461 base_ap=0.615288 donor_failures=84 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=-0.123139 ap=0.492149 base_ap=0.615288 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6065 delta_ap=-0.137651 ap=0.477637 base_ap=0.615288 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6065 delta_ap=-0.072943 ap=0.542345 base_ap=0.615288 donor_failures=18 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6065 delta_ap=-0.129819 ap=0.485469 base_ap=0.615288 donor_failures=131 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6065 delta_ap=-0.138597 ap=0.476691 base_ap=0.615288 donor_failures=152 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6065 delta_ap=-0.056530 ap=0.558758 base_ap=0.615288 donor_failures=45 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6065 delta_ap=-0.089096 ap=0.526192 base_ap=0.615288 donor_failures=53 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=-0.051829 ap=0.563459 base_ap=0.615288 donor_failures=79 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6065 delta_ap=-0.112712 ap=0.502576 base_ap=0.615288 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6065 delta_ap=-0.089228 ap=0.526060 base_ap=0.615288 donor_failures=261 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=3 origin=6066 ap=0.511787 rows=5672 failures=219 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.027471 ap=0.539258 base_ap=0.511787 donor_failures=130 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6066 delta_ap=0.027237 ap=0.539024 base_ap=0.511787 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6066 delta_ap=0.013497 ap=0.525284 base_ap=0.511787 donor_failures=57 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.003367 ap=0.508421 base_ap=0.511787 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6066 delta_ap=0.066453 ap=0.578240 base_ap=0.511787 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6066 delta_ap=0.049033 ap=0.560820 base_ap=0.511787 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6066 delta_ap=0.015903 ap=0.527690 base_ap=0.511787 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6066 delta_ap=0.009745 ap=0.521533 base_ap=0.511787 donor_failures=162 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6066 delta_ap=0.003484 ap=0.515271 base_ap=0.511787 donor_failures=33 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6066 delta_ap=0.019635 ap=0.531423 base_ap=0.511787 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.031792 ap=0.543580 base_ap=0.511787 donor_failures=93 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.003464 ap=0.515251 base_ap=0.511787 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6066 delta_ap=0.052406 ap=0.564193 base_ap=0.511787 donor_failures=266 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=3 origin=6067 ap=0.441588 rows=5530 failures=166 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.018288 ap=0.459876 base_ap=0.441588 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.016636 ap=0.424952 base_ap=0.441588 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6067 delta_ap=0.011792 ap=0.453380 base_ap=0.441588 donor_failures=74 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.007174 ap=0.448762 base_ap=0.441588 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6067 delta_ap=-0.025669 ap=0.415920 base_ap=0.441588 donor_failures=60 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.010975 ap=0.430614 base_ap=0.441588 donor_failures=14 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6067 delta_ap=-0.017439 ap=0.424149 base_ap=0.441588 donor_failures=145 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6067 delta_ap=-0.022183 ap=0.419406 base_ap=0.441588 donor_failures=171 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6067 delta_ap=-0.023175 ap=0.418413 base_ap=0.441588 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6067 delta_ap=-0.025646 ap=0.415942 base_ap=0.441588 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.018164 ap=0.423424 base_ap=0.441588 donor_failures=115 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.010582 ap=0.431006 base_ap=0.441588 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6067 delta_ap=0.009987 ap=0.451575 base_ap=0.441588 donor_failures=241 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=3 origin=6068 ap=0.462410 rows=984 failures=34 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6068 delta_ap=0.027053 ap=0.489463 base_ap=0.462410 donor_failures=167 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6068 delta_ap=-0.009265 ap=0.453145 base_ap=0.462410 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6068 delta_ap=-0.035979 ap=0.426431 base_ap=0.462410 donor_failures=50 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6068 delta_ap=-0.001503 ap=0.460907 base_ap=0.462410 donor_failures=130 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6068 delta_ap=-0.070844 ap=0.391566 base_ap=0.462410 donor_failures=84 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6068 delta_ap=-0.053597 ap=0.408813 base_ap=0.462410 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6068 delta_ap=-0.061968 ap=0.400442 base_ap=0.462410 donor_failures=136 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6068 delta_ap=-0.081221 ap=0.381189 base_ap=0.462410 donor_failures=198 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6068 delta_ap=0.026743 ap=0.489154 base_ap=0.462410 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6068 delta_ap=-0.023299 ap=0.439111 base_ap=0.462410 donor_failures=50 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6068 delta_ap=-0.013935 ap=0.448475 base_ap=0.462410 donor_failures=106 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6068 delta_ap=-0.010181 ap=0.452230 base_ap=0.462410 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6068 delta_ap=-0.012570 ap=0.449840 base_ap=0.462410 donor_failures=253 [POLICY_REPLAY] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 policy_sources= policy_donor_count=0 base_ap=0.441588 policy_ap=0.441588 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6065,6066 [POLICY_REPLAY] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6068 policy_sources= policy_donor_count=0 base_ap=0.462410 policy_ap=0.462410 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6065,6066,6067 [SELECTION_FINAL] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich selected=[] reason=no_stable_positive_directed_edge replay_mean=0.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common rows=6491 failures=209 prevalence=0.032198 ap=0.398430 auc=0.919633 f2_at20=0.423888 brier=0.025276 ece10=0.020799 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=0.876381 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common in_sample_bce=0.052735 local_bce=0.052735 donor_bce=nan in_sample_local_ap=0.771111 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native rows=6491 failures=209 prevalence=0.032198 ap=0.380598 auc=0.915330 f2_at20=0.423888 brier=0.025697 ece10=0.018062 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=1.592864 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native in_sample_bce=0.052042 local_bce=0.052042 donor_bce=nan in_sample_local_ap=0.776566 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all rows=6491 failures=209 prevalence=0.032198 ap=0.392247 auc=0.916924 f2_at20=0.416862 brier=0.026325 ece10=0.026233 rho=0.250000 donor_rows=4095 effective_n=27873.255133 seconds=0.909848 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all in_sample_bce=0.052854 local_bce=0.059225 donor_bce=0.055580 in_sample_local_ap=0.710914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.025454321335493413, 'khoa_quoc_te_hoc': 0.013116327393111221, 'khoa_tieng_anh': 0.05678136225963232, 'khoa_tieng_bo_dao_nha': 0.00493149961259421, 'khoa_tieng_duc': 0.012921744030428964, 'khoa_tieng_han_quoc': 0.020552757624850322, 'khoa_tieng_italia': 0.01306614073395788, 'khoa_tieng_nga': 0.008596006198492639, 'khoa_tieng_nhat': 0.021422659716841586, 'khoa_tieng_phap': 0.017675389166725366, 'khoa_tieng_tay_ban_nha': 0.009392829471015003, 'khoa_tieng_trung_quoc': 0.04028844121997605, 'khoa_viet_nam_hoc': 0.005800521236881032} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 21} positive_ess=978.390160 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal rows=6491 failures=209 prevalence=0.032198 ap=0.365734 auc=0.919110 f2_at20=0.423888 brier=0.027243 ece10=0.029728 rho=0.250000 donor_rows=4095 effective_n=36248.300240 seconds=8.375282 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal in_sample_bce=0.056366 local_bce=0.060923 donor_bce=0.042696 in_sample_local_ap=0.693002 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 21} positive_ess=1064.148595 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior rows=6491 failures=209 prevalence=0.032198 ap=0.398430 auc=0.919633 f2_at20=0.423888 brier=0.025276 ece10=0.020799 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=0.824005 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior in_sample_bce=0.052735 local_bce=0.052735 donor_bce=nan in_sample_local_ap=0.771111 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor rows=6491 failures=209 prevalence=0.032198 ap=0.406146 auc=0.923079 f2_at20=0.421546 brier=0.024851 ece10=0.019149 rho=0.250000 donor_rows=4096 effective_n=36253.196902 seconds=0.863293 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor in_sample_bce=0.044475 local_bce=0.055274 donor_bce=0.012080 in_sample_local_ap=0.746755 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334, np.str_('khoa_tieng_han_quoc'): 26} positive_ess=1209.056397 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common rows=6491 failures=209 prevalence=0.032198 ap=0.398430 auc=0.919633 f2_at20=0.423888 brier=0.025276 ece10=0.020799 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=0.847164 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common in_sample_bce=0.052735 local_bce=0.052735 donor_bce=nan in_sample_local_ap=0.771111 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native rows=6491 failures=209 prevalence=0.032198 ap=0.380598 auc=0.915330 f2_at20=0.423888 brier=0.025697 ece10=0.018062 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=1.498380 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native in_sample_bce=0.052042 local_bce=0.052042 donor_bce=nan in_sample_local_ap=0.776566 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator rows=6491 failures=209 prevalence=0.032198 ap=0.380598 auc=0.915330 f2_at20=0.423888 brier=0.025697 ece10=0.018062 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=2.857974 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator in_sample_bce=0.052042 local_bce=0.052042 donor_bce=nan in_sample_local_ap=0.776566 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator rows=6491 failures=209 prevalence=0.032198 ap=0.380598 auc=0.915330 f2_at20=0.423888 brier=0.025697 ece10=0.018062 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=2.914498 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator in_sample_bce=0.052042 local_bce=0.052042 donor_bce=nan in_sample_local_ap=0.776566 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator rows=6491 failures=209 prevalence=0.032198 ap=0.380598 auc=0.915330 f2_at20=0.423888 brier=0.025697 ece10=0.018062 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=2.897736 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator in_sample_bce=0.052042 local_bce=0.052042 donor_bce=nan in_sample_local_ap=0.776566 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan rows=6491 failures=209 prevalence=0.032198 ap=0.380598 auc=0.915330 f2_at20=0.423888 brier=0.025697 ece10=0.018062 rho=0.000000 donor_rows=0 effective_n=45639.000000 seconds=2.860299 [TRAINING_INSIGHT] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan in_sample_bce=0.052042 local_bce=0.052042 donor_bce=nan in_sample_local_ap=0.776566 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1334} positive_ess=1334.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=committee_reference ap=0.384599 [FACULTY_COMPLETE] fold=3 faculty=khoa_quan_tri_kinh_doanh_va_du_lich models=13 positive_rows=209 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quoc_te_hoc fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_quoc_te_hoc fold=3 origin=6063 ap=0.080562 rows=2231 failures=17 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.076479 ap=0.157041 base_ap=0.080562 donor_failures=151 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.036523 ap=0.117086 base_ap=0.080562 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6063 delta_ap=0.118596 ap=0.199159 base_ap=0.080562 donor_failures=72 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.095566 ap=0.176128 base_ap=0.080562 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6063 delta_ap=0.190511 ap=0.271073 base_ap=0.080562 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.038362 ap=0.118924 base_ap=0.080562 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6063 delta_ap=0.072130 ap=0.152692 base_ap=0.080562 donor_failures=142 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6063 delta_ap=0.089570 ap=0.170132 base_ap=0.080562 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6063 delta_ap=0.102134 ap=0.182697 base_ap=0.080562 donor_failures=60 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6063 delta_ap=0.006034 ap=0.086597 base_ap=0.080562 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.178026 ap=0.258589 base_ap=0.080562 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.142660 ap=0.223222 base_ap=0.080562 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.010956 ap=0.091519 base_ap=0.080562 donor_failures=217 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=3 origin=6064 ap=0.304731 rows=1690 failures=30 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.057059 ap=0.361790 base_ap=0.304731 donor_failures=132 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.052501 ap=0.357232 base_ap=0.304731 donor_failures=99 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6064 delta_ap=0.074061 ap=0.378792 base_ap=0.304731 donor_failures=73 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.034718 ap=0.339449 base_ap=0.304731 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6064 delta_ap=0.030964 ap=0.335695 base_ap=0.304731 donor_failures=55 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=0.007368 ap=0.312098 base_ap=0.304731 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6064 delta_ap=0.029000 ap=0.333731 base_ap=0.304731 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6064 delta_ap=0.054480 ap=0.359211 base_ap=0.304731 donor_failures=151 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6064 delta_ap=0.027618 ap=0.332349 base_ap=0.304731 donor_failures=38 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6064 delta_ap=0.075378 ap=0.380109 base_ap=0.304731 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.006415 ap=0.311146 base_ap=0.304731 donor_failures=72 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.058257 ap=0.362988 base_ap=0.304731 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6064 delta_ap=0.034887 ap=0.339618 base_ap=0.304731 donor_failures=271 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=3 origin=6066 ap=0.038442 rows=2363 failures=31 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.007348 ap=0.045790 base_ap=0.038442 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.038943 ap=0.077385 base_ap=0.038442 donor_failures=96 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6066 delta_ap=0.002430 ap=0.040872 base_ap=0.038442 donor_failures=68 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.005138 ap=0.043580 base_ap=0.038442 donor_failures=121 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.007284 ap=0.045725 base_ap=0.038442 donor_failures=73 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.005120 ap=0.033322 base_ap=0.038442 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6066 delta_ap=0.010259 ap=0.048701 base_ap=0.038442 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.011268 ap=0.049710 base_ap=0.038442 donor_failures=164 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.006777 ap=0.031665 base_ap=0.038442 donor_failures=49 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6066 delta_ap=0.005677 ap=0.044118 base_ap=0.038442 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.007537 ap=0.030905 base_ap=0.038442 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.061075 ap=0.099517 base_ap=0.038442 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.006846 ap=0.045288 base_ap=0.038442 donor_failures=266 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=3 origin=6067 ap=0.365801 rows=2095 failures=19 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.016253 ap=0.382054 base_ap=0.365801 donor_failures=132 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.039441 ap=0.405242 base_ap=0.365801 donor_failures=124 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6067 delta_ap=0.087604 ap=0.453404 base_ap=0.365801 donor_failures=76 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.003848 ap=0.369649 base_ap=0.365801 donor_failures=128 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6067 delta_ap=0.012829 ap=0.378630 base_ap=0.365801 donor_failures=76 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6067 delta_ap=0.092885 ap=0.458685 base_ap=0.365801 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6067 delta_ap=-0.033789 ap=0.332012 base_ap=0.365801 donor_failures=141 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6067 delta_ap=0.054007 ap=0.419808 base_ap=0.365801 donor_failures=204 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.069700 ap=0.296101 base_ap=0.365801 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6067 delta_ap=0.125614 ap=0.491415 base_ap=0.365801 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.102013 ap=0.263788 base_ap=0.365801 donor_failures=111 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.074669 ap=0.440470 base_ap=0.365801 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6067 delta_ap=0.081474 ap=0.447275 base_ap=0.365801 donor_failures=246 [POLICY_REPLAY] fold=3 faculty=khoa_quoc_te_hoc origin=6066 policy_sources=khoa_tieng_duc,khoa_tieng_trung_quoc,khoa_tieng_anh policy_donor_count=3 base_ap=0.038442 policy_ap=0.054529 delta_ap=0.016087 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_quoc_te_hoc origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_duc,khoa_tieng_anh policy_donor_count=3 base_ap=0.365801 policy_ap=0.404308 delta_ap=0.038507 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_quoc_te_hoc selected=['khoa_tieng_trung_quoc', 'khoa_tieng_anh', 'khoa_tieng_duc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.027297 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_local_common rows=2665 failures=26 prevalence=0.009756 ap=0.219179 auc=0.703719 f2_at20=0.117739 brier=0.008775 ece10=0.004957 rho=0.000000 donor_rows=0 effective_n=14897.000000 seconds=0.588839 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=hgb_local_common in_sample_bce=0.007906 local_bce=0.007906 donor_bce=nan in_sample_local_ap=0.975351 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_local_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 148} positive_ess=148.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_local_native rows=2665 failures=26 prevalence=0.009756 ap=0.190011 auc=0.698239 f2_at20=0.117739 brier=0.008831 ece10=0.005766 rho=0.000000 donor_rows=0 effective_n=14897.000000 seconds=0.812341 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=hgb_local_native in_sample_bce=0.008038 local_bce=0.008038 donor_bce=nan in_sample_local_ap=0.973886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_local_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 148} positive_ess=148.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_pool_all rows=2665 failures=26 prevalence=0.009756 ap=0.147011 auc=0.792229 f2_at20=0.117739 brier=0.009075 ece10=0.004012 rho=0.250000 donor_rows=4095 effective_n=16484.530439 seconds=0.631064 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=hgb_pool_all in_sample_bce=0.015963 local_bce=0.012960 donor_bce=0.046748 in_sample_local_ap=0.903139 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_pool_all mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.022967630814600137, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0362580319179362, 'khoa_tieng_anh': 0.051234261889780794, 'khoa_tieng_bo_dao_nha': 0.004449730203824814, 'khoa_tieng_duc': 0.01165938947890251, 'khoa_tieng_han_quoc': 0.018544912006406466, 'khoa_tieng_italia': 0.011789679740182153, 'khoa_tieng_nga': 0.007756242810202045, 'khoa_tieng_nhat': 0.019329831385335037, 'khoa_tieng_phap': 0.015948640214565817, 'khoa_tieng_tay_ban_nha': 0.008475222605678113, 'khoa_tieng_trung_quoc': 0.03635257180264521, 'khoa_viet_nam_hoc': 0.005233855129940702} source_failures={'khoa_quoc_te_hoc': 148, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 27} positive_ess=204.775981 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_pool_equal rows=2665 failures=26 prevalence=0.009756 ap=0.242697 auc=0.797826 f2_at20=0.125589 brier=0.008540 ece10=0.001932 rho=0.250000 donor_rows=4095 effective_n=18860.168496 seconds=0.617021 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=hgb_pool_equal in_sample_bce=0.018919 local_bce=0.013617 donor_bce=0.034825 in_sample_local_ap=0.895048 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_pool_equal mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quoc_te_hoc': 148, 'khoa_cong_nghe_thong_tin': 7, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 27} positive_ess=256.613756 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_language_prior rows=2665 failures=26 prevalence=0.009756 ap=0.219179 auc=0.703719 f2_at20=0.117739 brier=0.008775 ece10=0.004957 rho=0.000000 donor_rows=0 effective_n=14897.000000 seconds=0.575430 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=hgb_language_prior in_sample_bce=0.007906 local_bce=0.007906 donor_bce=nan in_sample_local_ap=0.975351 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_language_prior mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 148} positive_ess=148.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_random_donor rows=2665 failures=26 prevalence=0.009756 ap=0.181465 auc=0.756260 f2_at20=0.133438 brier=0.008896 ece10=0.003961 rho=0.250000 donor_rows=4095 effective_n=18860.168496 seconds=0.640136 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=hgb_random_donor in_sample_bce=0.021213 local_bce=0.014074 donor_bce=0.042630 in_sample_local_ap=0.891832 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=hgb_random_donor mass={'khoa_quoc_te_hoc': 0.7500000000000002, np.str_('khoa_tieng_nga'): 0.08333333333333333, np.str_('khoa_tieng_nhat'): 0.08333333333333333, np.str_('khoa_tieng_bo_dao_nha'): 0.08333333333333333} source_failures={'khoa_quoc_te_hoc': 148, np.str_('khoa_tieng_nga'): 72, np.str_('khoa_tieng_nhat'): 16, np.str_('khoa_tieng_bo_dao_nha'): 45} positive_ess=278.410022 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_common rows=2665 failures=26 prevalence=0.009756 ap=0.101007 auc=0.738989 f2_at20=0.125589 brier=0.009559 ece10=0.002734 rho=0.250000 donor_rows=4095 effective_n=18760.783347 seconds=0.655343 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_common in_sample_bce=0.012095 local_bce=0.011481 donor_bce=0.015192 in_sample_local_ap=0.928521 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_common mass={'khoa_quoc_te_hoc': 0.75, 'khoa_tieng_trung_quoc': 0.09775964470548555, 'khoa_tieng_anh': 0.08208788938218534, 'khoa_tieng_duc': 0.07015246591232913} source_failures={'khoa_quoc_te_hoc': 148, 'khoa_tieng_trung_quoc': 4, 'khoa_tieng_anh': 21, 'khoa_tieng_duc': 26} positive_ess=197.768227 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_native rows=2665 failures=26 prevalence=0.009756 ap=0.103461 auc=0.756158 f2_at20=0.125589 brier=0.009527 ece10=0.002789 rho=0.250000 donor_rows=4095 effective_n=18760.783347 seconds=0.891742 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_native in_sample_bce=0.012012 local_bce=0.011137 donor_bce=0.016010 in_sample_local_ap=0.936092 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_native mass={'khoa_quoc_te_hoc': 0.75, 'khoa_tieng_trung_quoc': 0.09775964470548555, 'khoa_tieng_anh': 0.08208788938218534, 'khoa_tieng_duc': 0.07015246591232913} source_failures={'khoa_quoc_te_hoc': 148, 'khoa_tieng_trung_quoc': 4, 'khoa_tieng_anh': 21, 'khoa_tieng_duc': 26} positive_ess=197.768227 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator rows=2665 failures=26 prevalence=0.009756 ap=0.190011 auc=0.698239 f2_at20=0.117739 brier=0.008831 ece10=0.005766 rho=0.000000 donor_rows=0 effective_n=14897.000000 seconds=1.268869 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator in_sample_bce=0.008038 local_bce=0.008038 donor_bce=nan in_sample_local_ap=0.973886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 148} positive_ess=148.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator rows=2665 failures=26 prevalence=0.009756 ap=0.190011 auc=0.698239 f2_at20=0.117739 brier=0.008831 ece10=0.005766 rho=0.000000 donor_rows=0 effective_n=14897.000000 seconds=1.400945 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator in_sample_bce=0.008038 local_bce=0.008038 donor_bce=nan in_sample_local_ap=0.973886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 148} positive_ess=148.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=native_placebo_indicator rows=2665 failures=26 prevalence=0.009756 ap=0.190011 auc=0.698239 f2_at20=0.117739 brier=0.008831 ece10=0.005766 rho=0.000000 donor_rows=0 effective_n=14897.000000 seconds=1.376535 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=native_placebo_indicator in_sample_bce=0.008038 local_bce=0.008038 donor_bce=nan in_sample_local_ap=0.973886 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=native_placebo_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 148} positive_ess=148.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan rows=2665 failures=26 prevalence=0.009756 ap=0.103461 auc=0.756158 f2_at20=0.125589 brier=0.009527 ece10=0.002789 rho=0.250000 donor_rows=4095 effective_n=18760.783347 seconds=1.432326 [TRAINING_INSIGHT] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.012012 local_bce=0.011137 donor_bce=0.016010 in_sample_local_ap=0.936092 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan mass={'khoa_quoc_te_hoc': 0.75, 'khoa_tieng_trung_quoc': 0.09775964470548555, 'khoa_tieng_anh': 0.08208788938218534, 'khoa_tieng_duc': 0.07015246591232913} source_failures={'khoa_quoc_te_hoc': 148, 'khoa_tieng_trung_quoc': 4, 'khoa_tieng_anh': 21, 'khoa_tieng_duc': 26} positive_ess=197.768227 [FACULTY_RESULTS] fold=3 faculty=khoa_quoc_te_hoc model=committee_reference ap=0.199138 [FACULTY_COMPLETE] fold=3 faculty=khoa_quoc_te_hoc models=13 positive_rows=26 num_edges=52 num_replays=2 chosen=['khoa_tieng_trung_quoc', 'khoa_tieng_anh', 'khoa_tieng_duc'] [SELECTION_ORIGINS] target=khoa_tieng_anh fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_anh fold=3 origin=6063 ap=0.267801 rows=9030 failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.065483 ap=0.333285 base_ap=0.267801 donor_failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.019804 ap=0.287605 base_ap=0.267801 donor_failures=76 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6063 delta_ap=0.153828 ap=0.421629 base_ap=0.267801 donor_failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.066332 ap=0.201469 base_ap=0.267801 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_duc origin=6063 delta_ap=-0.040329 ap=0.227473 base_ap=0.267801 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.009833 ap=0.257968 base_ap=0.267801 donor_failures=13 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_italia origin=6063 delta_ap=0.065313 ap=0.333114 base_ap=0.267801 donor_failures=143 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nga origin=6063 delta_ap=0.009086 ap=0.276887 base_ap=0.267801 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6063 delta_ap=0.092293 ap=0.360094 base_ap=0.267801 donor_failures=64 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_phap origin=6063 delta_ap=-0.017301 ap=0.250500 base_ap=0.267801 donor_failures=57 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.102574 ap=0.165227 base_ap=0.267801 donor_failures=92 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.048350 ap=0.219452 base_ap=0.267801 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6063 delta_ap=0.055271 ap=0.323072 base_ap=0.267801 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_anh fold=3 origin=6064 ap=0.587346 rows=9726 failures=240 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.025465 ap=0.561881 base_ap=0.587346 donor_failures=133 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.068775 ap=0.518570 base_ap=0.587346 donor_failures=107 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.024147 ap=0.563199 base_ap=0.587346 donor_failures=30 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.027825 ap=0.559521 base_ap=0.587346 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_duc origin=6064 delta_ap=-0.007754 ap=0.579592 base_ap=0.587346 donor_failures=58 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.024369 ap=0.562977 base_ap=0.587346 donor_failures=20 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_italia origin=6064 delta_ap=-0.033011 ap=0.554335 base_ap=0.587346 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nga origin=6064 delta_ap=-0.019037 ap=0.568309 base_ap=0.587346 donor_failures=144 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6064 delta_ap=-0.023917 ap=0.563429 base_ap=0.587346 donor_failures=49 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_phap origin=6064 delta_ap=-0.015208 ap=0.572138 base_ap=0.587346 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.014185 ap=0.573161 base_ap=0.587346 donor_failures=70 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.023957 ap=0.563389 base_ap=0.587346 donor_failures=33 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.045576 ap=0.541770 base_ap=0.587346 donor_failures=267 [SCREEN_BASE] target=khoa_tieng_anh fold=3 origin=6066 ap=0.551801 rows=8164 failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.034174 ap=0.585975 base_ap=0.551801 donor_failures=136 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.035602 ap=0.587403 base_ap=0.551801 donor_failures=117 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6066 delta_ap=0.007592 ap=0.559393 base_ap=0.551801 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.026306 ap=0.525496 base_ap=0.551801 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_duc origin=6066 delta_ap=0.026173 ap=0.577974 base_ap=0.551801 donor_failures=64 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.000509 ap=0.551293 base_ap=0.551801 donor_failures=14 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_italia origin=6066 delta_ap=0.043929 ap=0.595730 base_ap=0.551801 donor_failures=133 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nga origin=6066 delta_ap=0.043999 ap=0.595800 base_ap=0.551801 donor_failures=157 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6066 delta_ap=-0.076168 ap=0.475633 base_ap=0.551801 donor_failures=52 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_phap origin=6066 delta_ap=-0.014399 ap=0.537402 base_ap=0.551801 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.026702 ap=0.578503 base_ap=0.551801 donor_failures=84 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.036479 ap=0.588280 base_ap=0.551801 donor_failures=18 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6066 delta_ap=0.025067 ap=0.576868 base_ap=0.551801 donor_failures=264 [SCREEN_BASE] target=khoa_tieng_anh fold=3 origin=6067 ap=0.701950 rows=8328 failures=122 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.087859 ap=0.614092 base_ap=0.701950 donor_failures=135 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.019242 ap=0.682709 base_ap=0.701950 donor_failures=124 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.016829 ap=0.685121 base_ap=0.701950 donor_failures=38 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.021704 ap=0.680246 base_ap=0.701950 donor_failures=134 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_duc origin=6067 delta_ap=-0.053378 ap=0.648572 base_ap=0.701950 donor_failures=66 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.049563 ap=0.652387 base_ap=0.701950 donor_failures=16 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_italia origin=6067 delta_ap=-0.026329 ap=0.675621 base_ap=0.701950 donor_failures=147 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nga origin=6067 delta_ap=-0.034458 ap=0.667492 base_ap=0.701950 donor_failures=203 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6067 delta_ap=-0.029039 ap=0.672912 base_ap=0.701950 donor_failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_phap origin=6067 delta_ap=-0.067074 ap=0.634876 base_ap=0.701950 donor_failures=52 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.051573 ap=0.650378 base_ap=0.701950 donor_failures=98 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.024110 ap=0.677840 base_ap=0.701950 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.104672 ap=0.597278 base_ap=0.701950 donor_failures=245 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_anh origin=6066 policy_sources=khoa_quoc_te_hoc,khoa_tieng_nhat,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.551801 policy_ap=0.595190 delta_ap=0.043389 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_anh origin=6067 policy_sources=khoa_quoc_te_hoc,khoa_tieng_italia,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.701950 policy_ap=0.627922 delta_ap=-0.074028 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_anh selected=[] reason=prequential_policy_delta_-0.015320_below_threshold replay_mean=-0.015320 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=hgb_local_common rows=7279 failures=101 prevalence=0.013876 ap=0.379946 auc=0.952461 f2_at20=0.260753 brier=0.011400 ece10=0.007038 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=1.030236 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=hgb_local_common in_sample_bce=0.021616 local_bce=0.021616 donor_bce=nan in_sample_local_ap=0.901965 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=hgb_local_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=hgb_local_native rows=7279 failures=101 prevalence=0.013876 ap=0.358313 auc=0.953996 f2_at20=0.263441 brier=0.011375 ece10=0.006054 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=3.087654 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=hgb_local_native in_sample_bce=0.021634 local_bce=0.021634 donor_bce=nan in_sample_local_ap=0.903144 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=hgb_local_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=hgb_pool_all rows=7279 failures=101 prevalence=0.013876 ap=0.366074 auc=0.942375 f2_at20=0.252688 brier=0.011108 ece10=0.004944 rho=0.250000 donor_rows=4095 effective_n=33466.063843 seconds=1.056204 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=hgb_pool_all in_sample_bce=0.026004 local_bce=0.027080 donor_bce=0.034381 in_sample_local_ap=0.848876 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=hgb_pool_all mass={'khoa_tieng_anh': 0.7500000000000001, 'khoa_cong_nghe_thong_tin': 0.027264428173179575, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04304120502925433, 'khoa_quoc_te_hoc': 0.014049055222962854, 'khoa_tieng_bo_dao_nha': 0.005282188246211649, 'khoa_tieng_duc': 0.013840634654776321, 'khoa_tieng_han_quoc': 0.022014304629765852, 'khoa_tieng_italia': 0.013995299691801622, 'khoa_tieng_nga': 0.00920728510047569, 'khoa_tieng_nhat': 0.022946067169893884, 'khoa_tieng_phap': 0.018932320843188515, 'khoa_tieng_tay_ban_nha': 0.010060772042597015, 'khoa_tieng_trung_quoc': 0.04315343148904707, 'khoa_viet_nam_hoc': 0.006213007706845626} source_failures={'khoa_tieng_anh': 1091, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 5, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_bo_dao_nha': 2, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 14} positive_ess=632.860820 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=hgb_pool_equal rows=7279 failures=101 prevalence=0.013876 ap=0.400272 auc=0.953238 f2_at20=0.250000 brier=0.010712 ece10=0.004680 rho=0.250000 donor_rows=4095 effective_n=41693.076126 seconds=1.049189 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=hgb_pool_equal in_sample_bce=0.028035 local_bce=0.028424 donor_bce=0.026869 in_sample_local_ap=0.827732 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=hgb_pool_equal mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_anh': 1091, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 5, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_bo_dao_nha': 2, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 14} positive_ess=691.784317 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=hgb_language_prior rows=7279 failures=101 prevalence=0.013876 ap=0.382924 auc=0.958295 f2_at20=0.260753 brier=0.011135 ece10=0.006108 rho=0.250000 donor_rows=4095 effective_n=41693.076126 seconds=1.077711 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=hgb_language_prior in_sample_bce=0.029136 local_bce=0.029005 donor_bce=0.029529 in_sample_local_ap=0.818432 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=hgb_language_prior mass={'khoa_tieng_anh': 0.7500000000000001, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_anh': 1091, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 19, 'khoa_tieng_nga': 26, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 3} positive_ess=681.943606 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=hgb_random_donor rows=7279 failures=101 prevalence=0.013876 ap=0.346627 auc=0.961584 f2_at20=0.263441 brier=0.011663 ece10=0.006197 rho=0.250000 donor_rows=4096 effective_n=41699.554433 seconds=1.066988 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=hgb_random_donor in_sample_bce=0.021601 local_bce=0.026157 donor_bce=0.007932 in_sample_local_ap=0.849123 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=hgb_random_donor mass={'khoa_tieng_anh': 0.75, np.str_('khoa_tieng_trung_quoc'): 0.25} source_failures={'khoa_tieng_anh': 1091, np.str_('khoa_tieng_trung_quoc'): 35} positive_ess=790.659057 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=suite32_selective_common rows=7279 failures=101 prevalence=0.013876 ap=0.379946 auc=0.952461 f2_at20=0.260753 brier=0.011400 ece10=0.007038 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=1.009403 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=suite32_selective_common in_sample_bce=0.021616 local_bce=0.021616 donor_bce=nan in_sample_local_ap=0.901965 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=suite32_selective_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=suite32_selective_native rows=7279 failures=101 prevalence=0.013876 ap=0.358313 auc=0.953996 f2_at20=0.263441 brier=0.011375 ece10=0.006054 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=3.063755 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=suite32_selective_native in_sample_bce=0.021634 local_bce=0.021634 donor_bce=nan in_sample_local_ap=0.903144 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=suite32_selective_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=native_cache_nan_indicator rows=7279 failures=101 prevalence=0.013876 ap=0.358313 auc=0.953996 f2_at20=0.263441 brier=0.011375 ece10=0.006054 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=6.689054 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=native_cache_nan_indicator in_sample_bce=0.021634 local_bce=0.021634 donor_bce=nan in_sample_local_ap=0.903144 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=native_cache_nan_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=native_naive_zero_indicator rows=7279 failures=101 prevalence=0.013876 ap=0.358313 auc=0.953996 f2_at20=0.263441 brier=0.011375 ece10=0.006054 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=6.323717 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=native_naive_zero_indicator in_sample_bce=0.021634 local_bce=0.021634 donor_bce=nan in_sample_local_ap=0.903144 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=native_naive_zero_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=native_placebo_indicator rows=7279 failures=101 prevalence=0.013876 ap=0.358313 auc=0.953996 f2_at20=0.263441 brier=0.011375 ece10=0.006054 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=6.105708 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=native_placebo_indicator in_sample_bce=0.021634 local_bce=0.021634 donor_bce=nan in_sample_local_ap=0.903144 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=native_placebo_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan rows=7279 failures=101 prevalence=0.013876 ap=0.358313 auc=0.953996 f2_at20=0.263441 brier=0.011375 ece10=0.006054 rho=0.000000 donor_rows=0 effective_n=64490.000000 seconds=6.126916 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan in_sample_bce=0.021634 local_bce=0.021634 donor_bce=nan in_sample_local_ap=0.903144 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1091} positive_ess=1091.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_anh model=committee_reference ap=0.346834 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_anh models=13 positive_rows=101 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_bo_dao_nha fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=3 origin=6063 ap=0.089418 rows=856 failures=15 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.107344 ap=0.196762 base_ap=0.089418 donor_failures=151 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.020147 ap=0.109564 base_ap=0.089418 donor_failures=104 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6063 delta_ap=0.005047 ap=0.094465 base_ap=0.089418 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6063 delta_ap=0.037792 ap=0.127210 base_ap=0.089418 donor_failures=76 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6063 delta_ap=-0.006766 ap=0.082652 base_ap=0.089418 donor_failures=55 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.001305 ap=0.088112 base_ap=0.089418 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6063 delta_ap=-0.000978 ap=0.088440 base_ap=0.089418 donor_failures=145 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6063 delta_ap=0.085389 ap=0.174807 base_ap=0.089418 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6063 delta_ap=0.030483 ap=0.119901 base_ap=0.089418 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6063 delta_ap=0.003829 ap=0.093247 base_ap=0.089418 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.056117 ap=0.145535 base_ap=0.089418 donor_failures=85 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.037785 ap=0.127202 base_ap=0.089418 donor_failures=29 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6063 delta_ap=0.001778 ap=0.091196 base_ap=0.089418 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=3 origin=6064 ap=0.166066 rows=780 failures=26 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.000198 ap=0.166264 base_ap=0.166066 donor_failures=119 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.048872 ap=0.214938 base_ap=0.166066 donor_failures=96 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=0.009889 ap=0.175955 base_ap=0.166066 donor_failures=38 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6064 delta_ap=-0.000687 ap=0.165379 base_ap=0.166066 donor_failures=59 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6064 delta_ap=0.085866 ap=0.251931 base_ap=0.166066 donor_failures=60 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=0.006235 ap=0.172301 base_ap=0.166066 donor_failures=15 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6064 delta_ap=0.016252 ap=0.182318 base_ap=0.166066 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6064 delta_ap=0.010971 ap=0.177036 base_ap=0.166066 donor_failures=155 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6064 delta_ap=0.004263 ap=0.170329 base_ap=0.166066 donor_failures=55 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6064 delta_ap=0.008738 ap=0.174804 base_ap=0.166066 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.007671 ap=0.158395 base_ap=0.166066 donor_failures=77 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.031272 ap=0.197338 base_ap=0.166066 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.015997 ap=0.182063 base_ap=0.166066 donor_failures=272 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=3 origin=6066 ap=0.108033 rows=768 failures=30 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.031145 ap=0.139178 base_ap=0.108033 donor_failures=119 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.094510 ap=0.202543 base_ap=0.108033 donor_failures=114 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.010879 ap=0.118912 base_ap=0.108033 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.011782 ap=0.096250 base_ap=0.108033 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.006882 ap=0.101150 base_ap=0.108033 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=0.011619 ap=0.119652 base_ap=0.108033 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6066 delta_ap=0.043594 ap=0.151627 base_ap=0.108033 donor_failures=126 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6066 delta_ap=0.088038 ap=0.196071 base_ap=0.108033 donor_failures=178 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6066 delta_ap=-0.002200 ap=0.105833 base_ap=0.108033 donor_failures=54 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6066 delta_ap=0.094660 ap=0.202693 base_ap=0.108033 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.072328 ap=0.180361 base_ap=0.108033 donor_failures=103 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.005405 ap=0.102628 base_ap=0.108033 donor_failures=26 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=0.021231 ap=0.129264 base_ap=0.108033 donor_failures=259 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=3 origin=6067 ap=0.411685 rows=716 failures=30 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.034171 ap=0.445856 base_ap=0.411685 donor_failures=129 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.004556 ap=0.416242 base_ap=0.411685 donor_failures=114 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6067 delta_ap=0.085936 ap=0.497621 base_ap=0.411685 donor_failures=40 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6067 delta_ap=-0.006983 ap=0.404702 base_ap=0.411685 donor_failures=78 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6067 delta_ap=0.057841 ap=0.469526 base_ap=0.411685 donor_failures=69 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6067 delta_ap=0.011077 ap=0.422762 base_ap=0.411685 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6067 delta_ap=-0.046687 ap=0.364998 base_ap=0.411685 donor_failures=141 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6067 delta_ap=0.005822 ap=0.417507 base_ap=0.411685 donor_failures=212 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6067 delta_ap=-0.032624 ap=0.379061 base_ap=0.411685 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6067 delta_ap=0.027216 ap=0.438901 base_ap=0.411685 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.064566 ap=0.476251 base_ap=0.411685 donor_failures=106 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.039915 ap=0.451600 base_ap=0.411685 donor_failures=18 [TRANSFER_EDGE] fold=3 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6067 delta_ap=0.028705 ap=0.440390 base_ap=0.411685 donor_failures=252 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_bo_dao_nha origin=6066 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nga,khoa_tieng_duc policy_donor_count=3 base_ap=0.108033 policy_ap=0.153949 delta_ap=0.045917 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_bo_dao_nha origin=6067 policy_sources=khoa_tieng_nga,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.411685 policy_ap=0.406823 delta_ap=-0.004862 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_bo_dao_nha selected=['khoa_tieng_nga', 'khoa_tieng_tay_ban_nha', 'khoa_cong_nghe_thong_tin'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.020527 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common rows=1032 failures=50 prevalence=0.048450 ap=0.493155 auc=0.921202 f2_at20=0.540541 brier=0.036693 ece10=0.025823 rho=0.000000 donor_rows=0 effective_n=5601.000000 seconds=0.444670 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common in_sample_bce=0.029116 local_bce=0.029116 donor_bce=nan in_sample_local_ap=0.964859 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 183} positive_ess=183.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native rows=1032 failures=50 prevalence=0.048450 ap=0.435757 auc=0.919430 f2_at20=0.552826 brier=0.037714 ece10=0.025506 rho=0.000000 donor_rows=0 effective_n=5601.000000 seconds=0.715940 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native in_sample_bce=0.029096 local_bce=0.029096 donor_bce=nan in_sample_local_ap=0.966773 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 183} positive_ess=183.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all rows=1032 failures=50 prevalence=0.048450 ap=0.374632 auc=0.923768 f2_at20=0.565111 brier=0.037908 ece10=0.018781 rho=0.250000 donor_rows=4095 effective_n=8170.145230 seconds=0.557266 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all in_sample_bce=0.036119 local_bce=0.034766 donor_bce=0.057629 in_sample_local_ap=0.946827 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all mass={'khoa_tieng_bo_dao_nha': 0.7500000000000001, 'khoa_cong_nghe_thong_tin': 0.022308613547833495, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.035217669100988336, 'khoa_quoc_te_hoc': 0.011495379315879474, 'khoa_tieng_anh': 0.04976418151849817, 'khoa_tieng_duc': 0.01132484304489811, 'khoa_tieng_han_quoc': 0.018012797165239614, 'khoa_tieng_italia': 0.011451394847798307, 'khoa_tieng_nga': 0.007533690559235503, 'khoa_tieng_nhat': 0.018775194611979826, 'khoa_tieng_phap': 0.015491020995252763, 'khoa_tieng_tay_ban_nha': 0.008232040447190858, 'khoa_tieng_trung_quoc': 0.035309496323824456, 'khoa_viet_nam_hoc': 0.005083678521381081} source_failures={'khoa_tieng_bo_dao_nha': 183, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 6, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 17, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 20} positive_ess=237.361272 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal rows=1032 failures=50 prevalence=0.048450 ap=0.368863 auc=0.918391 f2_at20=0.552826 brier=0.037013 ece10=0.021432 rho=0.250000 donor_rows=4095 effective_n=8643.713963 seconds=0.558152 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal in_sample_bce=0.040035 local_bce=0.036423 donor_bce=0.050871 in_sample_local_ap=0.941121 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_bo_dao_nha': 183, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 6, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 17, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 20} positive_ess=258.703398 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior rows=1032 failures=50 prevalence=0.048450 ap=0.524500 auc=0.923544 f2_at20=0.528256 brier=0.034218 ece10=0.016671 rho=0.250000 donor_rows=4095 effective_n=8643.713963 seconds=0.519046 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior in_sample_bce=0.037382 local_bce=0.035071 donor_bce=0.044315 in_sample_local_ap=0.942602 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_bo_dao_nha': 183, 'khoa_tieng_anh': 8, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 22, 'khoa_tieng_nga': 27, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 3} positive_ess=248.829067 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor rows=1032 failures=50 prevalence=0.048450 ap=0.316484 auc=0.908248 f2_at20=0.540541 brier=0.039566 ece10=0.021345 rho=0.250000 donor_rows=4095 effective_n=8643.713963 seconds=0.519070 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor in_sample_bce=0.033042 local_bce=0.033652 donor_bce=0.031211 in_sample_local_ap=0.949248 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor mass={'khoa_tieng_bo_dao_nha': 0.75, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.08333333333333333, np.str_('khoa_tieng_han_quoc'): 0.08333333333333333, np.str_('khoa_tieng_duc'): 0.08333333333333333} source_failures={'khoa_tieng_bo_dao_nha': 183, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 33, np.str_('khoa_tieng_han_quoc'): 7, np.str_('khoa_tieng_duc'): 21} positive_ess=227.112802 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common rows=1032 failures=50 prevalence=0.048450 ap=0.364188 auc=0.913992 f2_at20=0.540541 brier=0.038687 ece10=0.023028 rho=0.250000 donor_rows=4095 effective_n=8641.890518 seconds=0.551712 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common in_sample_bce=0.043059 local_bce=0.038103 donor_bce=0.057790 in_sample_local_ap=0.926473 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_nga': 0.08671287554587351, 'khoa_tieng_tay_ban_nha': 0.08448863558916427, 'khoa_cong_nghe_thong_tin': 0.07879848886496225} source_failures={'khoa_tieng_bo_dao_nha': 183, 'khoa_tieng_nga': 60, 'khoa_tieng_tay_ban_nha': 28, 'khoa_cong_nghe_thong_tin': 39} positive_ess=277.433226 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native rows=1032 failures=50 prevalence=0.048450 ap=0.348188 auc=0.908798 f2_at20=0.528256 brier=0.039138 ece10=0.024316 rho=0.250000 donor_rows=4095 effective_n=8641.890518 seconds=0.955288 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native in_sample_bce=0.043256 local_bce=0.037852 donor_bce=0.059325 in_sample_local_ap=0.923092 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_nga': 0.08671287554587351, 'khoa_tieng_tay_ban_nha': 0.08448863558916427, 'khoa_cong_nghe_thong_tin': 0.07879848886496225} source_failures={'khoa_tieng_bo_dao_nha': 183, 'khoa_tieng_nga': 60, 'khoa_tieng_tay_ban_nha': 28, 'khoa_cong_nghe_thong_tin': 39} positive_ess=277.433226 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator rows=1032 failures=50 prevalence=0.048450 ap=0.435757 auc=0.919430 f2_at20=0.552826 brier=0.037714 ece10=0.025506 rho=0.000000 donor_rows=0 effective_n=5601.000000 seconds=1.522604 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator in_sample_bce=0.029096 local_bce=0.029096 donor_bce=nan in_sample_local_ap=0.966773 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 183} positive_ess=183.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator rows=1032 failures=50 prevalence=0.048450 ap=0.453266 auc=0.918635 f2_at20=0.528256 brier=0.037799 ece10=0.024711 rho=0.000000 donor_rows=0 effective_n=5601.000000 seconds=1.263675 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator in_sample_bce=0.029207 local_bce=0.029207 donor_bce=nan in_sample_local_ap=0.964914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 183} positive_ess=183.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator rows=1032 failures=50 prevalence=0.048450 ap=0.435757 auc=0.919430 f2_at20=0.552826 brier=0.037714 ece10=0.025506 rho=0.000000 donor_rows=0 effective_n=5601.000000 seconds=1.328108 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator in_sample_bce=0.029096 local_bce=0.029096 donor_bce=nan in_sample_local_ap=0.966773 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 183} positive_ess=183.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan rows=1032 failures=50 prevalence=0.048450 ap=0.348188 auc=0.908798 f2_at20=0.528256 brier=0.039138 ece10=0.024316 rho=0.250000 donor_rows=4095 effective_n=8641.890518 seconds=1.719658 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan in_sample_bce=0.043256 local_bce=0.037852 donor_bce=0.059325 in_sample_local_ap=0.923092 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_nga': 0.08671287554587351, 'khoa_tieng_tay_ban_nha': 0.08448863558916427, 'khoa_cong_nghe_thong_tin': 0.07879848886496225} source_failures={'khoa_tieng_bo_dao_nha': 183, 'khoa_tieng_nga': 60, 'khoa_tieng_tay_ban_nha': 28, 'khoa_cong_nghe_thong_tin': 39} positive_ess=277.433226 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_bo_dao_nha model=committee_reference ap=0.348897 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_bo_dao_nha models=13 positive_rows=50 num_edges=52 num_replays=2 chosen=['khoa_tieng_nga', 'khoa_tieng_tay_ban_nha', 'khoa_cong_nghe_thong_tin'] [SELECTION_ORIGINS] target=khoa_tieng_duc fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_duc fold=3 origin=6063 ap=0.154736 rows=2282 failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.241748 ap=0.396484 base_ap=0.154736 donor_failures=156 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.034936 ap=0.119800 base_ap=0.154736 donor_failures=80 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.283303 ap=0.438039 base_ap=0.154736 donor_failures=40 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_anh origin=6063 delta_ap=0.094632 ap=0.249368 base_ap=0.154736 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.077089 ap=0.231825 base_ap=0.154736 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.240058 ap=0.394794 base_ap=0.154736 donor_failures=26 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_italia origin=6063 delta_ap=0.053140 ap=0.207876 base_ap=0.154736 donor_failures=144 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nga origin=6063 delta_ap=0.176134 ap=0.330870 base_ap=0.154736 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6063 delta_ap=0.047244 ap=0.201980 base_ap=0.154736 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_phap origin=6063 delta_ap=0.168537 ap=0.323273 base_ap=0.154736 donor_failures=58 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.255093 ap=0.409829 base_ap=0.154736 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.214483 ap=0.369219 base_ap=0.154736 donor_failures=27 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.195454 ap=0.350190 base_ap=0.154736 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_duc fold=3 origin=6064 ap=0.370675 rows=1984 failures=53 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.033083 ap=0.337592 base_ap=0.370675 donor_failures=146 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.017863 ap=0.388538 base_ap=0.370675 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.018086 ap=0.352590 base_ap=0.370675 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_anh origin=6064 delta_ap=0.027043 ap=0.397718 base_ap=0.370675 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.031107 ap=0.401782 base_ap=0.370675 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.000141 ap=0.370535 base_ap=0.370675 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_italia origin=6064 delta_ap=0.012019 ap=0.382694 base_ap=0.370675 donor_failures=126 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nga origin=6064 delta_ap=0.015662 ap=0.386337 base_ap=0.370675 donor_failures=149 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6064 delta_ap=0.029705 ap=0.400380 base_ap=0.370675 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_phap origin=6064 delta_ap=0.001699 ap=0.372374 base_ap=0.370675 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.055595 ap=0.315080 base_ap=0.370675 donor_failures=81 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.017100 ap=0.353575 base_ap=0.370675 donor_failures=24 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.015285 ap=0.355391 base_ap=0.370675 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_duc fold=3 origin=6066 ap=0.139461 rows=1900 failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.091176 ap=0.230637 base_ap=0.139461 donor_failures=127 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.043303 ap=0.182764 base_ap=0.139461 donor_failures=105 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6066 delta_ap=0.039773 ap=0.179234 base_ap=0.139461 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_anh origin=6066 delta_ap=0.040320 ap=0.179781 base_ap=0.139461 donor_failures=75 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.044141 ap=0.183602 base_ap=0.139461 donor_failures=121 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6066 delta_ap=0.019134 ap=0.158595 base_ap=0.139461 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_italia origin=6066 delta_ap=0.032337 ap=0.171797 base_ap=0.139461 donor_failures=128 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nga origin=6066 delta_ap=0.046110 ap=0.185571 base_ap=0.139461 donor_failures=164 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6066 delta_ap=0.060129 ap=0.199590 base_ap=0.139461 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_phap origin=6066 delta_ap=0.045610 ap=0.185070 base_ap=0.139461 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.061351 ap=0.200812 base_ap=0.139461 donor_failures=88 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.001849 ap=0.141310 base_ap=0.139461 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.002621 ap=0.136840 base_ap=0.139461 donor_failures=263 [SCREEN_BASE] target=khoa_tieng_duc fold=3 origin=6067 ap=0.319202 rows=1893 failures=53 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.085807 ap=0.405009 base_ap=0.319202 donor_failures=124 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.049439 ap=0.368641 base_ap=0.319202 donor_failures=105 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6067 delta_ap=0.018309 ap=0.337511 base_ap=0.319202 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_anh origin=6067 delta_ap=0.038912 ap=0.358114 base_ap=0.319202 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.023611 ap=0.295591 base_ap=0.319202 donor_failures=127 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6067 delta_ap=0.056098 ap=0.375300 base_ap=0.319202 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_italia origin=6067 delta_ap=0.061069 ap=0.380271 base_ap=0.319202 donor_failures=138 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nga origin=6067 delta_ap=0.106831 ap=0.426033 base_ap=0.319202 donor_failures=198 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6067 delta_ap=-0.027542 ap=0.291660 base_ap=0.319202 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_phap origin=6067 delta_ap=-0.046186 ap=0.273016 base_ap=0.319202 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.029110 ap=0.348312 base_ap=0.319202 donor_failures=103 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.041247 ap=0.360449 base_ap=0.319202 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6067 delta_ap=0.019192 ap=0.338394 base_ap=0.319202 donor_failures=255 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_duc origin=6066 policy_sources=khoa_quoc_te_hoc,khoa_tieng_han_quoc,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.139461 policy_ap=0.192127 delta_ap=0.052666 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_duc origin=6067 policy_sources=khoa_quoc_te_hoc,khoa_cong_nghe_thong_tin,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.319202 policy_ap=0.339175 delta_ap=0.019973 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_duc selected=['khoa_cong_nghe_thong_tin', 'khoa_tieng_nga', 'khoa_quoc_te_hoc'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.036320 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=hgb_local_common rows=1909 failures=47 prevalence=0.024620 ap=0.277972 auc=0.886692 f2_at20=0.307018 brier=0.022998 ece10=0.013665 rho=0.000000 donor_rows=0 effective_n=14676.000000 seconds=0.563554 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=hgb_local_common in_sample_bce=0.020363 local_bce=0.020363 donor_bce=nan in_sample_local_ap=0.931375 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=hgb_local_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 271} positive_ess=271.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=hgb_local_native rows=1909 failures=47 prevalence=0.024620 ap=0.261247 auc=0.878288 f2_at20=0.298246 brier=0.025224 ece10=0.017865 rho=0.000000 donor_rows=0 effective_n=14676.000000 seconds=1.038582 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=hgb_local_native in_sample_bce=0.020089 local_bce=0.020089 donor_bce=nan in_sample_local_ap=0.934501 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=hgb_local_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 271} positive_ess=271.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=hgb_pool_all rows=1909 failures=47 prevalence=0.024620 ap=0.402220 auc=0.919601 f2_at20=0.368421 brier=0.020855 ece10=0.022767 rho=0.250000 donor_rows=4095 effective_n=16336.538610 seconds=0.593075 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=hgb_pool_all in_sample_bce=0.028064 local_bce=0.025983 donor_bce=0.057433 in_sample_local_ap=0.878230 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=hgb_pool_all mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.022951512052917882, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.036232585907406405, 'khoa_quoc_te_hoc': 0.01182665773269864, 'khoa_tieng_anh': 0.051198305509950685, 'khoa_tieng_bo_dao_nha': 0.004446607367983157, 'khoa_tieng_han_quoc': 0.01853189712387624, 'khoa_tieng_italia': 0.011781405702708454, 'khoa_tieng_nga': 0.007750799452529827, 'khoa_tieng_nhat': 0.019316265643706157, 'khoa_tieng_phap': 0.015937447404438827, 'khoa_tieng_tay_ban_nha': 0.008469274665531925, 'khoa_tieng_trung_quoc': 0.03632705944370171, 'khoa_viet_nam_hoc': 0.005230181992550087} source_failures={'khoa_tieng_duc': 271, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 18, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 11, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 19} positive_ess=309.874588 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=hgb_pool_equal rows=1909 failures=47 prevalence=0.024620 ap=0.431983 auc=0.930982 f2_at20=0.350877 brier=0.018971 ece10=0.012011 rho=0.250000 donor_rows=4095 effective_n=18660.059382 seconds=0.602168 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=hgb_pool_equal in_sample_bce=0.031847 local_bce=0.027447 donor_bce=0.045048 in_sample_local_ap=0.868583 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=hgb_pool_equal mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_duc': 271, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 18, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 11, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 19} positive_ess=377.376828 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=hgb_language_prior rows=1909 failures=47 prevalence=0.024620 ap=0.389750 auc=0.902501 f2_at20=0.333333 brier=0.019571 ece10=0.012557 rho=0.250000 donor_rows=4095 effective_n=18660.059382 seconds=0.574081 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=hgb_language_prior in_sample_bce=0.029864 local_bce=0.026557 donor_bce=0.039782 in_sample_local_ap=0.873277 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=hgb_language_prior mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_duc': 271, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 22, 'khoa_tieng_nhat': 7, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 16, 'khoa_tieng_trung_quoc': 4} positive_ess=366.551438 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=hgb_random_donor rows=1909 failures=47 prevalence=0.024620 ap=0.297706 auc=0.890795 f2_at20=0.324561 brier=0.020997 ece10=0.008811 rho=0.250000 donor_rows=4095 effective_n=18660.059382 seconds=0.614919 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=hgb_random_donor in_sample_bce=0.031941 local_bce=0.027284 donor_bce=0.045913 in_sample_local_ap=0.865164 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=hgb_random_donor mass={'khoa_tieng_duc': 0.75, np.str_('khoa_viet_nam_hoc'): 0.08333333333333333, np.str_('khoa_cong_nghe_thong_tin'): 0.08333333333333333, np.str_('khoa_quoc_te_hoc'): 0.08333333333333333} source_failures={'khoa_tieng_duc': 271, np.str_('khoa_viet_nam_hoc'): 79, np.str_('khoa_cong_nghe_thong_tin'): 35, np.str_('khoa_quoc_te_hoc'): 12} positive_ess=394.130895 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=suite32_selective_common rows=1909 failures=47 prevalence=0.024620 ap=0.261755 auc=0.883893 f2_at20=0.315789 brier=0.021610 ece10=0.007849 rho=0.250000 donor_rows=4095 effective_n=18631.285363 seconds=0.578156 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=suite32_selective_common in_sample_bce=0.031246 local_bce=0.026763 donor_bce=0.044098 in_sample_local_ap=0.875325 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=suite32_selective_common mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.0914998371804338, 'khoa_tieng_nga': 0.08179330956458888, 'khoa_quoc_te_hoc': 0.07670685325497732} source_failures={'khoa_tieng_duc': 271, 'khoa_cong_nghe_thong_tin': 35, 'khoa_tieng_nga': 61, 'khoa_quoc_te_hoc': 12} positive_ess=375.484810 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=suite32_selective_native rows=1909 failures=47 prevalence=0.024620 ap=0.329379 auc=0.886378 f2_at20=0.315789 brier=0.020543 ece10=0.008372 rho=0.250000 donor_rows=4095 effective_n=18631.285363 seconds=1.095628 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=suite32_selective_native in_sample_bce=0.031268 local_bce=0.026525 donor_bce=0.044879 in_sample_local_ap=0.875499 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=suite32_selective_native mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.0914998371804338, 'khoa_tieng_nga': 0.08179330956458888, 'khoa_quoc_te_hoc': 0.07670685325497732} source_failures={'khoa_tieng_duc': 271, 'khoa_cong_nghe_thong_tin': 35, 'khoa_tieng_nga': 61, 'khoa_quoc_te_hoc': 12} positive_ess=375.484810 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=native_cache_nan_indicator rows=1909 failures=47 prevalence=0.024620 ap=0.261247 auc=0.878288 f2_at20=0.298246 brier=0.025224 ece10=0.017865 rho=0.000000 donor_rows=0 effective_n=14676.000000 seconds=1.945374 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=native_cache_nan_indicator in_sample_bce=0.020089 local_bce=0.020089 donor_bce=nan in_sample_local_ap=0.934501 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=native_cache_nan_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 271} positive_ess=271.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=native_naive_zero_indicator rows=1909 failures=47 prevalence=0.024620 ap=0.277399 auc=0.871438 f2_at20=0.289474 brier=0.024595 ece10=0.016041 rho=0.000000 donor_rows=0 effective_n=14676.000000 seconds=1.918725 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=native_naive_zero_indicator in_sample_bce=0.020066 local_bce=0.020066 donor_bce=nan in_sample_local_ap=0.935586 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=native_naive_zero_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 271} positive_ess=271.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=native_placebo_indicator rows=1909 failures=47 prevalence=0.024620 ap=0.261247 auc=0.878288 f2_at20=0.298246 brier=0.025224 ece10=0.017865 rho=0.000000 donor_rows=0 effective_n=14676.000000 seconds=1.726727 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=native_placebo_indicator in_sample_bce=0.020089 local_bce=0.020089 donor_bce=nan in_sample_local_ap=0.934501 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=native_placebo_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 271} positive_ess=271.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan rows=1909 failures=47 prevalence=0.024620 ap=0.329379 auc=0.886378 f2_at20=0.315789 brier=0.020543 ece10=0.008372 rho=0.250000 donor_rows=4095 effective_n=18631.285363 seconds=2.080051 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan in_sample_bce=0.031268 local_bce=0.026525 donor_bce=0.044879 in_sample_local_ap=0.875499 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.0914998371804338, 'khoa_tieng_nga': 0.08179330956458888, 'khoa_quoc_te_hoc': 0.07670685325497732} source_failures={'khoa_tieng_duc': 271, 'khoa_cong_nghe_thong_tin': 35, 'khoa_tieng_nga': 61, 'khoa_quoc_te_hoc': 12} positive_ess=375.484810 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_duc model=committee_reference ap=0.247552 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_duc models=13 positive_rows=47 num_edges=52 num_replays=2 chosen=['khoa_cong_nghe_thong_tin', 'khoa_tieng_nga', 'khoa_quoc_te_hoc'] [SELECTION_ORIGINS] target=khoa_tieng_han_quoc fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_han_quoc fold=3 origin=6063 ap=0.008959 rows=3094 failures=9 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.009311 ap=0.018270 base_ap=0.008959 donor_failures=173 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.000393 ap=0.009352 base_ap=0.008959 donor_failures=83 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.015823 ap=0.024782 base_ap=0.008959 donor_failures=32 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6063 delta_ap=0.006861 ap=0.015820 base_ap=0.008959 donor_failures=81 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.008463 ap=0.017423 base_ap=0.008959 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6063 delta_ap=0.007318 ap=0.016277 base_ap=0.008959 donor_failures=56 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6063 delta_ap=-0.000758 ap=0.008201 base_ap=0.008959 donor_failures=156 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6063 delta_ap=0.010756 ap=0.019715 base_ap=0.008959 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6063 delta_ap=0.011560 ap=0.020519 base_ap=0.008959 donor_failures=50 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6063 delta_ap=0.002818 ap=0.011777 base_ap=0.008959 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.003963 ap=0.012923 base_ap=0.008959 donor_failures=88 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.022554 ap=0.031513 base_ap=0.008959 donor_failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.008273 ap=0.017232 base_ap=0.008959 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=3 origin=6064 ap=0.058115 rows=3265 failures=12 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.017394 ap=0.040721 base_ap=0.058115 donor_failures=134 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.030068 ap=0.028047 base_ap=0.058115 donor_failures=104 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.025869 ap=0.032246 base_ap=0.058115 donor_failures=28 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6064 delta_ap=-0.002097 ap=0.056019 base_ap=0.058115 donor_failures=59 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.024832 ap=0.033283 base_ap=0.058115 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6064 delta_ap=0.000592 ap=0.058707 base_ap=0.058115 donor_failures=56 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6064 delta_ap=-0.018631 ap=0.039484 base_ap=0.058115 donor_failures=121 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.001091 ap=0.057024 base_ap=0.058115 donor_failures=146 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.026247 ap=0.084362 base_ap=0.058115 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.031818 ap=0.026297 base_ap=0.058115 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.022991 ap=0.035125 base_ap=0.058115 donor_failures=66 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.110302 ap=0.168417 base_ap=0.058115 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.020562 ap=0.037553 base_ap=0.058115 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=3 origin=6066 ap=0.031615 rows=2904 failures=20 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.024422 ap=0.056038 base_ap=0.031615 donor_failures=126 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.014646 ap=0.046261 base_ap=0.031615 donor_failures=98 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.000701 ap=0.030914 base_ap=0.031615 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6066 delta_ap=0.005639 ap=0.037254 base_ap=0.031615 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.061755 ap=0.093370 base_ap=0.031615 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6066 delta_ap=0.008784 ap=0.040399 base_ap=0.031615 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6066 delta_ap=0.010347 ap=0.041962 base_ap=0.031615 donor_failures=137 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6066 delta_ap=0.022996 ap=0.054611 base_ap=0.031615 donor_failures=177 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6066 delta_ap=0.001549 ap=0.033164 base_ap=0.031615 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6066 delta_ap=0.009205 ap=0.040821 base_ap=0.031615 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.003121 ap=0.034737 base_ap=0.031615 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.018390 ap=0.050005 base_ap=0.031615 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.006265 ap=0.037880 base_ap=0.031615 donor_failures=263 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=3 origin=6067 ap=0.322291 rows=3337 failures=29 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.015578 ap=0.306713 base_ap=0.322291 donor_failures=143 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.020150 ap=0.302141 base_ap=0.322291 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.090708 ap=0.231583 base_ap=0.322291 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6067 delta_ap=-0.080988 ap=0.241303 base_ap=0.322291 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.071824 ap=0.394116 base_ap=0.322291 donor_failures=132 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6067 delta_ap=-0.031586 ap=0.290705 base_ap=0.322291 donor_failures=61 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6067 delta_ap=0.076170 ap=0.398461 base_ap=0.322291 donor_failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6067 delta_ap=0.027914 ap=0.350205 base_ap=0.322291 donor_failures=182 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.061671 ap=0.260620 base_ap=0.322291 donor_failures=45 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6067 delta_ap=0.017578 ap=0.339869 base_ap=0.322291 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.070067 ap=0.252224 base_ap=0.322291 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.019891 ap=0.302400 base_ap=0.322291 donor_failures=31 [TRANSFER_EDGE] fold=3 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.012761 ap=0.309530 base_ap=0.322291 donor_failures=250 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_han_quoc origin=6066 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_nhat,khoa_tieng_nga policy_donor_count=3 base_ap=0.031615 policy_ap=0.053513 delta_ap=0.021898 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_han_quoc origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_bo_dao_nha,khoa_tieng_nhat policy_donor_count=3 base_ap=0.322291 policy_ap=0.291289 delta_ap=-0.031002 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_han_quoc selected=[] reason=prequential_policy_delta_-0.004552_below_threshold replay_mean=-0.004552 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_local_common rows=3235 failures=42 prevalence=0.012983 ap=0.240807 auc=0.831626 f2_at20=0.214724 brier=0.011361 ece10=0.006567 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=0.642206 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=hgb_local_common in_sample_bce=0.005380 local_bce=0.005380 donor_bce=nan in_sample_local_ap=0.960749 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_local_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_local_native rows=3235 failures=42 prevalence=0.012983 ap=0.225760 auc=0.836473 f2_at20=0.214724 brier=0.011701 ece10=0.006966 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=1.146137 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=hgb_local_native in_sample_bce=0.005206 local_bce=0.005206 donor_bce=nan in_sample_local_ap=0.962291 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_local_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_pool_all rows=3235 failures=42 prevalence=0.012983 ap=0.245062 auc=0.841476 f2_at20=0.208589 brier=0.010972 ece10=0.005826 rho=0.250000 donor_rows=4095 effective_n=21009.162198 seconds=0.608394 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=hgb_pool_all in_sample_bce=0.012098 local_bce=0.009154 donor_bce=0.039909 in_sample_local_ap=0.824435 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_pool_all mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.023601078906464297, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03725802975482961, 'khoa_quoc_te_hoc': 0.012161372274977468, 'khoa_tieng_anh': 0.052647304693112504, 'khoa_tieng_bo_dao_nha': 0.0045724539244243, 'khoa_tieng_duc': 0.011980955864104808, 'khoa_tieng_italia': 0.012114839535521624, 'khoa_tieng_nga': 0.007970160268551052, 'khoa_tieng_nhat': 0.019862948836844786, 'khoa_tieng_phap': 0.016388504290808396, 'khoa_tieng_tay_ban_nha': 0.008708969552893846, 'khoa_tieng_trung_quoc': 0.03735517705299181, 'khoa_viet_nam_hoc': 0.005378205044475503} source_failures={'khoa_tieng_han_quoc': 124, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 7, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 27} positive_ess=152.031938 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_pool_equal rows=3235 failures=42 prevalence=0.012983 ap=0.263712 auc=0.831216 f2_at20=0.202454 brier=0.010873 ece10=0.004856 rho=0.250000 donor_rows=4095 effective_n=25406.713844 seconds=0.585985 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=hgb_pool_equal in_sample_bce=0.014751 local_bce=0.010079 donor_bce=0.028768 in_sample_local_ap=0.788854 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_pool_equal mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_han_quoc': 124, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 7, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 27} positive_ess=215.659446 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_language_prior rows=3235 failures=42 prevalence=0.012983 ap=0.287334 auc=0.839086 f2_at20=0.208589 brier=0.010717 ece10=0.003718 rho=0.250000 donor_rows=4095 effective_n=25406.713844 seconds=0.631959 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=hgb_language_prior in_sample_bce=0.013646 local_bce=0.009708 donor_bce=0.025459 in_sample_local_ap=0.806808 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_language_prior mass={'khoa_tieng_han_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_han_quoc': 124, 'khoa_tieng_anh': 13, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 11, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 23, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 13, 'khoa_tieng_trung_quoc': 2} positive_ess=208.031826 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_random_donor rows=3235 failures=42 prevalence=0.012983 ap=0.349236 auc=0.826842 f2_at20=0.214724 brier=0.010145 ece10=0.004287 rho=0.250000 donor_rows=4096 effective_n=25409.119338 seconds=0.592110 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=hgb_random_donor in_sample_bce=0.015599 local_bce=0.010383 donor_bce=0.031249 in_sample_local_ap=0.761157 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=hgb_random_donor mass={'khoa_tieng_han_quoc': 0.75, np.str_('khoa_tieng_italia'): 0.25} source_failures={'khoa_tieng_han_quoc': 124, np.str_('khoa_tieng_italia'): 148} positive_ess=249.428800 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_common rows=3235 failures=42 prevalence=0.012983 ap=0.240807 auc=0.831626 f2_at20=0.214724 brier=0.011361 ece10=0.006567 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=0.605468 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_common in_sample_bce=0.005380 local_bce=0.005380 donor_bce=nan in_sample_local_ap=0.960749 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_native rows=3235 failures=42 prevalence=0.012983 ap=0.225760 auc=0.836473 f2_at20=0.214724 brier=0.011701 ece10=0.006966 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=1.026912 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_native in_sample_bce=0.005206 local_bce=0.005206 donor_bce=nan in_sample_local_ap=0.962291 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator rows=3235 failures=42 prevalence=0.012983 ap=0.225760 auc=0.836473 f2_at20=0.214724 brier=0.011701 ece10=0.006966 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=1.879700 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator in_sample_bce=0.005206 local_bce=0.005206 donor_bce=nan in_sample_local_ap=0.962291 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator rows=3235 failures=42 prevalence=0.012983 ap=0.225760 auc=0.836473 f2_at20=0.214724 brier=0.011701 ece10=0.006966 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=1.911408 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator in_sample_bce=0.005206 local_bce=0.005206 donor_bce=nan in_sample_local_ap=0.962291 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=native_placebo_indicator rows=3235 failures=42 prevalence=0.012983 ap=0.225760 auc=0.836473 f2_at20=0.214724 brier=0.011701 ece10=0.006966 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=1.922331 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=native_placebo_indicator in_sample_bce=0.005206 local_bce=0.005206 donor_bce=nan in_sample_local_ap=0.962291 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=native_placebo_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan rows=3235 failures=42 prevalence=0.012983 ap=0.225760 auc=0.836473 f2_at20=0.214724 brier=0.011701 ece10=0.006966 rho=0.000000 donor_rows=0 effective_n=23343.000000 seconds=1.975008 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.005206 local_bce=0.005206 donor_bce=nan in_sample_local_ap=0.962291 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 124} positive_ess=124.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_han_quoc model=committee_reference ap=0.214705 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_han_quoc models=13 positive_rows=42 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_italia fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_italia fold=3 origin=6063 ap=0.083912 rows=2287 failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.057206 ap=0.141118 base_ap=0.083912 donor_failures=158 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.018201 ap=0.065711 base_ap=0.083912 donor_failures=95 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6063 delta_ap=0.056516 ap=0.140428 base_ap=0.083912 donor_failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_anh origin=6063 delta_ap=-0.013738 ap=0.070174 base_ap=0.083912 donor_failures=68 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.017146 ap=0.066766 base_ap=0.083912 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_duc origin=6063 delta_ap=0.024014 ap=0.107926 base_ap=0.083912 donor_failures=60 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.027860 ap=0.056052 base_ap=0.083912 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nga origin=6063 delta_ap=-0.020036 ap=0.063875 base_ap=0.083912 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6063 delta_ap=-0.000666 ap=0.083246 base_ap=0.083912 donor_failures=61 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_phap origin=6063 delta_ap=0.041292 ap=0.125204 base_ap=0.083912 donor_failures=57 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.010607 ap=0.094519 base_ap=0.083912 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.009881 ap=0.093793 base_ap=0.083912 donor_failures=28 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.015232 ap=0.068680 base_ap=0.083912 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_italia fold=3 origin=6064 ap=0.291326 rows=2017 failures=65 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.060461 ap=0.351787 base_ap=0.291326 donor_failures=113 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.029337 ap=0.320663 base_ap=0.291326 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6064 delta_ap=0.021641 ap=0.312967 base_ap=0.291326 donor_failures=31 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_anh origin=6064 delta_ap=-0.011410 ap=0.279917 base_ap=0.291326 donor_failures=59 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.040912 ap=0.332238 base_ap=0.291326 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_duc origin=6064 delta_ap=0.026775 ap=0.318102 base_ap=0.291326 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6064 delta_ap=0.043888 ap=0.335215 base_ap=0.291326 donor_failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nga origin=6064 delta_ap=0.075413 ap=0.366739 base_ap=0.291326 donor_failures=149 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6064 delta_ap=0.009554 ap=0.300881 base_ap=0.291326 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_phap origin=6064 delta_ap=-0.023837 ap=0.267489 base_ap=0.291326 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.045191 ap=0.336518 base_ap=0.291326 donor_failures=69 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.025413 ap=0.316739 base_ap=0.291326 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6064 delta_ap=0.052297 ap=0.343624 base_ap=0.291326 donor_failures=275 [SCREEN_BASE] target=khoa_tieng_italia fold=3 origin=6066 ap=0.222295 rows=2166 failures=111 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.041192 ap=0.263487 base_ap=0.222295 donor_failures=133 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.032169 ap=0.254464 base_ap=0.222295 donor_failures=101 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6066 delta_ap=0.061496 ap=0.283791 base_ap=0.222295 donor_failures=34 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_anh origin=6066 delta_ap=0.015187 ap=0.237482 base_ap=0.222295 donor_failures=73 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.068496 ap=0.290791 base_ap=0.222295 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_duc origin=6066 delta_ap=0.007412 ap=0.229707 base_ap=0.222295 donor_failures=62 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6066 delta_ap=0.043460 ap=0.265755 base_ap=0.222295 donor_failures=20 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nga origin=6066 delta_ap=0.000034 ap=0.222329 base_ap=0.222295 donor_failures=165 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6066 delta_ap=-0.004624 ap=0.217671 base_ap=0.222295 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_phap origin=6066 delta_ap=0.072768 ap=0.295062 base_ap=0.222295 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.004075 ap=0.218219 base_ap=0.222295 donor_failures=94 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.060428 ap=0.282723 base_ap=0.222295 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6066 delta_ap=0.006055 ap=0.228350 base_ap=0.222295 donor_failures=256 [SCREEN_BASE] target=khoa_tieng_italia fold=3 origin=6067 ap=0.508695 rows=2024 failures=71 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.014131 ap=0.522826 base_ap=0.508695 donor_failures=137 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.067390 ap=0.576085 base_ap=0.508695 donor_failures=102 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.026088 ap=0.482608 base_ap=0.508695 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_anh origin=6067 delta_ap=0.006597 ap=0.515293 base_ap=0.508695 donor_failures=76 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.016973 ap=0.525669 base_ap=0.508695 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_duc origin=6067 delta_ap=-0.040777 ap=0.467919 base_ap=0.508695 donor_failures=73 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6067 delta_ap=0.022246 ap=0.530942 base_ap=0.508695 donor_failures=12 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nga origin=6067 delta_ap=-0.056488 ap=0.452207 base_ap=0.508695 donor_failures=190 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6067 delta_ap=-0.030988 ap=0.477707 base_ap=0.508695 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_phap origin=6067 delta_ap=0.024770 ap=0.533466 base_ap=0.508695 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.000140 ap=0.508556 base_ap=0.508695 donor_failures=100 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.055412 ap=0.564107 base_ap=0.508695 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6067 delta_ap=0.072949 ap=0.581645 base_ap=0.508695 donor_failures=240 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_italia origin=6066 policy_sources=khoa_cong_nghe_thong_tin,khoa_quoc_te_hoc,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.222295 policy_ap=0.243294 delta_ap=0.021000 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_italia origin=6067 policy_sources=khoa_cong_nghe_thong_tin,khoa_quoc_te_hoc,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.508695 policy_ap=0.487516 delta_ap=-0.021179 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_italia selected=[] reason=prequential_policy_delta_-0.000090_below_threshold replay_mean=-0.000090 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=hgb_local_common rows=2145 failures=68 prevalence=0.031702 ap=0.250162 auc=0.858811 f2_at20=0.378031 brier=0.027494 ece10=0.011540 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=0.589948 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=hgb_local_common in_sample_bce=0.041276 local_bce=0.041276 donor_bce=nan in_sample_local_ap=0.898222 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=hgb_local_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=hgb_local_native rows=2145 failures=68 prevalence=0.031702 ap=0.240822 auc=0.868171 f2_at20=0.370899 brier=0.027316 ece10=0.009397 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=1.049501 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=hgb_local_native in_sample_bce=0.040356 local_bce=0.040356 donor_bce=nan in_sample_local_ap=0.902479 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=hgb_local_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=hgb_pool_all rows=2145 failures=68 prevalence=0.031702 ap=0.282964 auc=0.888159 f2_at20=0.370899 brier=0.026557 ece10=0.004824 rho=0.250000 donor_rows=4095 effective_n=16446.538708 seconds=0.598410 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=hgb_pool_all in_sample_bce=0.043229 local_bce=0.046265 donor_bce=0.054870 in_sample_local_ap=0.872980 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=hgb_pool_all mass={'khoa_tieng_italia': 0.7500000000000001, 'khoa_cong_nghe_thong_tin': 0.02296347132068158, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0362514654999857, 'khoa_quoc_te_hoc': 0.011832820209761103, 'khoa_tieng_anh': 0.05122498324008146, 'khoa_tieng_bo_dao_nha': 0.004448924346839763, 'khoa_tieng_duc': 0.011657277935050947, 'khoa_tieng_han_quoc': 0.018541553477643382, 'khoa_tieng_nga': 0.007754838135725157, 'khoa_tieng_nhat': 0.019326330705759374, 'khoa_tieng_phap': 0.015945751876952016, 'khoa_tieng_tay_ban_nha': 0.008473687722207924, 'khoa_tieng_trung_quoc': 0.03634598826329117, 'khoa_viet_nam_hoc': 0.005232907266020417} source_failures={'khoa_tieng_italia': 508, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 20} positive_ess=529.083315 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=hgb_pool_equal rows=2145 failures=68 prevalence=0.031702 ap=0.262529 auc=0.877198 f2_at20=0.370899 brier=0.026909 ece10=0.006871 rho=0.250000 donor_rows=4095 effective_n=18808.720379 seconds=0.621843 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=hgb_pool_equal in_sample_bce=0.046754 local_bce=0.047238 donor_bce=0.045304 in_sample_local_ap=0.868714 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=hgb_pool_equal mass={'khoa_tieng_italia': 0.7499999999999999, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_italia': 508, 'khoa_cong_nghe_thong_tin': 8, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 20} positive_ess=608.536270 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=hgb_language_prior rows=2145 failures=68 prevalence=0.031702 ap=0.356070 auc=0.848916 f2_at20=0.328103 brier=0.025961 ece10=0.013704 rho=0.250000 donor_rows=4095 effective_n=18808.720379 seconds=0.579809 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=hgb_language_prior in_sample_bce=0.044246 local_bce=0.047076 donor_bce=0.035757 in_sample_local_ap=0.867078 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=hgb_language_prior mass={'khoa_tieng_italia': 0.7499999999999999, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_italia': 508, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 2} positive_ess=581.301711 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=hgb_random_donor rows=2145 failures=68 prevalence=0.031702 ap=0.276046 auc=0.851525 f2_at20=0.342368 brier=0.027170 ece10=0.010426 rho=0.250000 donor_rows=4096 effective_n=18810.038682 seconds=0.585112 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=hgb_random_donor in_sample_bce=0.051424 local_bce=0.048854 donor_bce=0.059133 in_sample_local_ap=0.856499 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=hgb_random_donor mass={'khoa_tieng_italia': 0.75, np.str_('khoa_cong_nghe_thong_tin'): 0.25} source_failures={'khoa_tieng_italia': 508, np.str_('khoa_cong_nghe_thong_tin'): 117} positive_ess=621.222483 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=suite32_selective_common rows=2145 failures=68 prevalence=0.031702 ap=0.250162 auc=0.858811 f2_at20=0.378031 brier=0.027494 ece10=0.011540 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=0.539389 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=suite32_selective_common in_sample_bce=0.041276 local_bce=0.041276 donor_bce=nan in_sample_local_ap=0.898222 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=suite32_selective_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=suite32_selective_native rows=2145 failures=68 prevalence=0.031702 ap=0.240822 auc=0.868171 f2_at20=0.370899 brier=0.027316 ece10=0.009397 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=1.143251 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=suite32_selective_native in_sample_bce=0.040356 local_bce=0.040356 donor_bce=nan in_sample_local_ap=0.902479 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=suite32_selective_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=native_cache_nan_indicator rows=2145 failures=68 prevalence=0.031702 ap=0.240822 auc=0.868171 f2_at20=0.370899 brier=0.027316 ece10=0.009397 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=2.113910 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=native_cache_nan_indicator in_sample_bce=0.040356 local_bce=0.040356 donor_bce=nan in_sample_local_ap=0.902479 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=native_cache_nan_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=native_naive_zero_indicator rows=2145 failures=68 prevalence=0.031702 ap=0.242461 auc=0.882629 f2_at20=0.378031 brier=0.027137 ece10=0.008638 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=1.944390 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=native_naive_zero_indicator in_sample_bce=0.040702 local_bce=0.040702 donor_bce=nan in_sample_local_ap=0.900052 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=native_naive_zero_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=native_placebo_indicator rows=2145 failures=68 prevalence=0.031702 ap=0.240822 auc=0.868171 f2_at20=0.370899 brier=0.027316 ece10=0.009397 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=1.905938 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=native_placebo_indicator in_sample_bce=0.040356 local_bce=0.040356 donor_bce=nan in_sample_local_ap=0.902479 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=native_placebo_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan rows=2145 failures=68 prevalence=0.031702 ap=0.240822 auc=0.868171 f2_at20=0.370899 brier=0.027316 ece10=0.009397 rho=0.000000 donor_rows=0 effective_n=14840.000000 seconds=2.176776 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan in_sample_bce=0.040356 local_bce=0.040356 donor_bce=nan in_sample_local_ap=0.902479 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 508} positive_ess=508.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_italia model=committee_reference ap=0.207328 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_italia models=13 positive_rows=68 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nga fold=3 origins=[6064, 6065, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nga fold=3 origin=6064 ap=0.350087 rows=1355 failures=78 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.005035 ap=0.345052 base_ap=0.350087 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.022688 ap=0.372775 base_ap=0.350087 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.009253 ap=0.340834 base_ap=0.350087 donor_failures=29 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_anh origin=6064 delta_ap=-0.012188 ap=0.337899 base_ap=0.350087 donor_failures=58 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.019141 ap=0.330946 base_ap=0.350087 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_duc origin=6064 delta_ap=-0.032752 ap=0.317335 base_ap=0.350087 donor_failures=55 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6064 delta_ap=0.016299 ap=0.366387 base_ap=0.350087 donor_failures=25 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_italia origin=6064 delta_ap=-0.014466 ap=0.335621 base_ap=0.350087 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6064 delta_ap=-0.020571 ap=0.329517 base_ap=0.350087 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_phap origin=6064 delta_ap=0.016254 ap=0.366341 base_ap=0.350087 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.041626 ap=0.308461 base_ap=0.350087 donor_failures=73 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.025044 ap=0.375131 base_ap=0.350087 donor_failures=24 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.015188 ap=0.334900 base_ap=0.350087 donor_failures=271 [SCREEN_BASE] target=khoa_tieng_nga fold=3 origin=6065 ap=0.305556 rows=45 failures=3 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=0.067460 ap=0.373016 base_ap=0.305556 donor_failures=114 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6065 delta_ap=0.248016 ap=0.553571 base_ap=0.305556 donor_failures=112 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6065 delta_ap=0.083333 ap=0.388889 base_ap=0.305556 donor_failures=38 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_anh origin=6065 delta_ap=0.004701 ap=0.310256 base_ap=0.305556 donor_failures=59 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=0.027778 ap=0.333333 base_ap=0.305556 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_duc origin=6065 delta_ap=0.000000 ap=0.305556 base_ap=0.305556 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6065 delta_ap=0.222222 ap=0.527778 base_ap=0.305556 donor_failures=16 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_italia origin=6065 delta_ap=0.036111 ap=0.341667 base_ap=0.305556 donor_failures=121 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6065 delta_ap=0.081349 ap=0.386905 base_ap=0.305556 donor_failures=45 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_phap origin=6065 delta_ap=-0.026984 ap=0.278571 base_ap=0.305556 donor_failures=49 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=0.047258 ap=0.352814 base_ap=0.305556 donor_failures=81 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6065 delta_ap=0.011905 ap=0.317460 base_ap=0.305556 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6065 delta_ap=0.208333 ap=0.513889 base_ap=0.305556 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_nga fold=3 origin=6066 ap=0.304964 rows=1588 failures=102 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.010621 ap=0.294343 base_ap=0.304964 donor_failures=116 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.017891 ap=0.322855 base_ap=0.304964 donor_failures=108 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6066 delta_ap=0.006047 ap=0.311011 base_ap=0.304964 donor_failures=27 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_anh origin=6066 delta_ap=0.061837 ap=0.366801 base_ap=0.304964 donor_failures=69 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.020088 ap=0.325052 base_ap=0.304964 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_duc origin=6066 delta_ap=0.013281 ap=0.318245 base_ap=0.304964 donor_failures=66 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.012024 ap=0.292940 base_ap=0.304964 donor_failures=18 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_italia origin=6066 delta_ap=-0.019181 ap=0.285783 base_ap=0.304964 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6066 delta_ap=-0.000366 ap=0.304599 base_ap=0.304964 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_phap origin=6066 delta_ap=0.027079 ap=0.332043 base_ap=0.304964 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.002918 ap=0.307883 base_ap=0.304964 donor_failures=89 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.015868 ap=0.320833 base_ap=0.304964 donor_failures=26 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6066 delta_ap=0.008367 ap=0.313331 base_ap=0.304964 donor_failures=262 [SCREEN_BASE] target=khoa_tieng_nga fold=3 origin=6067 ap=0.806834 rows=1567 failures=83 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.025779 ap=0.781054 base_ap=0.806834 donor_failures=110 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.028988 ap=0.777846 base_ap=0.806834 donor_failures=124 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.005143 ap=0.801691 base_ap=0.806834 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_anh origin=6067 delta_ap=0.002080 ap=0.808914 base_ap=0.806834 donor_failures=66 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.038298 ap=0.768535 base_ap=0.806834 donor_failures=132 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_duc origin=6067 delta_ap=-0.038252 ap=0.768582 base_ap=0.806834 donor_failures=60 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6067 delta_ap=0.010485 ap=0.817319 base_ap=0.806834 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_italia origin=6067 delta_ap=-0.022965 ap=0.783869 base_ap=0.806834 donor_failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6067 delta_ap=-0.013923 ap=0.792911 base_ap=0.806834 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_phap origin=6067 delta_ap=0.051518 ap=0.858352 base_ap=0.806834 donor_failures=56 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.006041 ap=0.800793 base_ap=0.806834 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.041732 ap=0.765102 base_ap=0.806834 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.024497 ap=0.782337 base_ap=0.806834 donor_failures=254 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_nga origin=6066 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_han_quoc,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.304964 policy_ap=0.277284 delta_ap=-0.027681 policy_donor_rows=4095 selected_using_origins=6064,6065 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_nga origin=6067 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_han_quoc,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.806834 policy_ap=0.768273 delta_ap=-0.038561 policy_donor_rows=4095 selected_using_origins=6064,6065,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_nga selected=[] reason=prequential_policy_delta_-0.033121_below_threshold replay_mean=-0.033121 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=hgb_local_common rows=1996 failures=92 prevalence=0.046092 ap=0.480890 auc=0.929211 f2_at20=0.520833 brier=0.032731 ece10=0.013700 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=0.559321 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=hgb_local_common in_sample_bce=0.048493 local_bce=0.048493 donor_bce=nan in_sample_local_ap=0.921459 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=hgb_local_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=hgb_local_native rows=1996 failures=92 prevalence=0.046092 ap=0.456076 auc=0.929802 f2_at20=0.540365 brier=0.034126 ece10=0.019387 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=1.074614 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=hgb_local_native in_sample_bce=0.048475 local_bce=0.048475 donor_bce=nan in_sample_local_ap=0.921866 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=hgb_local_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=hgb_pool_all rows=1996 failures=92 prevalence=0.046092 ap=0.471541 auc=0.937197 f2_at20=0.527344 brier=0.032488 ece10=0.011287 rho=0.250000 donor_rows=4095 effective_n=12494.487700 seconds=0.550223 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=hgb_pool_all in_sample_bce=0.051910 local_bce=0.054784 donor_bce=0.060612 in_sample_local_ap=0.894159 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=hgb_pool_all mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.022598931885834356, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03567598243990294, 'khoa_quoc_te_hoc': 0.011644977111839308, 'khoa_tieng_anh': 0.05041179928458864, 'khoa_tieng_bo_dao_nha': 0.004378298771793782, 'khoa_tieng_duc': 0.011472221527378243, 'khoa_tieng_han_quoc': 0.018247210896265353, 'khoa_tieng_italia': 0.011600420241638942, 'khoa_tieng_nhat': 0.01901952997973835, 'khoa_tieng_phap': 0.015692617004777747, 'khoa_tieng_tay_ban_nha': 0.008339170022763088, 'khoa_tieng_trung_quoc': 0.03576900467768967, 'khoa_viet_nam_hoc': 0.005149836155789579} source_failures={'khoa_tieng_nga': 455, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 12} positive_ess=514.639713 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=hgb_pool_equal rows=1996 failures=92 prevalence=0.046092 ap=0.427408 auc=0.934372 f2_at20=0.540365 brier=0.034256 ece10=0.013094 rho=0.250000 donor_rows=4095 effective_n=13721.561629 seconds=0.559144 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=hgb_pool_equal in_sample_bce=0.056032 local_bce=0.056174 donor_bce=0.055606 in_sample_local_ap=0.890468 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=hgb_pool_equal mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nga': 455, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 12} positive_ess=549.364396 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=hgb_language_prior rows=1996 failures=92 prevalence=0.046092 ap=0.471439 auc=0.933167 f2_at20=0.533854 brier=0.032336 ece10=0.009187 rho=0.250000 donor_rows=4095 effective_n=13721.561629 seconds=0.557211 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=hgb_language_prior in_sample_bce=0.051063 local_bce=0.054491 donor_bce=0.040780 in_sample_local_ap=0.894182 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=hgb_language_prior mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nga': 455, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 12, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 21, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 15, 'khoa_tieng_trung_quoc': 4} positive_ess=534.863379 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=hgb_random_donor rows=1996 failures=92 prevalence=0.046092 ap=0.458334 auc=0.932705 f2_at20=0.540365 brier=0.032496 ece10=0.012753 rho=0.250000 donor_rows=4096 effective_n=13722.263238 seconds=0.533657 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=hgb_random_donor in_sample_bce=0.057380 local_bce=0.056008 donor_bce=0.061494 in_sample_local_ap=0.893325 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=hgb_random_donor mass={'khoa_tieng_nga': 0.75, np.str_('khoa_tieng_bo_dao_nha'): 0.25} source_failures={'khoa_tieng_nga': 455, np.str_('khoa_tieng_bo_dao_nha'): 143} positive_ess=592.961553 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=suite32_selective_common rows=1996 failures=92 prevalence=0.046092 ap=0.480890 auc=0.929211 f2_at20=0.520833 brier=0.032731 ece10=0.013700 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=0.479860 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=suite32_selective_common in_sample_bce=0.048493 local_bce=0.048493 donor_bce=nan in_sample_local_ap=0.921459 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=suite32_selective_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=suite32_selective_native rows=1996 failures=92 prevalence=0.046092 ap=0.456076 auc=0.929802 f2_at20=0.540365 brier=0.034126 ece10=0.019387 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=1.092380 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=suite32_selective_native in_sample_bce=0.048475 local_bce=0.048475 donor_bce=nan in_sample_local_ap=0.921866 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=suite32_selective_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=native_cache_nan_indicator rows=1996 failures=92 prevalence=0.046092 ap=0.456076 auc=0.929802 f2_at20=0.540365 brier=0.034126 ece10=0.019387 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=1.736601 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=native_cache_nan_indicator in_sample_bce=0.048475 local_bce=0.048475 donor_bce=nan in_sample_local_ap=0.921866 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=native_cache_nan_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=native_naive_zero_indicator rows=1996 failures=92 prevalence=0.046092 ap=0.456076 auc=0.929802 f2_at20=0.540365 brier=0.034126 ece10=0.019387 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=1.889462 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=native_naive_zero_indicator in_sample_bce=0.048475 local_bce=0.048475 donor_bce=nan in_sample_local_ap=0.921866 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=native_naive_zero_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=native_placebo_indicator rows=1996 failures=92 prevalence=0.046092 ap=0.456076 auc=0.929802 f2_at20=0.540365 brier=0.034126 ece10=0.019387 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=1.792516 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=native_placebo_indicator in_sample_bce=0.048475 local_bce=0.048475 donor_bce=nan in_sample_local_ap=0.921866 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=native_placebo_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan rows=1996 failures=92 prevalence=0.046092 ap=0.456076 auc=0.929802 f2_at20=0.540365 brier=0.034126 ece10=0.019387 rho=0.000000 donor_rows=0 effective_n=9763.000000 seconds=1.954371 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan in_sample_bce=0.048475 local_bce=0.048475 donor_bce=nan in_sample_local_ap=0.921866 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 455} positive_ess=455.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nga model=committee_reference ap=0.421984 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_nga models=13 positive_rows=92 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nhat fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nhat fold=3 origin=6063 ap=0.029536 rows=3170 failures=7 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.018092 ap=0.011444 base_ap=0.029536 donor_failures=180 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.009266 ap=0.020270 base_ap=0.029536 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.001109 ap=0.028427 base_ap=0.029536 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6063 delta_ap=0.048809 ap=0.078345 base_ap=0.029536 donor_failures=78 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.014853 ap=0.014683 base_ap=0.029536 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6063 delta_ap=0.026651 ap=0.056187 base_ap=0.029536 donor_failures=74 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.009285 ap=0.020251 base_ap=0.029536 donor_failures=15 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6063 delta_ap=0.001184 ap=0.030720 base_ap=0.029536 donor_failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6063 delta_ap=-0.012417 ap=0.017119 base_ap=0.029536 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6063 delta_ap=-0.002794 ap=0.026742 base_ap=0.029536 donor_failures=66 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.018391 ap=0.011145 base_ap=0.029536 donor_failures=84 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6063 delta_ap=-0.018105 ap=0.011431 base_ap=0.029536 donor_failures=30 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.016702 ap=0.012834 base_ap=0.029536 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_nhat fold=3 origin=6064 ap=0.141061 rows=2945 failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.050716 ap=0.090345 base_ap=0.141061 donor_failures=126 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.027849 ap=0.168910 base_ap=0.141061 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.061869 ap=0.079192 base_ap=0.141061 donor_failures=28 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6064 delta_ap=-0.094315 ap=0.046746 base_ap=0.141061 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.152554 ap=0.293615 base_ap=0.141061 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6064 delta_ap=0.032377 ap=0.173438 base_ap=0.141061 donor_failures=61 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.069503 ap=0.071558 base_ap=0.141061 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6064 delta_ap=-0.043216 ap=0.097845 base_ap=0.141061 donor_failures=129 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6064 delta_ap=-0.016203 ap=0.124858 base_ap=0.141061 donor_failures=142 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6064 delta_ap=0.046865 ap=0.187926 base_ap=0.141061 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.046850 ap=0.094211 base_ap=0.141061 donor_failures=77 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.003023 ap=0.138038 base_ap=0.141061 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.094238 ap=0.046823 base_ap=0.141061 donor_failures=271 [SCREEN_BASE] target=khoa_tieng_nhat fold=3 origin=6066 ap=0.403395 rows=2840 failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.124976 ap=0.528371 base_ap=0.403395 donor_failures=109 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.000776 ap=0.402619 base_ap=0.403395 donor_failures=104 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.196743 ap=0.206652 base_ap=0.403395 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6066 delta_ap=-0.181842 ap=0.221553 base_ap=0.403395 donor_failures=70 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.230485 ap=0.172911 base_ap=0.403395 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6066 delta_ap=-0.165522 ap=0.237873 base_ap=0.403395 donor_failures=72 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.009996 ap=0.393400 base_ap=0.403395 donor_failures=11 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6066 delta_ap=0.064305 ap=0.467700 base_ap=0.403395 donor_failures=129 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6066 delta_ap=-0.216244 ap=0.187151 base_ap=0.403395 donor_failures=161 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6066 delta_ap=-0.149717 ap=0.253678 base_ap=0.403395 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.025938 ap=0.429333 base_ap=0.403395 donor_failures=83 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.117763 ap=0.521158 base_ap=0.403395 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6066 delta_ap=0.121348 ap=0.524744 base_ap=0.403395 donor_failures=270 [SCREEN_BASE] target=khoa_tieng_nhat fold=3 origin=6067 ap=0.486281 rows=2448 failures=11 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.130945 ap=0.355336 base_ap=0.486281 donor_failures=138 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.022956 ap=0.509237 base_ap=0.486281 donor_failures=116 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6067 delta_ap=0.047505 ap=0.533786 base_ap=0.486281 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6067 delta_ap=-0.181800 ap=0.304481 base_ap=0.486281 donor_failures=62 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.061646 ap=0.424635 base_ap=0.486281 donor_failures=133 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6067 delta_ap=-0.219549 ap=0.266732 base_ap=0.486281 donor_failures=68 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6067 delta_ap=0.000755 ap=0.487036 base_ap=0.486281 donor_failures=16 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6067 delta_ap=-0.031945 ap=0.454336 base_ap=0.486281 donor_failures=144 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6067 delta_ap=0.138236 ap=0.624517 base_ap=0.486281 donor_failures=192 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6067 delta_ap=-0.150704 ap=0.335578 base_ap=0.486281 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.035358 ap=0.521639 base_ap=0.486281 donor_failures=107 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.185441 ap=0.300840 base_ap=0.486281 donor_failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.026451 ap=0.459830 base_ap=0.486281 donor_failures=253 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_nhat origin=6066 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_duc,khoa_tieng_phap policy_donor_count=3 base_ap=0.403395 policy_ap=0.545950 delta_ap=0.142555 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_nhat origin=6067 policy_sources=khoa_tieng_italia policy_donor_count=1 base_ap=0.486281 policy_ap=0.454336 delta_ap=-0.031945 policy_donor_rows=4096 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_nhat selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.055305 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=hgb_local_common rows=2588 failures=17 prevalence=0.006569 ap=0.223512 auc=0.924703 f2_at20=0.127986 brier=0.005614 ece10=0.003068 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=0.610446 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=hgb_local_common in_sample_bce=0.007402 local_bce=0.007402 donor_bce=nan in_sample_local_ap=0.955067 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=hgb_local_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=hgb_local_native rows=2588 failures=17 prevalence=0.006569 ap=0.203819 auc=0.925092 f2_at20=0.127986 brier=0.005842 ece10=0.003302 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=1.207451 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=hgb_local_native in_sample_bce=0.006985 local_bce=0.006985 donor_bce=nan in_sample_local_ap=0.963361 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=hgb_local_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=hgb_pool_all rows=2588 failures=17 prevalence=0.006569 ap=0.400407 auc=0.924314 f2_at20=0.127986 brier=0.004997 ece10=0.003399 rho=0.250000 donor_rows=4095 effective_n=21426.189385 seconds=0.682922 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=hgb_pool_all in_sample_bce=0.013861 local_bce=0.011169 donor_bce=0.043134 in_sample_local_ap=0.897091 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=hgb_pool_all mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.02367746881224447, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.037378623283363034, 'khoa_quoc_te_hoc': 0.012200735139951777, 'khoa_tieng_anh': 0.052817708879337455, 'khoa_tieng_bo_dao_nha': 0.004587253642939512, 'khoa_tieng_duc': 0.012019734773037007, 'khoa_tieng_han_quoc': 0.019118061379599538, 'khoa_tieng_italia': 0.012154051787399098, 'khoa_tieng_nga': 0.007995957385470175, 'khoa_tieng_phap': 0.016441549166579306, 'khoa_tieng_tay_ban_nha': 0.008737157983017088, 'khoa_tieng_trung_quoc': 0.03747608501939407, 'khoa_viet_nam_hoc': 0.005395612747667471} source_failures={'khoa_tieng_nhat': 227, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 0, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 13, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 33} positive_ess=233.756008 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=hgb_pool_equal rows=2588 failures=17 prevalence=0.006569 ap=0.366637 auc=0.924875 f2_at20=0.127986 brier=0.005268 ece10=0.002007 rho=0.250000 donor_rows=4095 effective_n=26054.442520 seconds=0.672675 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=hgb_pool_equal in_sample_bce=0.017078 local_bce=0.012510 donor_bce=0.030783 in_sample_local_ap=0.867267 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=hgb_pool_equal mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nhat': 227, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 0, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 13, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 33} positive_ess=302.314702 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=hgb_language_prior rows=2588 failures=17 prevalence=0.006569 ap=0.314771 auc=0.944769 f2_at20=0.136519 brier=0.005744 ece10=0.003117 rho=0.250000 donor_rows=4095 effective_n=26054.442520 seconds=0.663980 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=hgb_language_prior in_sample_bce=0.013474 local_bce=0.011263 donor_bce=0.020109 in_sample_local_ap=0.893930 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=hgb_language_prior mass={'khoa_tieng_nhat': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nhat': 227, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 17, 'khoa_tieng_nga': 20, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 4} positive_ess=278.055779 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=hgb_random_donor rows=2588 failures=17 prevalence=0.006569 ap=0.319690 auc=0.924943 f2_at20=0.127986 brier=0.005391 ece10=0.000850 rho=0.250000 donor_rows=4096 effective_n=26056.972236 seconds=0.658625 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=hgb_random_donor in_sample_bce=0.015576 local_bce=0.011339 donor_bce=0.028288 in_sample_local_ap=0.887202 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=hgb_random_donor mass={'khoa_tieng_nhat': 0.75, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.25} source_failures={'khoa_tieng_nhat': 227, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 115} positive_ess=305.010234 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_common rows=2588 failures=17 prevalence=0.006569 ap=0.223512 auc=0.924703 f2_at20=0.127986 brier=0.005614 ece10=0.003068 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=0.628436 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_common in_sample_bce=0.007402 local_bce=0.007402 donor_bce=nan in_sample_local_ap=0.955067 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_native rows=2588 failures=17 prevalence=0.006569 ap=0.203819 auc=0.925092 f2_at20=0.127986 brier=0.005842 ece10=0.003302 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=1.220475 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_native in_sample_bce=0.006985 local_bce=0.006985 donor_bce=nan in_sample_local_ap=0.963361 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=native_cache_nan_indicator rows=2588 failures=17 prevalence=0.006569 ap=0.203819 auc=0.925092 f2_at20=0.127986 brier=0.005842 ece10=0.003302 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=2.341356 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=native_cache_nan_indicator in_sample_bce=0.006985 local_bce=0.006985 donor_bce=nan in_sample_local_ap=0.963361 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=native_cache_nan_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=native_naive_zero_indicator rows=2588 failures=17 prevalence=0.006569 ap=0.203819 auc=0.925092 f2_at20=0.127986 brier=0.005842 ece10=0.003302 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=2.338558 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=native_naive_zero_indicator in_sample_bce=0.006985 local_bce=0.006985 donor_bce=nan in_sample_local_ap=0.963361 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=native_naive_zero_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=native_placebo_indicator rows=2588 failures=17 prevalence=0.006569 ap=0.203819 auc=0.925092 f2_at20=0.127986 brier=0.005842 ece10=0.003302 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=2.376969 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=native_placebo_indicator in_sample_bce=0.006985 local_bce=0.006985 donor_bce=nan in_sample_local_ap=0.963361 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=native_placebo_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan rows=2588 failures=17 prevalence=0.006569 ap=0.203819 auc=0.925092 f2_at20=0.127986 brier=0.005842 ece10=0.003302 rho=0.000000 donor_rows=0 effective_n=24331.000000 seconds=2.202571 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan in_sample_bce=0.006985 local_bce=0.006985 donor_bce=nan in_sample_local_ap=0.963361 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 227} positive_ess=227.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_nhat model=committee_reference ap=0.377311 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_nhat models=13 positive_rows=17 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_phap fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_phap fold=3 origin=6063 ap=0.080351 rows=2758 failures=12 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.087330 ap=0.167680 base_ap=0.080351 donor_failures=173 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.125479 ap=0.205830 base_ap=0.080351 donor_failures=93 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6063 delta_ap=0.005580 ap=0.085930 base_ap=0.080351 donor_failures=30 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_anh origin=6063 delta_ap=0.264496 ap=0.344847 base_ap=0.080351 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=0.304796 ap=0.385146 base_ap=0.080351 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_duc origin=6063 delta_ap=0.066167 ap=0.146518 base_ap=0.080351 donor_failures=56 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.000066 ap=0.080285 base_ap=0.080351 donor_failures=30 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_italia origin=6063 delta_ap=-0.002621 ap=0.077729 base_ap=0.080351 donor_failures=144 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nga origin=6063 delta_ap=-0.011270 ap=0.069081 base_ap=0.080351 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6063 delta_ap=0.234588 ap=0.314939 base_ap=0.080351 donor_failures=58 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.224554 ap=0.304905 base_ap=0.080351 donor_failures=86 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.047007 ap=0.127358 base_ap=0.080351 donor_failures=33 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6063 delta_ap=0.147395 ap=0.227746 base_ap=0.080351 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_phap fold=3 origin=6064 ap=0.453203 rows=2471 failures=32 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.089743 ap=0.363460 base_ap=0.453203 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.061288 ap=0.391916 base_ap=0.453203 donor_failures=98 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.060562 ap=0.392641 base_ap=0.453203 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_anh origin=6064 delta_ap=-0.059716 ap=0.393487 base_ap=0.453203 donor_failures=70 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.013975 ap=0.467178 base_ap=0.453203 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_duc origin=6064 delta_ap=-0.048742 ap=0.404461 base_ap=0.453203 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6064 delta_ap=0.032196 ap=0.485399 base_ap=0.453203 donor_failures=13 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_italia origin=6064 delta_ap=-0.026555 ap=0.426648 base_ap=0.453203 donor_failures=112 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nga origin=6064 delta_ap=-0.028031 ap=0.425172 base_ap=0.453203 donor_failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6064 delta_ap=-0.009317 ap=0.443886 base_ap=0.453203 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.048705 ap=0.404498 base_ap=0.453203 donor_failures=85 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.071600 ap=0.381603 base_ap=0.453203 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.067054 ap=0.386149 base_ap=0.453203 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_phap fold=3 origin=6066 ap=0.252625 rows=2606 failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.031588 ap=0.284213 base_ap=0.252625 donor_failures=136 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.024648 ap=0.277274 base_ap=0.252625 donor_failures=109 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6066 delta_ap=0.027608 ap=0.280233 base_ap=0.252625 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_anh origin=6066 delta_ap=-0.014923 ap=0.237702 base_ap=0.252625 donor_failures=78 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.019002 ap=0.271627 base_ap=0.252625 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_duc origin=6066 delta_ap=0.028688 ap=0.281313 base_ap=0.252625 donor_failures=62 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6066 delta_ap=0.034447 ap=0.287072 base_ap=0.252625 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_italia origin=6066 delta_ap=0.032191 ap=0.284817 base_ap=0.252625 donor_failures=117 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nga origin=6066 delta_ap=0.039246 ap=0.291872 base_ap=0.252625 donor_failures=175 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6066 delta_ap=-0.014737 ap=0.237889 base_ap=0.252625 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.015057 ap=0.267683 base_ap=0.252625 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.044157 ap=0.296783 base_ap=0.252625 donor_failures=27 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6066 delta_ap=0.002786 ap=0.255411 base_ap=0.252625 donor_failures=250 [SCREEN_BASE] target=khoa_tieng_phap fold=3 origin=6067 ap=0.534354 rows=2204 failures=40 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.057677 ap=0.476676 base_ap=0.534354 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.021499 ap=0.555853 base_ap=0.534354 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6067 delta_ap=0.055412 ap=0.589765 base_ap=0.534354 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_anh origin=6067 delta_ap=-0.031832 ap=0.502522 base_ap=0.534354 donor_failures=68 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.106720 ap=0.427633 base_ap=0.534354 donor_failures=129 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_duc origin=6067 delta_ap=-0.009500 ap=0.524853 base_ap=0.534354 donor_failures=74 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.060901 ap=0.473452 base_ap=0.534354 donor_failures=19 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_italia origin=6067 delta_ap=0.033016 ap=0.567369 base_ap=0.534354 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nga origin=6067 delta_ap=-0.136328 ap=0.398026 base_ap=0.534354 donor_failures=194 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6067 delta_ap=-0.015455 ap=0.518899 base_ap=0.534354 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.042481 ap=0.576835 base_ap=0.534354 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.019758 ap=0.554111 base_ap=0.534354 donor_failures=26 [TRANSFER_EDGE] fold=3 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.070520 ap=0.463834 base_ap=0.534354 donor_failures=238 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_phap origin=6066 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_nhat,khoa_tieng_anh policy_donor_count=3 base_ap=0.252625 policy_ap=0.267560 delta_ap=0.014934 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_phap origin=6067 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_tay_ban_nha,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.534354 policy_ap=0.581133 delta_ap=0.046779 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_phap selected=['khoa_tieng_tay_ban_nha', 'khoa_tieng_bo_dao_nha', 'khoa_quan_tri_kinh_doanh_va_du_lich'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.030857 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=hgb_local_common rows=2693 failures=43 prevalence=0.015967 ap=0.119395 auc=0.829223 f2_at20=0.196906 brier=0.015911 ece10=0.007927 rho=0.000000 donor_rows=0 effective_n=20075.000000 seconds=0.619630 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=hgb_local_common in_sample_bce=0.011939 local_bce=0.011939 donor_bce=nan in_sample_local_ap=0.943659 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=hgb_local_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 260} positive_ess=260.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=hgb_local_native rows=2693 failures=43 prevalence=0.015967 ap=0.095496 auc=0.816972 f2_at20=0.210970 brier=0.016459 ece10=0.007776 rho=0.000000 donor_rows=0 effective_n=20075.000000 seconds=1.549440 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=hgb_local_native in_sample_bce=0.011774 local_bce=0.011774 donor_bce=nan in_sample_local_ap=0.947086 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=hgb_local_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 260} positive_ess=260.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=hgb_pool_all rows=2693 failures=43 prevalence=0.015967 ap=0.114523 auc=0.836955 f2_at20=0.232068 brier=0.015714 ece10=0.006272 rho=0.250000 donor_rows=4095 effective_n=19485.098328 seconds=0.667430 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=hgb_pool_all in_sample_bce=0.019193 local_bce=0.017026 donor_bce=0.047006 in_sample_local_ap=0.883404 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=hgb_pool_all mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.023351879135649296, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03686462856699752, 'khoa_quoc_te_hoc': 0.012032962417287013, 'khoa_tieng_anh': 0.05209141077336642, 'khoa_tieng_bo_dao_nha': 0.004524174162531017, 'khoa_tieng_duc': 0.011854450992555832, 'khoa_tieng_han_quoc': 0.018855168269230768, 'khoa_tieng_italia': 0.011986921009098428, 'khoa_tieng_nga': 0.007886004704301076, 'khoa_tieng_nhat': 0.019653219344499588, 'khoa_tieng_tay_ban_nha': 0.008617013027295286, 'khoa_tieng_trung_quoc': 0.036960750103391234, 'khoa_viet_nam_hoc': 0.005321417493796526} source_failures={'khoa_tieng_phap': 260, 'khoa_cong_nghe_thong_tin': 5, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 16, 'khoa_tieng_nga': 20, 'khoa_tieng_nhat': 4, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 18} positive_ess=280.133625 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=hgb_pool_equal rows=2693 failures=43 prevalence=0.015967 ap=0.103410 auc=0.834164 f2_at20=0.218003 brier=0.015271 ece10=0.005874 rho=0.250000 donor_rows=4095 effective_n=23104.057615 seconds=0.678442 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=hgb_pool_equal in_sample_bce=0.022624 local_bce=0.018221 donor_bce=0.035835 in_sample_local_ap=0.872768 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=hgb_pool_equal mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_phap': 260, 'khoa_cong_nghe_thong_tin': 5, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 16, 'khoa_tieng_nga': 20, 'khoa_tieng_nhat': 4, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 18} positive_ess=347.411254 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=hgb_language_prior rows=2693 failures=43 prevalence=0.015967 ap=0.148598 auc=0.848157 f2_at20=0.225035 brier=0.015883 ece10=0.010540 rho=0.250000 donor_rows=4095 effective_n=23104.057615 seconds=0.701525 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=hgb_language_prior in_sample_bce=0.018999 local_bce=0.016995 donor_bce=0.025011 in_sample_local_ap=0.887763 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=hgb_language_prior mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_phap': 260, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 24, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 6, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 4} positive_ess=332.979572 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=hgb_random_donor rows=2693 failures=43 prevalence=0.015967 ap=0.184018 auc=0.873883 f2_at20=0.239100 brier=0.014510 ece10=0.003280 rho=0.250000 donor_rows=4095 effective_n=23104.057615 seconds=0.637325 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=hgb_random_donor in_sample_bce=0.013361 local_bce=0.014958 donor_bce=0.008568 in_sample_local_ap=0.910650 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=hgb_random_donor mass={'khoa_tieng_phap': 0.75, np.str_('khoa_tieng_nhat'): 0.08333333333333333, np.str_('khoa_tieng_han_quoc'): 0.08333333333333333, np.str_('khoa_tieng_duc'): 0.08333333333333333} source_failures={'khoa_tieng_phap': 260, np.str_('khoa_tieng_nhat'): 8, np.str_('khoa_tieng_han_quoc'): 7, np.str_('khoa_tieng_duc'): 28} positive_ess=291.006662 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=suite32_selective_common rows=2693 failures=43 prevalence=0.015967 ap=0.214321 auc=0.845270 f2_at20=0.225035 brier=0.014648 ece10=0.007730 rho=0.250000 donor_rows=4095 effective_n=22393.730129 seconds=0.640606 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=suite32_selective_common in_sample_bce=0.021801 local_bce=0.017219 donor_bce=0.038706 in_sample_local_ap=0.883234 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=suite32_selective_common mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_tay_ban_nha': 0.10151192417987955, 'khoa_tieng_bo_dao_nha': 0.10049607016880198, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04799200565131851} source_failures={'khoa_tieng_phap': 260, 'khoa_tieng_tay_ban_nha': 26, 'khoa_tieng_bo_dao_nha': 52, 'khoa_quan_tri_kinh_doanh_va_du_lich': 40} positive_ess=340.094126 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=suite32_selective_native rows=2693 failures=43 prevalence=0.015967 ap=0.281621 auc=0.813861 f2_at20=0.196906 brier=0.014619 ece10=0.010393 rho=0.250000 donor_rows=4095 effective_n=22393.730129 seconds=1.800423 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=suite32_selective_native in_sample_bce=0.021682 local_bce=0.017014 donor_bce=0.039328 in_sample_local_ap=0.884819 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=suite32_selective_native mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_tay_ban_nha': 0.10151192417987955, 'khoa_tieng_bo_dao_nha': 0.10049607016880198, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04799200565131851} source_failures={'khoa_tieng_phap': 260, 'khoa_tieng_tay_ban_nha': 26, 'khoa_tieng_bo_dao_nha': 52, 'khoa_quan_tri_kinh_doanh_va_du_lich': 40} positive_ess=340.094126 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=native_cache_nan_indicator rows=2693 failures=43 prevalence=0.015967 ap=0.095496 auc=0.816972 f2_at20=0.210970 brier=0.016459 ece10=0.007776 rho=0.000000 donor_rows=0 effective_n=20075.000000 seconds=3.502045 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=native_cache_nan_indicator in_sample_bce=0.011774 local_bce=0.011774 donor_bce=nan in_sample_local_ap=0.947086 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=native_cache_nan_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 260} positive_ess=260.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=native_naive_zero_indicator rows=2693 failures=43 prevalence=0.015967 ap=0.095496 auc=0.816972 f2_at20=0.210970 brier=0.016459 ece10=0.007776 rho=0.000000 donor_rows=0 effective_n=20075.000000 seconds=3.377559 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=native_naive_zero_indicator in_sample_bce=0.011774 local_bce=0.011774 donor_bce=nan in_sample_local_ap=0.947086 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=native_naive_zero_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 260} positive_ess=260.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=native_placebo_indicator rows=2693 failures=43 prevalence=0.015967 ap=0.095496 auc=0.816972 f2_at20=0.210970 brier=0.016459 ece10=0.007776 rho=0.000000 donor_rows=0 effective_n=20075.000000 seconds=3.555237 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=native_placebo_indicator in_sample_bce=0.011774 local_bce=0.011774 donor_bce=nan in_sample_local_ap=0.947086 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=native_placebo_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 260} positive_ess=260.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan rows=2693 failures=43 prevalence=0.015967 ap=0.281621 auc=0.813861 f2_at20=0.196906 brier=0.014619 ece10=0.010393 rho=0.250000 donor_rows=4095 effective_n=22393.730129 seconds=4.101341 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan in_sample_bce=0.021682 local_bce=0.017014 donor_bce=0.039328 in_sample_local_ap=0.884819 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_tay_ban_nha': 0.10151192417987955, 'khoa_tieng_bo_dao_nha': 0.10049607016880198, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04799200565131851} source_failures={'khoa_tieng_phap': 260, 'khoa_tieng_tay_ban_nha': 26, 'khoa_tieng_bo_dao_nha': 52, 'khoa_quan_tri_kinh_doanh_va_du_lich': 40} positive_ess=340.094126 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_phap model=committee_reference ap=0.101848 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_phap models=13 positive_rows=43 num_edges=52 num_replays=2 chosen=['khoa_tieng_tay_ban_nha', 'khoa_tieng_bo_dao_nha', 'khoa_quan_tri_kinh_doanh_va_du_lich'] [SELECTION_ORIGINS] target=khoa_tieng_tay_ban_nha fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=3 origin=6063 ap=0.379429 rows=1560 failures=18 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.095393 ap=0.474822 base_ap=0.379429 donor_failures=142 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.062376 ap=0.441805 base_ap=0.379429 donor_failures=98 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6063 delta_ap=0.038941 ap=0.418369 base_ap=0.379429 donor_failures=31 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6063 delta_ap=0.036468 ap=0.415897 base_ap=0.379429 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.072353 ap=0.307076 base_ap=0.379429 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6063 delta_ap=-0.201998 ap=0.177430 base_ap=0.379429 donor_failures=70 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6063 delta_ap=0.004441 ap=0.383869 base_ap=0.379429 donor_failures=22 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6063 delta_ap=0.103473 ap=0.482902 base_ap=0.379429 donor_failures=153 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6063 delta_ap=-0.008221 ap=0.371207 base_ap=0.379429 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6063 delta_ap=0.009514 ap=0.388943 base_ap=0.379429 donor_failures=52 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6063 delta_ap=-0.067359 ap=0.312070 base_ap=0.379429 donor_failures=57 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.049317 ap=0.428746 base_ap=0.379429 donor_failures=27 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6063 delta_ap=-0.049309 ap=0.330119 base_ap=0.379429 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=3 origin=6064 ap=0.314265 rows=1388 failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.014530 ap=0.328796 base_ap=0.314265 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.050752 ap=0.365017 base_ap=0.314265 donor_failures=87 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=0.014440 ap=0.328706 base_ap=0.314265 donor_failures=31 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6064 delta_ap=0.046908 ap=0.361173 base_ap=0.314265 donor_failures=68 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.014368 ap=0.328634 base_ap=0.314265 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6064 delta_ap=-0.003246 ap=0.311019 base_ap=0.314265 donor_failures=52 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=0.065352 ap=0.379617 base_ap=0.314265 donor_failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6064 delta_ap=-0.025316 ap=0.288949 base_ap=0.314265 donor_failures=136 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6064 delta_ap=0.040284 ap=0.354549 base_ap=0.314265 donor_failures=151 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6064 delta_ap=0.039221 ap=0.353487 base_ap=0.314265 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6064 delta_ap=0.022175 ap=0.336441 base_ap=0.314265 donor_failures=55 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.036185 ap=0.350450 base_ap=0.314265 donor_failures=27 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.045966 ap=0.360232 base_ap=0.314265 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=3 origin=6066 ap=0.241542 rows=1401 failures=63 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.000281 ap=0.241261 base_ap=0.241542 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.000638 ap=0.242180 base_ap=0.241542 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.028049 ap=0.269591 base_ap=0.241542 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.005694 ap=0.235848 base_ap=0.241542 donor_failures=77 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.031542 ap=0.210000 base_ap=0.241542 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.072995 ap=0.168547 base_ap=0.241542 donor_failures=64 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.016297 ap=0.225246 base_ap=0.241542 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6066 delta_ap=-0.034853 ap=0.206690 base_ap=0.241542 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6066 delta_ap=-0.054393 ap=0.187149 base_ap=0.241542 donor_failures=162 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6066 delta_ap=-0.000489 ap=0.241054 base_ap=0.241542 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6066 delta_ap=0.011202 ap=0.252744 base_ap=0.241542 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.002829 ap=0.244371 base_ap=0.241542 donor_failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=0.006171 ap=0.247713 base_ap=0.241542 donor_failures=283 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=3 origin=6067 ap=0.373007 rows=1242 failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.009190 ap=0.382197 base_ap=0.373007 donor_failures=120 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.009948 ap=0.363059 base_ap=0.373007 donor_failures=117 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6067 delta_ap=0.012892 ap=0.385899 base_ap=0.373007 donor_failures=48 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6067 delta_ap=-0.011640 ap=0.361367 base_ap=0.373007 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.014736 ap=0.387743 base_ap=0.373007 donor_failures=129 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6067 delta_ap=0.054750 ap=0.427757 base_ap=0.373007 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6067 delta_ap=0.032735 ap=0.405742 base_ap=0.373007 donor_failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6067 delta_ap=-0.030898 ap=0.342109 base_ap=0.373007 donor_failures=148 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6067 delta_ap=-0.009385 ap=0.363622 base_ap=0.373007 donor_failures=180 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6067 delta_ap=0.080599 ap=0.453606 base_ap=0.373007 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6067 delta_ap=0.010065 ap=0.383072 base_ap=0.373007 donor_failures=38 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.014527 ap=0.358480 base_ap=0.373007 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6067 delta_ap=0.060595 ap=0.433602 base_ap=0.373007 donor_failures=253 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_tay_ban_nha origin=6066 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_cong_nghe_thong_tin,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.241542 policy_ap=0.226532 delta_ap=-0.015010 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_tay_ban_nha origin=6067 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_cong_nghe_thong_tin,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.373007 policy_ap=0.401070 delta_ap=0.028063 policy_donor_rows=4095 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_tay_ban_nha selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.006526 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common rows=1417 failures=54 prevalence=0.038109 ap=0.312679 auc=0.922849 f2_at20=0.490000 brier=0.033347 ece10=0.027233 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=0.405710 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common in_sample_bce=0.022093 local_bce=0.022093 donor_bce=nan in_sample_local_ap=0.956587 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native rows=1417 failures=54 prevalence=0.038109 ap=0.339948 auc=0.916572 f2_at20=0.470000 brier=0.032625 ece10=0.023899 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=0.897503 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native in_sample_bce=0.021696 local_bce=0.021696 donor_bce=nan in_sample_local_ap=0.957633 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all rows=1417 failures=54 prevalence=0.038109 ap=0.373178 auc=0.926177 f2_at20=0.490000 brier=0.031253 ece10=0.019628 rho=0.250000 donor_rows=4095 effective_n=13290.989127 seconds=0.536796 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all in_sample_bce=0.029926 local_bce=0.028476 donor_bce=0.058377 in_sample_local_ap=0.921077 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.02266306273537131, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0357772231124044, 'khoa_quoc_te_hoc': 0.011678023022097073, 'khoa_tieng_anh': 0.05055485699772037, 'khoa_tieng_bo_dao_nha': 0.004390723430675016, 'khoa_tieng_duc': 0.011504777194891364, 'khoa_tieng_han_quoc': 0.018298992508881787, 'khoa_tieng_italia': 0.011633339709197863, 'khoa_tieng_nga': 0.007653389190087517, 'khoa_tieng_nhat': 0.01907350326580143, 'khoa_tieng_phap': 0.01573714923599374, 'khoa_tieng_trung_quoc': 0.03587050932705363, 'khoa_viet_nam_hoc': 0.005164450269824496} source_failures={'khoa_tieng_tay_ban_nha': 275, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 3, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 23} positive_ess=333.323430 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal rows=1417 failures=54 prevalence=0.038109 ap=0.398818 auc=0.932971 f2_at20=0.480000 brier=0.031379 ece10=0.018035 rho=0.250000 donor_rows=4095 effective_n=14707.980557 seconds=0.568071 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal in_sample_bce=0.033785 local_bce=0.029613 donor_bce=0.046299 in_sample_local_ap=0.914692 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_tay_ban_nha': 275, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 3, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 23} positive_ess=381.556425 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior rows=1417 failures=54 prevalence=0.038109 ap=0.292666 auc=0.922359 f2_at20=0.480000 brier=0.032048 ece10=0.018934 rho=0.250000 donor_rows=4095 effective_n=14707.980557 seconds=0.481298 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior in_sample_bce=0.028940 local_bce=0.028083 donor_bce=0.031511 in_sample_local_ap=0.921618 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_tay_ban_nha': 275, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 19, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_trung_quoc': 4} positive_ess=351.886764 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor rows=1417 failures=54 prevalence=0.038109 ap=0.508372 auc=0.920858 f2_at20=0.490000 brier=0.029628 ece10=0.018062 rho=0.250000 donor_rows=4096 effective_n=14708.786670 seconds=0.456643 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor in_sample_bce=0.043751 local_bce=0.031772 donor_bce=0.079690 in_sample_local_ap=0.900381 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor mass={'khoa_tieng_tay_ban_nha': 0.75, np.str_('khoa_viet_nam_hoc'): 0.25} source_failures={'khoa_tieng_tay_ban_nha': 275, np.str_('khoa_viet_nam_hoc'): 245} positive_ess=517.452402 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common rows=1417 failures=54 prevalence=0.038109 ap=0.312679 auc=0.922849 f2_at20=0.490000 brier=0.033347 ece10=0.027233 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=0.409146 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common in_sample_bce=0.022093 local_bce=0.022093 donor_bce=nan in_sample_local_ap=0.956587 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native rows=1417 failures=54 prevalence=0.038109 ap=0.339948 auc=0.916572 f2_at20=0.470000 brier=0.032625 ece10=0.023899 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=0.877534 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native in_sample_bce=0.021696 local_bce=0.021696 donor_bce=nan in_sample_local_ap=0.957633 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator rows=1417 failures=54 prevalence=0.038109 ap=0.339948 auc=0.916572 f2_at20=0.470000 brier=0.032625 ece10=0.023899 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=1.593281 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator in_sample_bce=0.021696 local_bce=0.021696 donor_bce=nan in_sample_local_ap=0.957633 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator rows=1417 failures=54 prevalence=0.038109 ap=0.332276 auc=0.922237 f2_at20=0.470000 brier=0.033018 ece10=0.024555 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=1.572268 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator in_sample_bce=0.021690 local_bce=0.021690 donor_bce=nan in_sample_local_ap=0.957814 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator rows=1417 failures=54 prevalence=0.038109 ap=0.339948 auc=0.916572 f2_at20=0.470000 brier=0.032625 ece10=0.023899 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=1.536706 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator in_sample_bce=0.021696 local_bce=0.021696 donor_bce=nan in_sample_local_ap=0.957633 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan rows=1417 failures=54 prevalence=0.038109 ap=0.339948 auc=0.916572 f2_at20=0.470000 brier=0.032625 ece10=0.023899 rho=0.000000 donor_rows=0 effective_n=10668.000000 seconds=1.544134 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan in_sample_bce=0.021696 local_bce=0.021696 donor_bce=nan in_sample_local_ap=0.957633 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 275} positive_ess=275.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_tay_ban_nha model=committee_reference ap=0.440128 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_tay_ban_nha models=13 positive_rows=54 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_trung_quoc fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=3 origin=6063 ap=0.037228 rows=7413 failures=13 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=0.072045 ap=0.109273 base_ap=0.037228 donor_failures=168 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=0.047872 ap=0.085100 base_ap=0.037228 donor_failures=84 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6063 delta_ap=0.061709 ap=0.098936 base_ap=0.037228 donor_failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6063 delta_ap=0.136813 ap=0.174040 base_ap=0.037228 donor_failures=70 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.006550 ap=0.030678 base_ap=0.037228 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6063 delta_ap=0.062504 ap=0.099731 base_ap=0.037228 donor_failures=67 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6063 delta_ap=0.110831 ap=0.148059 base_ap=0.037228 donor_failures=17 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6063 delta_ap=0.039281 ap=0.076509 base_ap=0.037228 donor_failures=137 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6063 delta_ap=0.063979 ap=0.101206 base_ap=0.037228 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6063 delta_ap=-0.004314 ap=0.032914 base_ap=0.037228 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6063 delta_ap=0.073032 ap=0.110259 base_ap=0.037228 donor_failures=46 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=0.031815 ap=0.069043 base_ap=0.037228 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6063 delta_ap=0.088077 ap=0.125305 base_ap=0.037228 donor_failures=217 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=3 origin=6064 ap=0.253471 rows=5347 failures=29 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.089033 ap=0.164438 base_ap=0.253471 donor_failures=135 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.049758 ap=0.203713 base_ap=0.253471 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.062935 ap=0.190536 base_ap=0.253471 donor_failures=36 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6064 delta_ap=-0.005600 ap=0.247871 base_ap=0.253471 donor_failures=74 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.080471 ap=0.173001 base_ap=0.253471 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6064 delta_ap=-0.083219 ap=0.170252 base_ap=0.253471 donor_failures=52 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6064 delta_ap=0.018358 ap=0.271829 base_ap=0.253471 donor_failures=16 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6064 delta_ap=0.015796 ap=0.269267 base_ap=0.253471 donor_failures=118 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.037827 ap=0.215644 base_ap=0.253471 donor_failures=150 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.095291 ap=0.348762 base_ap=0.253471 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.064232 ap=0.189240 base_ap=0.253471 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.031049 ap=0.284520 base_ap=0.253471 donor_failures=83 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.064510 ap=0.188961 base_ap=0.253471 donor_failures=270 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=3 origin=6066 ap=0.246158 rows=7393 failures=35 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.125801 ap=0.120357 base_ap=0.246158 donor_failures=133 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.155464 ap=0.090694 base_ap=0.246158 donor_failures=119 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.049903 ap=0.196255 base_ap=0.246158 donor_failures=33 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6066 delta_ap=-0.167608 ap=0.078550 base_ap=0.246158 donor_failures=58 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.085225 ap=0.160932 base_ap=0.246158 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6066 delta_ap=-0.060748 ap=0.185409 base_ap=0.246158 donor_failures=66 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.140210 ap=0.105947 base_ap=0.246158 donor_failures=12 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6066 delta_ap=0.080369 ap=0.326527 base_ap=0.246158 donor_failures=125 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6066 delta_ap=-0.190140 ap=0.056018 base_ap=0.246158 donor_failures=171 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.176176 ap=0.069982 base_ap=0.246158 donor_failures=41 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6066 delta_ap=-0.115201 ap=0.130957 base_ap=0.246158 donor_failures=43 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.172281 ap=0.073877 base_ap=0.246158 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.068793 ap=0.177364 base_ap=0.246158 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=3 origin=6067 ap=0.099941 rows=5584 failures=76 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.022439 ap=0.122380 base_ap=0.099941 donor_failures=127 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.043196 ap=0.143136 base_ap=0.099941 donor_failures=112 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6067 delta_ap=0.011286 ap=0.111226 base_ap=0.099941 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6067 delta_ap=0.087186 ap=0.187126 base_ap=0.099941 donor_failures=83 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.082129 ap=0.182070 base_ap=0.099941 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6067 delta_ap=0.056414 ap=0.156354 base_ap=0.099941 donor_failures=64 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.014513 ap=0.085427 base_ap=0.099941 donor_failures=24 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6067 delta_ap=-0.031929 ap=0.068012 base_ap=0.099941 donor_failures=159 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6067 delta_ap=-0.007554 ap=0.092387 base_ap=0.099941 donor_failures=183 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6067 delta_ap=0.015043 ap=0.114984 base_ap=0.099941 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6067 delta_ap=0.031868 ap=0.131808 base_ap=0.099941 donor_failures=53 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.020248 ap=0.120189 base_ap=0.099941 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.013433 ap=0.086508 base_ap=0.099941 donor_failures=254 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_trung_quoc origin=6066 policy_sources=khoa_tieng_anh,khoa_tieng_han_quoc,khoa_tieng_nhat policy_donor_count=3 base_ap=0.246158 policy_ap=0.087338 delta_ap=-0.158820 policy_donor_rows=4095 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_tieng_trung_quoc origin=6067 policy_sources=khoa_tieng_italia policy_donor_count=1 base_ap=0.099941 policy_ap=0.068012 delta_ap=-0.031929 policy_donor_rows=4096 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_tieng_trung_quoc selected=[] reason=prequential_policy_delta_-0.095374_below_threshold replay_mean=-0.095374 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_local_common rows=6920 failures=103 prevalence=0.014884 ap=0.139252 auc=0.904858 f2_at20=0.247773 brier=0.013966 ece10=0.008789 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=0.843594 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_local_common in_sample_bce=0.008993 local_bce=0.008993 donor_bce=nan in_sample_local_ap=0.881085 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_local_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_local_native rows=6920 failures=103 prevalence=0.014884 ap=0.110304 auc=0.902643 f2_at20=0.228285 brier=0.014472 ece10=0.008564 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=2.121617 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_local_native in_sample_bce=0.009016 local_bce=0.009016 donor_bce=nan in_sample_local_ap=0.882025 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_local_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_pool_all rows=6920 failures=103 prevalence=0.014884 ap=0.145849 auc=0.922106 f2_at20=0.253341 brier=0.013993 ece10=0.008625 rho=0.250000 donor_rows=4095 effective_n=27903.656770 seconds=0.848023 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_pool_all in_sample_bce=0.016825 local_bce=0.013964 donor_bce=0.036913 in_sample_local_ap=0.738415 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_pool_all mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02546499378129173, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.040200513704059956, 'khoa_quoc_te_hoc': 0.013121826785192076, 'khoa_tieng_anh': 0.05680516945539618, 'khoa_tieng_bo_dao_nha': 0.004933567283604807, 'khoa_tieng_duc': 0.01292716183791897, 'khoa_tieng_han_quoc': 0.020561374951113553, 'khoa_tieng_italia': 0.013071619083859193, 'khoa_tieng_nga': 0.008599610317770708, 'khoa_tieng_nhat': 0.021431641774216848, 'khoa_tieng_phap': 0.017682800074694967, 'khoa_tieng_tay_ban_nha': 0.009396767681038401, 'khoa_viet_nam_hoc': 0.005802953269842612} source_failures={'khoa_tieng_trung_quoc': 296, 'khoa_cong_nghe_thong_tin': 16, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 4, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 9, 'khoa_viet_nam_hoc': 13} positive_ess=176.008294 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal rows=6920 failures=103 prevalence=0.014884 ap=0.137053 auc=0.917053 f2_at20=0.258909 brier=0.014887 ece10=0.010510 rho=0.250000 donor_rows=4095 effective_n=36290.464697 seconds=0.875067 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal in_sample_bce=0.017803 local_bce=0.014046 donor_bce=0.029074 in_sample_local_ap=0.734900 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_trung_quoc': 296, 'khoa_cong_nghe_thong_tin': 16, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 4, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 9, 'khoa_viet_nam_hoc': 13} positive_ess=274.140472 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_language_prior rows=6920 failures=103 prevalence=0.014884 ap=0.130947 auc=0.909340 f2_at20=0.244989 brier=0.014312 ece10=0.006933 rho=0.250000 donor_rows=4095 effective_n=36290.464697 seconds=0.867837 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_language_prior in_sample_bce=0.017749 local_bce=0.014227 donor_bce=0.028314 in_sample_local_ap=0.721619 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_language_prior mass={'khoa_tieng_trung_quoc': 0.7500000000000001, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776} source_failures={'khoa_tieng_trung_quoc': 296, 'khoa_tieng_anh': 12, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 16, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 20, 'khoa_tieng_nga': 27, 'khoa_tieng_nhat': 8, 'khoa_tieng_phap': 8, 'khoa_tieng_tay_ban_nha': 11} positive_ess=280.684680 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_random_donor rows=6920 failures=103 prevalence=0.014884 ap=0.091498 auc=0.902359 f2_at20=0.253341 brier=0.014988 ece10=0.008789 rho=0.250000 donor_rows=4096 effective_n=36295.372758 seconds=0.916503 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_random_donor in_sample_bce=0.008606 local_bce=0.010543 donor_bce=0.002792 in_sample_local_ap=0.836428 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=hgb_random_donor mass={'khoa_tieng_trung_quoc': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_tieng_trung_quoc': 296, np.str_('khoa_tieng_han_quoc'): 23} positive_ess=236.862639 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_common rows=6920 failures=103 prevalence=0.014884 ap=0.139252 auc=0.904858 f2_at20=0.247773 brier=0.013966 ece10=0.008789 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=0.864810 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_common in_sample_bce=0.008993 local_bce=0.008993 donor_bce=nan in_sample_local_ap=0.881085 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_native rows=6920 failures=103 prevalence=0.014884 ap=0.110304 auc=0.902643 f2_at20=0.228285 brier=0.014472 ece10=0.008564 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=2.058712 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_native in_sample_bce=0.009016 local_bce=0.009016 donor_bce=nan in_sample_local_ap=0.882025 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator rows=6920 failures=103 prevalence=0.014884 ap=0.110304 auc=0.902643 f2_at20=0.228285 brier=0.014472 ece10=0.008564 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=3.902262 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator in_sample_bce=0.009016 local_bce=0.009016 donor_bce=nan in_sample_local_ap=0.882025 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator rows=6920 failures=103 prevalence=0.014884 ap=0.110304 auc=0.902643 f2_at20=0.228285 brier=0.014472 ece10=0.008564 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=3.895774 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator in_sample_bce=0.009016 local_bce=0.009016 donor_bce=nan in_sample_local_ap=0.882025 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator rows=6920 failures=103 prevalence=0.014884 ap=0.110304 auc=0.902643 f2_at20=0.228285 brier=0.014472 ece10=0.008564 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=3.870952 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator in_sample_bce=0.009016 local_bce=0.009016 donor_bce=nan in_sample_local_ap=0.882025 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan rows=6920 failures=103 prevalence=0.014884 ap=0.110304 auc=0.902643 f2_at20=0.228285 brier=0.014472 ece10=0.008564 rho=0.000000 donor_rows=0 effective_n=45758.000000 seconds=3.960437 [TRAINING_INSIGHT] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.009016 local_bce=0.009016 donor_bce=nan in_sample_local_ap=0.882025 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 296} positive_ess=296.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_tieng_trung_quoc model=committee_reference ap=0.120361 [FACULTY_COMPLETE] fold=3 faculty=khoa_tieng_trung_quoc models=13 positive_rows=103 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_viet_nam_hoc fold=3 origins=[6063, 6064, 6066, 6067] availability=semester_proxy [SCREEN_BASE] target=khoa_viet_nam_hoc fold=3 origin=6063 ap=0.532428 rows=993 failures=71 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6063 delta_ap=-0.020981 ap=0.511447 base_ap=0.532428 donor_failures=170 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6063 delta_ap=-0.109203 ap=0.423224 base_ap=0.532428 donor_failures=90 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6063 delta_ap=-0.088774 ap=0.443653 base_ap=0.532428 donor_failures=32 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6063 delta_ap=-0.024484 ap=0.507943 base_ap=0.532428 donor_failures=65 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6063 delta_ap=-0.101499 ap=0.430929 base_ap=0.532428 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6063 delta_ap=-0.017253 ap=0.515175 base_ap=0.532428 donor_failures=63 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6063 delta_ap=-0.034522 ap=0.497905 base_ap=0.532428 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6063 delta_ap=-0.090992 ap=0.441436 base_ap=0.532428 donor_failures=149 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6063 delta_ap=-0.041682 ap=0.490746 base_ap=0.532428 donor_failures=172 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6063 delta_ap=-0.078420 ap=0.454008 base_ap=0.532428 donor_failures=50 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6063 delta_ap=-0.123285 ap=0.409142 base_ap=0.532428 donor_failures=54 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6063 delta_ap=-0.028624 ap=0.503804 base_ap=0.532428 donor_failures=87 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6063 delta_ap=0.008438 ap=0.540866 base_ap=0.532428 donor_failures=32 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=3 origin=6064 ap=0.556851 rows=818 failures=49 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.003189 ap=0.553662 base_ap=0.556851 donor_failures=107 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.003118 ap=0.553732 base_ap=0.556851 donor_failures=96 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6064 delta_ap=0.007794 ap=0.564645 base_ap=0.556851 donor_failures=30 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6064 delta_ap=-0.011847 ap=0.545004 base_ap=0.556851 donor_failures=79 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.050816 ap=0.607667 base_ap=0.556851 donor_failures=97 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6064 delta_ap=-0.017238 ap=0.539613 base_ap=0.556851 donor_failures=54 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.005079 ap=0.551772 base_ap=0.556851 donor_failures=24 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6064 delta_ap=0.006959 ap=0.563810 base_ap=0.556851 donor_failures=122 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6064 delta_ap=-0.017295 ap=0.539555 base_ap=0.556851 donor_failures=152 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6064 delta_ap=-0.003221 ap=0.553630 base_ap=0.556851 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6064 delta_ap=0.002860 ap=0.559711 base_ap=0.556851 donor_failures=44 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.007610 ap=0.549241 base_ap=0.556851 donor_failures=81 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.012580 ap=0.544271 base_ap=0.556851 donor_failures=15 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=3 origin=6066 ap=0.183767 rows=813 failures=30 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.019319 ap=0.203086 base_ap=0.183767 donor_failures=131 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.003041 ap=0.186808 base_ap=0.183767 donor_failures=104 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.010757 ap=0.173010 base_ap=0.183767 donor_failures=37 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6066 delta_ap=0.003058 ap=0.186825 base_ap=0.183767 donor_failures=61 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.020007 ap=0.163759 base_ap=0.183767 donor_failures=123 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.028051 ap=0.211818 base_ap=0.183767 donor_failures=74 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.018831 ap=0.164936 base_ap=0.183767 donor_failures=23 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6066 delta_ap=-0.057377 ap=0.126390 base_ap=0.183767 donor_failures=113 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6066 delta_ap=-0.037094 ap=0.146673 base_ap=0.183767 donor_failures=167 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6066 delta_ap=0.022827 ap=0.206593 base_ap=0.183767 donor_failures=39 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6066 delta_ap=-0.037638 ap=0.146129 base_ap=0.183767 donor_failures=58 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.052147 ap=0.131620 base_ap=0.183767 donor_failures=91 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.020551 ap=0.204318 base_ap=0.183767 donor_failures=16 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=3 origin=6067 ap=0.421828 rows=556 failures=39 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.089549 ap=0.332278 base_ap=0.421828 donor_failures=132 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.015166 ap=0.436994 base_ap=0.421828 donor_failures=113 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.054559 ap=0.367269 base_ap=0.421828 donor_failures=47 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6067 delta_ap=0.019236 ap=0.441064 base_ap=0.421828 donor_failures=53 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.051334 ap=0.370494 base_ap=0.421828 donor_failures=130 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6067 delta_ap=-0.023149 ap=0.398679 base_ap=0.421828 donor_failures=82 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.010246 ap=0.411581 base_ap=0.421828 donor_failures=21 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6067 delta_ap=-0.085952 ap=0.335876 base_ap=0.421828 donor_failures=152 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6067 delta_ap=-0.038055 ap=0.383773 base_ap=0.421828 donor_failures=200 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.045034 ap=0.376794 base_ap=0.421828 donor_failures=42 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6067 delta_ap=-0.028446 ap=0.393382 base_ap=0.421828 donor_failures=51 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.051169 ap=0.370659 base_ap=0.421828 donor_failures=108 [TRANSFER_EDGE] fold=3 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.000739 ap=0.421088 base_ap=0.421828 donor_failures=23 [POLICY_REPLAY] fold=3 faculty=khoa_viet_nam_hoc origin=6066 policy_sources= policy_donor_count=0 base_ap=0.183767 policy_ap=0.183767 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6063,6064 [POLICY_REPLAY] fold=3 faculty=khoa_viet_nam_hoc origin=6067 policy_sources=khoa_tieng_trung_quoc policy_donor_count=1 base_ap=0.421828 policy_ap=0.421088 delta_ap=-0.000739 policy_donor_rows=4096 selected_using_origins=6063,6064,6066 [SELECTION_FINAL] fold=3 faculty=khoa_viet_nam_hoc selected=[] reason=prequential_policy_delta_-0.000370_below_threshold replay_mean=-0.000370 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_local_common rows=773 failures=33 prevalence=0.042691 ap=0.412899 auc=0.875655 f2_at20=0.435540 brier=0.033490 ece10=0.023049 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.338123 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=hgb_local_common in_sample_bce=0.058608 local_bce=0.058608 donor_bce=nan in_sample_local_ap=0.926390 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_local_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_local_native rows=773 failures=33 prevalence=0.042691 ap=0.441784 auc=0.894595 f2_at20=0.470383 brier=0.031792 ece10=0.022232 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.398495 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=hgb_local_native in_sample_bce=0.058772 local_bce=0.058772 donor_bce=nan in_sample_local_ap=0.926656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_local_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_pool_all rows=773 failures=33 prevalence=0.042691 ap=0.428330 auc=0.902969 f2_at20=0.435540 brier=0.031578 ece10=0.011785 rho=0.250000 donor_rows=4095 effective_n=9305.576009 seconds=0.507852 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=hgb_pool_all in_sample_bce=0.061794 local_bce=0.066564 donor_bce=0.060063 in_sample_local_ap=0.903966 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_pool_all mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02237678449244716, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.035325287701514906, 'khoa_quoc_te_hoc': 0.011530507041991883, 'khoa_tieng_anh': 0.04991625153642052, 'khoa_tieng_bo_dao_nha': 0.004335260115606936, 'khoa_tieng_duc': 0.011359449644107731, 'khoa_tieng_han_quoc': 0.01806784089959163, 'khoa_tieng_italia': 0.011486388165614522, 'khoa_tieng_nga': 0.007556712106529284, 'khoa_tieng_nhat': 0.01883256809013254, 'khoa_tieng_phap': 0.015538358653956303, 'khoa_tieng_tay_ban_nha': 0.00825719602094176, 'khoa_tieng_trung_quoc': 0.035417395531144834} source_failures={'khoa_viet_nam_hoc': 406, 'khoa_cong_nghe_thong_tin': 17, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 8, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 22, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1} positive_ess=464.910338 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_pool_equal rows=773 failures=33 prevalence=0.042691 ap=0.444998 auc=0.913186 f2_at20=0.435540 brier=0.032104 ece10=0.010721 rho=0.250000 donor_rows=4095 effective_n=9935.910503 seconds=0.511576 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=hgb_pool_equal in_sample_bce=0.064093 local_bce=0.066937 donor_bce=0.055561 in_sample_local_ap=0.901846 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_pool_equal mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232} source_failures={'khoa_viet_nam_hoc': 406, 'khoa_cong_nghe_thong_tin': 17, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 8, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 22, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1} positive_ess=482.669525 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_language_prior rows=773 failures=33 prevalence=0.042691 ap=0.412899 auc=0.875655 f2_at20=0.435540 brier=0.033490 ece10=0.023049 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.400192 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=hgb_language_prior in_sample_bce=0.058608 local_bce=0.058608 donor_bce=nan in_sample_local_ap=0.926390 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_language_prior mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_random_donor rows=773 failures=33 prevalence=0.042691 ap=0.387246 auc=0.899242 f2_at20=0.470383 brier=0.032576 ece10=0.010323 rho=0.250000 donor_rows=4096 effective_n=9936.278376 seconds=0.463109 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=hgb_random_donor in_sample_bce=0.051652 local_bce=0.061878 donor_bce=0.020972 in_sample_local_ap=0.914310 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=hgb_random_donor mass={'khoa_viet_nam_hoc': 0.75, np.str_('khoa_tieng_nhat'): 0.25} source_failures={'khoa_viet_nam_hoc': 406, np.str_('khoa_tieng_nhat'): 40} positive_ess=437.630116 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_common rows=773 failures=33 prevalence=0.042691 ap=0.412899 auc=0.875655 f2_at20=0.435540 brier=0.033490 ece10=0.023049 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.380988 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_common in_sample_bce=0.058608 local_bce=0.058608 donor_bce=nan in_sample_local_ap=0.926390 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_native rows=773 failures=33 prevalence=0.042691 ap=0.441784 auc=0.894595 f2_at20=0.470383 brier=0.031792 ece10=0.022232 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.437951 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_native in_sample_bce=0.058772 local_bce=0.058772 donor_bce=nan in_sample_local_ap=0.926656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator rows=773 failures=33 prevalence=0.042691 ap=0.441784 auc=0.894595 f2_at20=0.470383 brier=0.031792 ece10=0.022232 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.737074 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator in_sample_bce=0.058772 local_bce=0.058772 donor_bce=nan in_sample_local_ap=0.926656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator rows=773 failures=33 prevalence=0.042691 ap=0.441784 auc=0.894595 f2_at20=0.470383 brier=0.031792 ece10=0.022232 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.753818 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator in_sample_bce=0.058772 local_bce=0.058772 donor_bce=nan in_sample_local_ap=0.926656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=native_placebo_indicator rows=773 failures=33 prevalence=0.042691 ap=0.441784 auc=0.894595 f2_at20=0.470383 brier=0.031792 ece10=0.022232 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.701448 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=native_placebo_indicator in_sample_bce=0.058772 local_bce=0.058772 donor_bce=nan in_sample_local_ap=0.926656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=native_placebo_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan rows=773 failures=33 prevalence=0.042691 ap=0.441784 auc=0.894595 f2_at20=0.470383 brier=0.031792 ece10=0.022232 rho=0.000000 donor_rows=0 effective_n=6588.000000 seconds=0.699664 [TRAINING_INSIGHT] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.058772 local_bce=0.058772 donor_bce=nan in_sample_local_ap=0.926656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=3 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 406} positive_ess=406.000000 [FACULTY_RESULTS] fold=3 faculty=khoa_viet_nam_hoc model=committee_reference ap=0.488667 [FACULTY_COMPLETE] fold=3 faculty=khoa_viet_nam_hoc models=13 positive_rows=33 num_edges=52 num_replays=2 chosen=[] [FOLD_START] fold=4 stage=development [DATA] fold=4 common_raw=136 course_features=17 common_total=153 faculties=14 history_rows=376999 target_rows=41376 target_failures=1100 history_last=6069 target_first=6070 [OBSERVATION_AUDIT] fold=4 faculty=khoa_cong_nghe_thong_tin raw_columns=275 common_raw_columns=136 native_raw_columns=139 history_rows=35187 history_failures=1135 target_rows=5072 target_failures=244 target_prevalence=0.048107 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.652537 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich raw_columns=301 common_raw_columns=136 native_raw_columns=165 history_rows=52130 history_failures=1543 target_rows=6440 target_failures=241 target_prevalence=0.037422 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.690796 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_quoc_te_hoc raw_columns=254 common_raw_columns=136 native_raw_columns=118 history_rows=17562 history_failures=174 target_rows=2202 target_failures=35 target_prevalence=0.015895 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.619716 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_anh raw_columns=511 common_raw_columns=136 native_raw_columns=375 history_rows=71769 history_failures=1192 target_rows=6636 target_failures=68 target_prevalence=0.010247 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.805113 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_bo_dao_nha raw_columns=400 common_raw_columns=136 native_raw_columns=264 history_rows=6633 history_failures=233 target_rows=869 target_failures=64 target_prevalence=0.073648 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.750870 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_duc raw_columns=381 common_raw_columns=136 native_raw_columns=245 history_rows=16585 history_failures=318 target_rows=1746 target_failures=44 target_prevalence=0.025200 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.746094 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_han_quoc raw_columns=333 common_raw_columns=136 native_raw_columns=197 history_rows=26578 history_failures=166 target_rows=3351 target_failures=38 target_prevalence=0.011340 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.681313 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_italia raw_columns=420 common_raw_columns=136 native_raw_columns=284 history_rows=16985 history_failures=576 target_rows=1795 target_failures=34 target_prevalence=0.018942 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.741106 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_nga raw_columns=469 common_raw_columns=136 native_raw_columns=333 history_rows=11759 history_failures=547 target_rows=1659 target_failures=114 target_prevalence=0.068716 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.790805 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_nhat raw_columns=362 common_raw_columns=136 native_raw_columns=226 history_rows=26919 history_failures=244 target_rows=2413 target_failures=38 target_prevalence=0.015748 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.740509 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_phap raw_columns=495 common_raw_columns=136 native_raw_columns=359 history_rows=22768 history_failures=303 target_rows=2275 target_failures=19 target_prevalence=0.008352 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.786675 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_tay_ban_nha raw_columns=396 common_raw_columns=136 native_raw_columns=260 history_rows=12085 history_failures=329 target_rows=1306 target_failures=39 target_prevalence=0.029862 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.745580 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_tieng_trung_quoc raw_columns=436 common_raw_columns=136 native_raw_columns=300 history_rows=52678 history_failures=399 target_rows=5107 target_failures=104 target_prevalence=0.020364 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.758164 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=4 faculty=khoa_viet_nam_hoc raw_columns=181 common_raw_columns=136 native_raw_columns=45 history_rows=7361 history_failures=439 target_rows=505 target_failures=18 target_prevalence=0.035644 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.495830 history_latest_semester=6069 target_first_semester=6070 availability_provenance=semester_proxy; source release timestamps not verified [SELECTION_ORIGINS] target=khoa_cong_nghe_thong_tin fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=4 origin=6064 ap=0.271040 rows=3992 failures=116 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.009550 ap=0.280590 base_ap=0.271040 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6064 delta_ap=0.010505 ap=0.281545 base_ap=0.271040 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6064 delta_ap=0.015131 ap=0.286171 base_ap=0.271040 donor_failures=65 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.031939 ap=0.302978 base_ap=0.271040 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6064 delta_ap=0.021208 ap=0.292248 base_ap=0.271040 donor_failures=50 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6064 delta_ap=0.024686 ap=0.295726 base_ap=0.271040 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6064 delta_ap=-0.022724 ap=0.248316 base_ap=0.271040 donor_failures=128 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6064 delta_ap=0.011532 ap=0.282572 base_ap=0.271040 donor_failures=138 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6064 delta_ap=-0.027700 ap=0.243340 base_ap=0.271040 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6064 delta_ap=-0.017929 ap=0.253111 base_ap=0.271040 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.009236 ap=0.261804 base_ap=0.271040 donor_failures=79 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.021895 ap=0.249144 base_ap=0.271040 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.041079 ap=0.229961 base_ap=0.271040 donor_failures=269 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=4 origin=6066 ap=0.240638 rows=5222 failures=189 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.030239 ap=0.270878 base_ap=0.240638 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6066 delta_ap=0.054930 ap=0.295569 base_ap=0.240638 donor_failures=29 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6066 delta_ap=-0.015215 ap=0.225424 base_ap=0.240638 donor_failures=71 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.009284 ap=0.249923 base_ap=0.240638 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6066 delta_ap=-0.035136 ap=0.205502 base_ap=0.240638 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6066 delta_ap=0.053349 ap=0.293987 base_ap=0.240638 donor_failures=18 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6066 delta_ap=0.021540 ap=0.262178 base_ap=0.240638 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6066 delta_ap=0.049580 ap=0.290218 base_ap=0.240638 donor_failures=156 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6066 delta_ap=-0.075073 ap=0.165566 base_ap=0.240638 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6066 delta_ap=0.036508 ap=0.277147 base_ap=0.240638 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.029523 ap=0.270162 base_ap=0.240638 donor_failures=78 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.022819 ap=0.263457 base_ap=0.240638 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6066 delta_ap=0.042613 ap=0.283251 base_ap=0.240638 donor_failures=257 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=4 origin=6067 ap=0.399778 rows=4598 failures=152 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.002647 ap=0.397131 base_ap=0.399778 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.038235 ap=0.361543 base_ap=0.399778 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6067 delta_ap=-0.044878 ap=0.354901 base_ap=0.399778 donor_failures=63 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.030891 ap=0.368887 base_ap=0.399778 donor_failures=136 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6067 delta_ap=-0.029601 ap=0.370177 base_ap=0.399778 donor_failures=71 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.020678 ap=0.379101 base_ap=0.399778 donor_failures=15 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6067 delta_ap=-0.056357 ap=0.343421 base_ap=0.399778 donor_failures=143 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6067 delta_ap=-0.018664 ap=0.381115 base_ap=0.399778 donor_failures=200 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6067 delta_ap=-0.054266 ap=0.345512 base_ap=0.399778 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6067 delta_ap=-0.005811 ap=0.393967 base_ap=0.399778 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.036235 ap=0.363543 base_ap=0.399778 donor_failures=106 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.023626 ap=0.376152 base_ap=0.399778 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.022974 ap=0.376804 base_ap=0.399778 donor_failures=244 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=4 origin=6069 ap=0.242588 rows=6277 failures=218 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.032802 ap=0.275390 base_ap=0.242588 donor_failures=117 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6069 delta_ap=0.000797 ap=0.243385 base_ap=0.242588 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6069 delta_ap=0.064278 ap=0.306866 base_ap=0.242588 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.021673 ap=0.264261 base_ap=0.242588 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6069 delta_ap=0.040785 ap=0.283373 base_ap=0.242588 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6069 delta_ap=0.039569 ap=0.282157 base_ap=0.242588 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6069 delta_ap=-0.035654 ap=0.206934 base_ap=0.242588 donor_failures=145 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6069 delta_ap=0.047756 ap=0.290344 base_ap=0.242588 donor_failures=185 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6069 delta_ap=0.032030 ap=0.274618 base_ap=0.242588 donor_failures=36 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6069 delta_ap=0.001430 ap=0.244018 base_ap=0.242588 donor_failures=55 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.059455 ap=0.302043 base_ap=0.242588 donor_failures=110 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.032209 ap=0.274797 base_ap=0.242588 donor_failures=31 [TRANSFER_EDGE] fold=4 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6069 delta_ap=0.068320 ap=0.310908 base_ap=0.242588 donor_failures=256 [POLICY_REPLAY] fold=4 faculty=khoa_cong_nghe_thong_tin origin=6067 policy_sources=khoa_tieng_han_quoc,khoa_quoc_te_hoc,khoa_tieng_nga policy_donor_count=3 base_ap=0.399778 policy_ap=0.386075 delta_ap=-0.013703 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_cong_nghe_thong_tin origin=6069 policy_sources=khoa_tieng_han_quoc,khoa_tieng_nga,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.242588 policy_ap=0.264308 delta_ap=0.021720 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_cong_nghe_thong_tin selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.004009 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common rows=5072 failures=244 prevalence=0.048107 ap=0.345897 auc=0.873713 f2_at20=0.457057 brier=0.038324 ece10=0.013361 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=0.760680 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common in_sample_bce=0.063666 local_bce=0.063666 donor_bce=nan in_sample_local_ap=0.745118 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native rows=5072 failures=244 prevalence=0.048107 ap=0.341917 auc=0.874177 f2_at20=0.449523 brier=0.038613 ece10=0.015262 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=1.183735 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native in_sample_bce=0.063073 local_bce=0.063073 donor_bce=nan in_sample_local_ap=0.752376 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all rows=5072 failures=244 prevalence=0.048107 ap=0.351752 auc=0.874108 f2_at20=0.444500 brier=0.038034 ece10=0.011566 rho=0.250000 donor_rows=4095 effective_n=25315.413515 seconds=0.771202 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all in_sample_bce=0.059866 local_bce=0.070341 donor_bce=0.049241 in_sample_local_ap=0.687801 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03812768422407639, 'khoa_quoc_te_hoc': 0.012844780171556294, 'khoa_tieng_anh': 0.05249157431570571, 'khoa_tieng_bo_dao_nha': 0.004851351035071911, 'khoa_tieng_duc': 0.012130206078195031, 'khoa_tieng_han_quoc': 0.019439048365768315, 'khoa_tieng_italia': 0.012422764560635671, 'khoa_tieng_nga': 0.008600487987548712, 'khoa_tieng_nhat': 0.019688454472048963, 'khoa_tieng_phap': 0.01665242882052122, 'khoa_tieng_tay_ban_nha': 0.008838923150737832, 'khoa_tieng_trung_quoc': 0.03852848934502007, 'khoa_viet_nam_hoc': 0.005383807473113875} source_failures={'khoa_cong_nghe_thong_tin': 1135, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 3, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 24} positive_ess=872.817317 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal rows=5072 failures=244 prevalence=0.048107 ap=0.373835 auc=0.869309 f2_at20=0.444500 brier=0.037691 ece10=0.011333 rho=0.250000 donor_rows=4095 effective_n=32001.502457 seconds=0.780436 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal in_sample_bce=0.063743 local_bce=0.071906 donor_bce=0.039251 in_sample_local_ap=0.676277 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_cong_nghe_thong_tin': 1135, 'khoa_quan_tri_kinh_doanh_va_du_lich': 4, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 3, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 24} positive_ess=1035.496098 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior rows=5072 failures=244 prevalence=0.048107 ap=0.345897 auc=0.873713 f2_at20=0.457057 brier=0.038324 ece10=0.013361 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=0.762929 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior in_sample_bce=0.063666 local_bce=0.063666 donor_bce=nan in_sample_local_ap=0.745118 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor rows=5072 failures=244 prevalence=0.048107 ap=0.342563 auc=0.862722 f2_at20=0.426921 brier=0.038622 ece10=0.014005 rho=0.250000 donor_rows=4096 effective_n=32005.318899 seconds=0.788569 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor in_sample_bce=0.054528 local_bce=0.068216 donor_bce=0.013464 in_sample_local_ap=0.705994 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor mass={'khoa_cong_nghe_thong_tin': 0.75, np.str_('khoa_tieng_nhat'): 0.25} source_failures={'khoa_cong_nghe_thong_tin': 1135, np.str_('khoa_tieng_nhat'): 47} positive_ess=1060.154506 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common rows=5072 failures=244 prevalence=0.048107 ap=0.345897 auc=0.873713 f2_at20=0.457057 brier=0.038324 ece10=0.013361 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=0.734498 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common in_sample_bce=0.063666 local_bce=0.063666 donor_bce=nan in_sample_local_ap=0.745118 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native rows=5072 failures=244 prevalence=0.048107 ap=0.341917 auc=0.874177 f2_at20=0.449523 brier=0.038613 ece10=0.015262 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=1.216439 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native in_sample_bce=0.063073 local_bce=0.063073 donor_bce=nan in_sample_local_ap=0.752376 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator rows=5072 failures=244 prevalence=0.048107 ap=0.341917 auc=0.874177 f2_at20=0.449523 brier=0.038613 ece10=0.015262 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=2.301573 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator in_sample_bce=0.063073 local_bce=0.063073 donor_bce=nan in_sample_local_ap=0.752376 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator rows=5072 failures=244 prevalence=0.048107 ap=0.341917 auc=0.874177 f2_at20=0.449523 brier=0.038613 ece10=0.015262 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=2.320686 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator in_sample_bce=0.063073 local_bce=0.063073 donor_bce=nan in_sample_local_ap=0.752376 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator rows=5072 failures=244 prevalence=0.048107 ap=0.341917 auc=0.874177 f2_at20=0.449523 brier=0.038613 ece10=0.015262 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=2.202784 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator in_sample_bce=0.063073 local_bce=0.063073 donor_bce=nan in_sample_local_ap=0.752376 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan rows=5072 failures=244 prevalence=0.048107 ap=0.341917 auc=0.874177 f2_at20=0.449523 brier=0.038613 ece10=0.015262 rho=0.000000 donor_rows=0 effective_n=35187.000000 seconds=2.212163 [TRAINING_INSIGHT] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan in_sample_bce=0.063073 local_bce=0.063073 donor_bce=nan in_sample_local_ap=0.752376 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1135} positive_ess=1135.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_cong_nghe_thong_tin model=committee_reference ap=0.378777 [FACULTY_COMPLETE] fold=4 faculty=khoa_cong_nghe_thong_tin models=13 positive_rows=244 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=4 origins=[6066, 6067, 6068, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=4 origin=6066 ap=0.522075 rows=5672 failures=219 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.002623 ap=0.524698 base_ap=0.522075 donor_failures=130 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6066 delta_ap=0.012549 ap=0.534624 base_ap=0.522075 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6066 delta_ap=0.004597 ap=0.526673 base_ap=0.522075 donor_failures=57 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.017189 ap=0.504887 base_ap=0.522075 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6066 delta_ap=0.051669 ap=0.573745 base_ap=0.522075 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6066 delta_ap=0.027428 ap=0.549503 base_ap=0.522075 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6066 delta_ap=0.008970 ap=0.531046 base_ap=0.522075 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6066 delta_ap=-0.006701 ap=0.515374 base_ap=0.522075 donor_failures=162 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6066 delta_ap=-0.017891 ap=0.504184 base_ap=0.522075 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6066 delta_ap=-0.005620 ap=0.516456 base_ap=0.522075 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.027276 ap=0.549351 base_ap=0.522075 donor_failures=93 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.011448 ap=0.510627 base_ap=0.522075 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6066 delta_ap=0.041854 ap=0.563930 base_ap=0.522075 donor_failures=266 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=4 origin=6067 ap=0.443977 rows=5530 failures=166 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.009679 ap=0.453656 base_ap=0.443977 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.021815 ap=0.422162 base_ap=0.443977 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6067 delta_ap=-0.007702 ap=0.436275 base_ap=0.443977 donor_failures=74 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.006942 ap=0.450919 base_ap=0.443977 donor_failures=125 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6067 delta_ap=-0.020378 ap=0.423599 base_ap=0.443977 donor_failures=60 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6067 delta_ap=0.003660 ap=0.447637 base_ap=0.443977 donor_failures=14 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6067 delta_ap=-0.025254 ap=0.418723 base_ap=0.443977 donor_failures=145 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6067 delta_ap=-0.023359 ap=0.420618 base_ap=0.443977 donor_failures=171 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6067 delta_ap=-0.009085 ap=0.434892 base_ap=0.443977 donor_failures=36 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6067 delta_ap=-0.030429 ap=0.413548 base_ap=0.443977 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.023397 ap=0.420580 base_ap=0.443977 donor_failures=115 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.001167 ap=0.445144 base_ap=0.443977 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6067 delta_ap=0.003069 ap=0.447046 base_ap=0.443977 donor_failures=241 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=4 origin=6068 ap=0.462909 rows=984 failures=34 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6068 delta_ap=0.011142 ap=0.474051 base_ap=0.462909 donor_failures=167 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6068 delta_ap=-0.028210 ap=0.434699 base_ap=0.462909 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6068 delta_ap=-0.042488 ap=0.420421 base_ap=0.462909 donor_failures=50 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6068 delta_ap=0.012567 ap=0.475476 base_ap=0.462909 donor_failures=130 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6068 delta_ap=-0.077520 ap=0.385389 base_ap=0.462909 donor_failures=84 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6068 delta_ap=-0.037331 ap=0.425579 base_ap=0.462909 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6068 delta_ap=-0.064161 ap=0.398749 base_ap=0.462909 donor_failures=136 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6068 delta_ap=-0.025640 ap=0.437270 base_ap=0.462909 donor_failures=198 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6068 delta_ap=0.035321 ap=0.498230 base_ap=0.462909 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6068 delta_ap=-0.018296 ap=0.444613 base_ap=0.462909 donor_failures=50 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6068 delta_ap=-0.005098 ap=0.457811 base_ap=0.462909 donor_failures=106 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6068 delta_ap=-0.018453 ap=0.444457 base_ap=0.462909 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6068 delta_ap=-0.008807 ap=0.454102 base_ap=0.462909 donor_failures=253 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=4 origin=6069 ap=0.397222 rows=6491 failures=209 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.026477 ap=0.423698 base_ap=0.397222 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6069 delta_ap=0.000790 ap=0.398011 base_ap=0.397222 donor_failures=32 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6069 delta_ap=-0.050218 ap=0.347004 base_ap=0.397222 donor_failures=74 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.023634 ap=0.373587 base_ap=0.397222 donor_failures=140 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6069 delta_ap=0.002174 ap=0.399396 base_ap=0.397222 donor_failures=81 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6069 delta_ap=0.007572 ap=0.404794 base_ap=0.397222 donor_failures=26 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6069 delta_ap=-0.046550 ap=0.350672 base_ap=0.397222 donor_failures=121 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6069 delta_ap=-0.029265 ap=0.367956 base_ap=0.397222 donor_failures=203 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6069 delta_ap=-0.025787 ap=0.371435 base_ap=0.397222 donor_failures=32 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6069 delta_ap=-0.031236 ap=0.365985 base_ap=0.397222 donor_failures=53 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.066498 ap=0.330724 base_ap=0.397222 donor_failures=102 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.000740 ap=0.397962 base_ap=0.397222 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.008033 ap=0.389188 base_ap=0.397222 donor_failures=251 [POLICY_REPLAY] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6068 policy_sources=khoa_viet_nam_hoc,khoa_tieng_duc,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.462909 policy_ap=0.380141 delta_ap=-0.082768 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 policy_sources=khoa_viet_nam_hoc,khoa_cong_nghe_thong_tin policy_donor_count=2 base_ap=0.397222 policy_ap=0.424358 delta_ap=0.027136 policy_donor_rows=4096 selected_using_origins=6066,6067,6068 [SELECTION_FINAL] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich selected=[] reason=prequential_policy_delta_-0.027816_below_threshold replay_mean=-0.027816 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common rows=6440 failures=241 prevalence=0.037422 ap=0.342272 auc=0.895878 f2_at20=0.415187 brier=0.029617 ece10=0.006395 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=0.934185 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common in_sample_bce=0.056475 local_bce=0.056475 donor_bce=nan in_sample_local_ap=0.738682 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native rows=6440 failures=241 prevalence=0.037422 ap=0.348667 auc=0.897729 f2_at20=0.415187 brier=0.029360 ece10=0.004484 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=1.815401 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native in_sample_bce=0.056409 local_bce=0.056409 donor_bce=nan in_sample_local_ap=0.738470 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all rows=6440 failures=241 prevalence=0.037422 ap=0.332245 auc=0.891234 f2_at20=0.401865 brier=0.029828 ece10=0.005159 rho=0.250000 donor_rows=4095 effective_n=29375.931364 seconds=0.964373 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all in_sample_bce=0.055311 local_bce=0.062990 donor_bce=0.051328 in_sample_local_ap=0.671899 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.027077837528357584, 'khoa_quoc_te_hoc': 0.013514678224145732, 'khoa_tieng_anh': 0.05522918468675066, 'khoa_tieng_bo_dao_nha': 0.0051043651441042386, 'khoa_tieng_duc': 0.012762836712644174, 'khoa_tieng_han_quoc': 0.020452859460274756, 'khoa_tieng_italia': 0.013070653094016357, 'khoa_tieng_nga': 0.009049032071388775, 'khoa_tieng_nhat': 0.020715272925394542, 'khoa_tieng_phap': 0.017520908427704707, 'khoa_tieng_tay_ban_nha': 0.009299902422207105, 'khoa_tieng_trung_quoc': 0.04053787834480976, 'khoa_viet_nam_hoc': 0.005664590958201613} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 9, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 26} positive_ess=924.788099 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal rows=6440 failures=241 prevalence=0.037422 ap=0.335934 auc=0.892552 f2_at20=0.415187 brier=0.029764 ece10=0.003492 rho=0.250000 donor_rows=4095 effective_n=38383.520818 seconds=0.942938 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal in_sample_bce=0.057714 local_bce=0.063766 donor_bce=0.039557 in_sample_local_ap=0.664124 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 9, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 26} positive_ess=1144.298844 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior rows=6440 failures=241 prevalence=0.037422 ap=0.342272 auc=0.895878 f2_at20=0.415187 brier=0.029617 ece10=0.006395 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=0.903041 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior in_sample_bce=0.056475 local_bce=0.056475 donor_bce=nan in_sample_local_ap=0.738682 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor rows=6440 failures=241 prevalence=0.037422 ap=0.358291 auc=0.893669 f2_at20=0.419627 brier=0.029189 ece10=0.003544 rho=0.250000 donor_rows=4096 effective_n=38389.011394 seconds=0.980972 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor in_sample_bce=0.051426 local_bce=0.061571 donor_bce=0.020991 in_sample_local_ap=0.686617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, np.str_('khoa_tieng_anh'): 0.25} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543, np.str_('khoa_tieng_anh'): 69} positive_ess=1210.083172 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common rows=6440 failures=241 prevalence=0.037422 ap=0.342272 auc=0.895878 f2_at20=0.415187 brier=0.029617 ece10=0.006395 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=0.896614 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common in_sample_bce=0.056475 local_bce=0.056475 donor_bce=nan in_sample_local_ap=0.738682 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native rows=6440 failures=241 prevalence=0.037422 ap=0.348667 auc=0.897729 f2_at20=0.415187 brier=0.029360 ece10=0.004484 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=1.781866 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native in_sample_bce=0.056409 local_bce=0.056409 donor_bce=nan in_sample_local_ap=0.738470 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator rows=6440 failures=241 prevalence=0.037422 ap=0.348667 auc=0.897729 f2_at20=0.415187 brier=0.029360 ece10=0.004484 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=3.129458 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator in_sample_bce=0.056409 local_bce=0.056409 donor_bce=nan in_sample_local_ap=0.738470 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator rows=6440 failures=241 prevalence=0.037422 ap=0.348673 auc=0.897748 f2_at20=0.415187 brier=0.029360 ece10=0.004478 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=3.313353 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator in_sample_bce=0.056409 local_bce=0.056409 donor_bce=nan in_sample_local_ap=0.738423 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator rows=6440 failures=241 prevalence=0.037422 ap=0.348667 auc=0.897729 f2_at20=0.415187 brier=0.029360 ece10=0.004484 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=3.176000 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator in_sample_bce=0.056409 local_bce=0.056409 donor_bce=nan in_sample_local_ap=0.738470 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan rows=6440 failures=241 prevalence=0.037422 ap=0.348667 auc=0.897729 f2_at20=0.415187 brier=0.029360 ece10=0.004484 rho=0.000000 donor_rows=0 effective_n=52130.000000 seconds=3.134211 [TRAINING_INSIGHT] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan in_sample_bce=0.056409 local_bce=0.056409 donor_bce=nan in_sample_local_ap=0.738470 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1543} positive_ess=1543.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=committee_reference ap=0.432508 [FACULTY_COMPLETE] fold=4 faculty=khoa_quan_tri_kinh_doanh_va_du_lich models=13 positive_rows=241 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quoc_te_hoc fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_quoc_te_hoc fold=4 origin=6064 ap=0.315234 rows=1690 failures=30 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.049455 ap=0.364689 base_ap=0.315234 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.051472 ap=0.366706 base_ap=0.315234 donor_failures=99 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6064 delta_ap=0.059330 ap=0.374564 base_ap=0.315234 donor_failures=73 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.032495 ap=0.347729 base_ap=0.315234 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6064 delta_ap=0.030464 ap=0.345699 base_ap=0.315234 donor_failures=55 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=0.002927 ap=0.318161 base_ap=0.315234 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6064 delta_ap=0.019451 ap=0.334686 base_ap=0.315234 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6064 delta_ap=0.047955 ap=0.363189 base_ap=0.315234 donor_failures=151 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6064 delta_ap=0.016809 ap=0.332043 base_ap=0.315234 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6064 delta_ap=0.074728 ap=0.389962 base_ap=0.315234 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.003136 ap=0.312098 base_ap=0.315234 donor_failures=72 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.052002 ap=0.367236 base_ap=0.315234 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6064 delta_ap=0.027186 ap=0.342420 base_ap=0.315234 donor_failures=271 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=4 origin=6066 ap=0.039158 rows=2363 failures=31 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.005777 ap=0.044935 base_ap=0.039158 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.017013 ap=0.056171 base_ap=0.039158 donor_failures=96 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6066 delta_ap=0.011127 ap=0.050285 base_ap=0.039158 donor_failures=68 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.011079 ap=0.050237 base_ap=0.039158 donor_failures=121 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.024292 ap=0.063450 base_ap=0.039158 donor_failures=73 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.003241 ap=0.035917 base_ap=0.039158 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6066 delta_ap=0.009992 ap=0.049150 base_ap=0.039158 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.002849 ap=0.042006 base_ap=0.039158 donor_failures=164 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.009561 ap=0.029597 base_ap=0.039158 donor_failures=49 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6066 delta_ap=0.000998 ap=0.040156 base_ap=0.039158 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.009098 ap=0.030060 base_ap=0.039158 donor_failures=91 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.049026 ap=0.088184 base_ap=0.039158 donor_failures=25 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.017755 ap=0.056913 base_ap=0.039158 donor_failures=266 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=4 origin=6067 ap=0.390762 rows=2095 failures=19 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.021619 ap=0.412381 base_ap=0.390762 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.055572 ap=0.446333 base_ap=0.390762 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6067 delta_ap=0.019389 ap=0.410151 base_ap=0.390762 donor_failures=76 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.033156 ap=0.357606 base_ap=0.390762 donor_failures=128 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6067 delta_ap=0.031663 ap=0.422425 base_ap=0.390762 donor_failures=76 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6067 delta_ap=0.068901 ap=0.459663 base_ap=0.390762 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6067 delta_ap=0.012849 ap=0.403610 base_ap=0.390762 donor_failures=141 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6067 delta_ap=0.022789 ap=0.413550 base_ap=0.390762 donor_failures=204 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.033348 ap=0.357413 base_ap=0.390762 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6067 delta_ap=0.058186 ap=0.448947 base_ap=0.390762 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.112833 ap=0.277928 base_ap=0.390762 donor_failures=111 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.076113 ap=0.466875 base_ap=0.390762 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6067 delta_ap=0.012534 ap=0.403295 base_ap=0.390762 donor_failures=246 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=4 origin=6069 ap=0.236975 rows=2665 failures=26 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.025252 ap=0.211723 base_ap=0.236975 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.059341 ap=0.177634 base_ap=0.236975 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6069 delta_ap=-0.143974 ap=0.093000 base_ap=0.236975 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.016971 ap=0.220003 base_ap=0.236975 donor_failures=146 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6069 delta_ap=-0.147578 ap=0.089396 base_ap=0.236975 donor_failures=80 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.095316 ap=0.141659 base_ap=0.236975 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6069 delta_ap=-0.080343 ap=0.156632 base_ap=0.236975 donor_failures=134 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6069 delta_ap=-0.010877 ap=0.226097 base_ap=0.236975 donor_failures=182 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6069 delta_ap=-0.081110 ap=0.155865 base_ap=0.236975 donor_failures=29 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6069 delta_ap=-0.134017 ap=0.102958 base_ap=0.236975 donor_failures=50 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.083206 ap=0.153769 base_ap=0.236975 donor_failures=110 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.133565 ap=0.103410 base_ap=0.236975 donor_failures=20 [TRANSFER_EDGE] fold=4 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.073701 ap=0.163274 base_ap=0.236975 donor_failures=237 [POLICY_REPLAY] fold=4 faculty=khoa_quoc_te_hoc origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_phap,khoa_tieng_anh policy_donor_count=3 base_ap=0.390762 policy_ap=0.466797 delta_ap=0.076035 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_quoc_te_hoc origin=6069 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_phap,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.236975 policy_ap=0.205569 delta_ap=-0.031406 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_quoc_te_hoc selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.022315 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_local_common rows=2202 failures=35 prevalence=0.015895 ap=0.124237 auc=0.836654 f2_at20=0.197935 brier=0.015049 ece10=0.008588 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=0.616320 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=hgb_local_common in_sample_bce=0.009406 local_bce=0.009406 donor_bce=nan in_sample_local_ap=0.960713 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_local_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_local_native rows=2202 failures=35 prevalence=0.015895 ap=0.121579 auc=0.832566 f2_at20=0.189329 brier=0.015364 ece10=0.010267 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=0.855101 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=hgb_local_native in_sample_bce=0.009381 local_bce=0.009381 donor_bce=nan in_sample_local_ap=0.959753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_local_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_pool_all rows=2202 failures=35 prevalence=0.015895 ap=0.118835 auc=0.838724 f2_at20=0.180723 brier=0.015224 ece10=0.005782 rho=0.250000 donor_rows=4095 effective_n=18273.800222 seconds=0.640560 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=hgb_pool_all in_sample_bce=0.017404 local_bce=0.014990 donor_bce=0.040437 in_sample_local_ap=0.878410 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_pool_all mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02447369079977854, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03625809251690839, 'khoa_tieng_anh': 0.04991764898994817, 'khoa_tieng_bo_dao_nha': 0.004613464946569218, 'khoa_tieng_duc': 0.011535401196871775, 'khoa_tieng_han_quoc': 0.018485854266533496, 'khoa_tieng_italia': 0.011813614068668501, 'khoa_tieng_nga': 0.00817876289864427, 'khoa_tieng_nhat': 0.018723030739740205, 'khoa_tieng_phap': 0.015835876662669675, 'khoa_tieng_tay_ban_nha': 0.008405506389158601, 'khoa_tieng_trung_quoc': 0.0366392441512699, 'khoa_viet_nam_hoc': 0.00511981237323926} source_failures={'khoa_quoc_te_hoc': 174, 'khoa_cong_nghe_thong_tin': 6, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 6, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 12} positive_ess=211.943245 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_pool_equal rows=2202 failures=35 prevalence=0.015895 ap=0.132220 auc=0.857881 f2_at20=0.215146 brier=0.015026 ece10=0.003520 rho=0.250000 donor_rows=4095 effective_n=21145.271514 seconds=0.638411 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=hgb_pool_equal in_sample_bce=0.019666 local_bce=0.015836 donor_bce=0.031157 in_sample_local_ap=0.872297 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_pool_equal mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quoc_te_hoc': 174, 'khoa_cong_nghe_thong_tin': 6, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 6, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 10, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 12} positive_ess=258.798592 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_language_prior rows=2202 failures=35 prevalence=0.015895 ap=0.124237 auc=0.836654 f2_at20=0.197935 brier=0.015049 ece10=0.008588 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=0.587459 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=hgb_language_prior in_sample_bce=0.009406 local_bce=0.009406 donor_bce=nan in_sample_local_ap=0.960713 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_language_prior mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_random_donor rows=2202 failures=35 prevalence=0.015895 ap=0.104702 auc=0.853847 f2_at20=0.197935 brier=0.016020 ece10=0.008312 rho=0.250000 donor_rows=4096 effective_n=21146.937714 seconds=0.744526 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=hgb_random_donor in_sample_bce=0.015620 local_bce=0.014158 donor_bce=0.020007 in_sample_local_ap=0.888789 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=hgb_random_donor mass={'khoa_quoc_te_hoc': 0.75, np.str_('khoa_tieng_duc'): 0.25} source_failures={'khoa_quoc_te_hoc': 174, np.str_('khoa_tieng_duc'): 75} positive_ess=241.652820 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_common rows=2202 failures=35 prevalence=0.015895 ap=0.124237 auc=0.836654 f2_at20=0.197935 brier=0.015049 ece10=0.008588 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=0.618562 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_common in_sample_bce=0.009406 local_bce=0.009406 donor_bce=nan in_sample_local_ap=0.960713 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_native rows=2202 failures=35 prevalence=0.015895 ap=0.121579 auc=0.832566 f2_at20=0.189329 brier=0.015364 ece10=0.010267 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=0.843952 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_native in_sample_bce=0.009381 local_bce=0.009381 donor_bce=nan in_sample_local_ap=0.959753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator rows=2202 failures=35 prevalence=0.015895 ap=0.121579 auc=0.832566 f2_at20=0.189329 brier=0.015364 ece10=0.010267 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=1.441606 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator in_sample_bce=0.009381 local_bce=0.009381 donor_bce=nan in_sample_local_ap=0.959753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator rows=2202 failures=35 prevalence=0.015895 ap=0.121579 auc=0.832566 f2_at20=0.189329 brier=0.015364 ece10=0.010267 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=1.501521 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator in_sample_bce=0.009381 local_bce=0.009381 donor_bce=nan in_sample_local_ap=0.959753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=native_placebo_indicator rows=2202 failures=35 prevalence=0.015895 ap=0.121579 auc=0.832566 f2_at20=0.189329 brier=0.015364 ece10=0.010267 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=1.528081 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=native_placebo_indicator in_sample_bce=0.009381 local_bce=0.009381 donor_bce=nan in_sample_local_ap=0.959753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=native_placebo_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan rows=2202 failures=35 prevalence=0.015895 ap=0.121579 auc=0.832566 f2_at20=0.189329 brier=0.015364 ece10=0.010267 rho=0.000000 donor_rows=0 effective_n=17562.000000 seconds=1.474652 [TRAINING_INSIGHT] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.009381 local_bce=0.009381 donor_bce=nan in_sample_local_ap=0.959753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 174} positive_ess=174.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_quoc_te_hoc model=committee_reference ap=0.097129 [FACULTY_COMPLETE] fold=4 faculty=khoa_quoc_te_hoc models=13 positive_rows=35 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_anh fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_anh fold=4 origin=6064 ap=0.586755 rows=9726 failures=240 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.037241 ap=0.549514 base_ap=0.586755 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.065140 ap=0.521614 base_ap=0.586755 donor_failures=107 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.015691 ap=0.571064 base_ap=0.586755 donor_failures=30 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.028643 ap=0.558112 base_ap=0.586755 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_duc origin=6064 delta_ap=-0.031569 ap=0.555185 base_ap=0.586755 donor_failures=58 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.019524 ap=0.567231 base_ap=0.586755 donor_failures=20 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_italia origin=6064 delta_ap=-0.031283 ap=0.555472 base_ap=0.586755 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nga origin=6064 delta_ap=-0.029132 ap=0.557623 base_ap=0.586755 donor_failures=144 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6064 delta_ap=-0.027391 ap=0.559364 base_ap=0.586755 donor_failures=49 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_phap origin=6064 delta_ap=-0.019976 ap=0.566779 base_ap=0.586755 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.030381 ap=0.556374 base_ap=0.586755 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.001196 ap=0.587951 base_ap=0.586755 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.043015 ap=0.543740 base_ap=0.586755 donor_failures=267 [SCREEN_BASE] target=khoa_tieng_anh fold=4 origin=6066 ap=0.517452 rows=8164 failures=148 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.066856 ap=0.584308 base_ap=0.517452 donor_failures=136 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.075876 ap=0.593328 base_ap=0.517452 donor_failures=117 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6066 delta_ap=0.053101 ap=0.570554 base_ap=0.517452 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.009759 ap=0.507693 base_ap=0.517452 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_duc origin=6066 delta_ap=0.065674 ap=0.583127 base_ap=0.517452 donor_failures=64 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6066 delta_ap=0.019185 ap=0.536637 base_ap=0.517452 donor_failures=14 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_italia origin=6066 delta_ap=0.076882 ap=0.594335 base_ap=0.517452 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nga origin=6066 delta_ap=0.085494 ap=0.602946 base_ap=0.517452 donor_failures=157 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6066 delta_ap=-0.028009 ap=0.489443 base_ap=0.517452 donor_failures=52 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_phap origin=6066 delta_ap=0.041539 ap=0.558991 base_ap=0.517452 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.055993 ap=0.573445 base_ap=0.517452 donor_failures=84 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.055175 ap=0.572628 base_ap=0.517452 donor_failures=18 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6066 delta_ap=0.054533 ap=0.571985 base_ap=0.517452 donor_failures=264 [SCREEN_BASE] target=khoa_tieng_anh fold=4 origin=6067 ap=0.700871 rows=8328 failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.091600 ap=0.609271 base_ap=0.700871 donor_failures=135 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.018129 ap=0.682741 base_ap=0.700871 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.030812 ap=0.670058 base_ap=0.700871 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.005371 ap=0.695499 base_ap=0.700871 donor_failures=134 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_duc origin=6067 delta_ap=-0.037574 ap=0.663297 base_ap=0.700871 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.055810 ap=0.645060 base_ap=0.700871 donor_failures=16 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_italia origin=6067 delta_ap=-0.027365 ap=0.673506 base_ap=0.700871 donor_failures=147 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nga origin=6067 delta_ap=-0.032095 ap=0.668776 base_ap=0.700871 donor_failures=203 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6067 delta_ap=-0.029226 ap=0.671645 base_ap=0.700871 donor_failures=35 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_phap origin=6067 delta_ap=-0.058156 ap=0.642714 base_ap=0.700871 donor_failures=52 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.043539 ap=0.657331 base_ap=0.700871 donor_failures=98 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.027478 ap=0.673393 base_ap=0.700871 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.084024 ap=0.616847 base_ap=0.700871 donor_failures=245 [SCREEN_BASE] target=khoa_tieng_anh fold=4 origin=6069 ap=0.376445 rows=7279 failures=101 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.002737 ap=0.379182 base_ap=0.376445 donor_failures=105 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.010485 ap=0.365960 base_ap=0.376445 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.012018 ap=0.364427 base_ap=0.376445 donor_failures=56 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.023136 ap=0.353309 base_ap=0.376445 donor_failures=144 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_duc origin=6069 delta_ap=0.032393 ap=0.408838 base_ap=0.376445 donor_failures=84 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6069 delta_ap=0.013582 ap=0.390027 base_ap=0.376445 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_italia origin=6069 delta_ap=0.009059 ap=0.385504 base_ap=0.376445 donor_failures=147 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nga origin=6069 delta_ap=-0.039381 ap=0.337064 base_ap=0.376445 donor_failures=180 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6069 delta_ap=-0.044144 ap=0.332301 base_ap=0.376445 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_phap origin=6069 delta_ap=-0.026750 ap=0.349695 base_ap=0.376445 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.001611 ap=0.378056 base_ap=0.376445 donor_failures=103 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.017485 ap=0.358960 base_ap=0.376445 donor_failures=35 [TRANSFER_EDGE] fold=4 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6069 delta_ap=0.017654 ap=0.394100 base_ap=0.376445 donor_failures=247 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_anh origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_nga,khoa_tieng_italia policy_donor_count=3 base_ap=0.700871 policy_ap=0.628718 delta_ap=-0.072153 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_anh origin=6069 policy_sources=khoa_tieng_trung_quoc policy_donor_count=1 base_ap=0.376445 policy_ap=0.358960 delta_ap=-0.017485 policy_donor_rows=4096 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_anh selected=[] reason=prequential_policy_delta_-0.044819_below_threshold replay_mean=-0.044819 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=hgb_local_common rows=6636 failures=68 prevalence=0.010247 ap=0.401581 auc=0.912975 f2_at20=0.184375 brier=0.007694 ece10=0.003544 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=1.157661 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=hgb_local_common in_sample_bce=0.023247 local_bce=0.023247 donor_bce=nan in_sample_local_ap=0.877419 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=hgb_local_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=hgb_local_native rows=6636 failures=68 prevalence=0.010247 ap=0.393225 auc=0.904179 f2_at20=0.187500 brier=0.007696 ece10=0.003084 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=3.347402 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=hgb_local_native in_sample_bce=0.023147 local_bce=0.023147 donor_bce=nan in_sample_local_ap=0.877393 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=hgb_local_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=hgb_pool_all rows=6636 failures=68 prevalence=0.010247 ap=0.310898 auc=0.922453 f2_at20=0.187500 brier=0.008221 ece10=0.002034 rho=0.250000 donor_rows=4095 effective_n=34488.680946 seconds=1.172929 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=hgb_pool_all in_sample_bce=0.028498 local_bce=0.029258 donor_bce=0.038125 in_sample_local_ap=0.805722 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=hgb_pool_all mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.028820070111063786, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.042697310225076175, 'khoa_quoc_te_hoc': 0.014384234839301511, 'khoa_tieng_bo_dao_nha': 0.005432788389083642, 'khoa_tieng_duc': 0.013584018608917866, 'khoa_tieng_han_quoc': 0.021768830062575763, 'khoa_tieng_italia': 0.013911640402319563, 'khoa_tieng_nga': 0.00963126167152639, 'khoa_tieng_nhat': 0.02204812764145071, 'khoa_tieng_phap': 0.018648232480424597, 'khoa_tieng_tay_ban_nha': 0.009898273433148773, 'khoa_tieng_trung_quoc': 0.043146152082036496, 'khoa_viet_nam_hoc': 0.0060290600530747305} source_failures={'khoa_tieng_anh': 1192, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 14, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 3, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 14} positive_ess=523.027015 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=hgb_pool_equal rows=6636 failures=68 prevalence=0.010247 ap=0.354139 auc=0.909089 f2_at20=0.168750 brier=0.008182 ece10=0.003785 rho=0.250000 donor_rows=4095 effective_n=43289.741494 seconds=1.178981 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=hgb_pool_equal in_sample_bce=0.030410 local_bce=0.030343 donor_bce=0.030613 in_sample_local_ap=0.788760 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=hgb_pool_equal mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_anh': 1192, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 14, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 2, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 3, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 14} positive_ess=684.555989 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=hgb_language_prior rows=6636 failures=68 prevalence=0.010247 ap=0.326813 auc=0.922069 f2_at20=0.181250 brier=0.008225 ece10=0.002459 rho=0.250000 donor_rows=4095 effective_n=43289.741494 seconds=1.144500 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=hgb_language_prior in_sample_bce=0.028658 local_bce=0.029204 donor_bce=0.027022 in_sample_local_ap=0.805591 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=hgb_language_prior mass={'khoa_tieng_anh': 0.75, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_anh': 1192, 'khoa_tieng_bo_dao_nha': 18, 'khoa_tieng_duc': 17, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 26, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 9} positive_ess=685.115381 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=hgb_random_donor rows=6636 failures=68 prevalence=0.010247 ap=0.212314 auc=0.894474 f2_at20=0.178125 brier=0.009227 ece10=0.003630 rho=0.250000 donor_rows=4096 effective_n=43296.725525 seconds=1.187844 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=hgb_random_donor in_sample_bce=0.032389 local_bce=0.029661 donor_bce=0.040573 in_sample_local_ap=0.798806 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=hgb_random_donor mass={'khoa_tieng_anh': 0.75, np.str_('khoa_tieng_bo_dao_nha'): 0.25} source_failures={'khoa_tieng_anh': 1192, np.str_('khoa_tieng_bo_dao_nha'): 135} positive_ess=676.588048 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=suite32_selective_common rows=6636 failures=68 prevalence=0.010247 ap=0.401581 auc=0.912975 f2_at20=0.184375 brier=0.007694 ece10=0.003544 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=1.115766 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=suite32_selective_common in_sample_bce=0.023247 local_bce=0.023247 donor_bce=nan in_sample_local_ap=0.877419 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=suite32_selective_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=suite32_selective_native rows=6636 failures=68 prevalence=0.010247 ap=0.393225 auc=0.904179 f2_at20=0.187500 brier=0.007696 ece10=0.003084 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=3.474250 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=suite32_selective_native in_sample_bce=0.023147 local_bce=0.023147 donor_bce=nan in_sample_local_ap=0.877393 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=suite32_selective_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=native_cache_nan_indicator rows=6636 failures=68 prevalence=0.010247 ap=0.393225 auc=0.904179 f2_at20=0.187500 brier=0.007696 ece10=0.003084 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=7.748819 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=native_cache_nan_indicator in_sample_bce=0.023147 local_bce=0.023147 donor_bce=nan in_sample_local_ap=0.877393 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=native_cache_nan_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=native_naive_zero_indicator rows=6636 failures=68 prevalence=0.010247 ap=0.393225 auc=0.904179 f2_at20=0.187500 brier=0.007696 ece10=0.003084 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=7.929675 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=native_naive_zero_indicator in_sample_bce=0.023147 local_bce=0.023147 donor_bce=nan in_sample_local_ap=0.877393 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=native_naive_zero_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=native_placebo_indicator rows=6636 failures=68 prevalence=0.010247 ap=0.393225 auc=0.904179 f2_at20=0.187500 brier=0.007696 ece10=0.003084 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=7.190686 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=native_placebo_indicator in_sample_bce=0.023147 local_bce=0.023147 donor_bce=nan in_sample_local_ap=0.877393 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=native_placebo_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan rows=6636 failures=68 prevalence=0.010247 ap=0.393225 auc=0.904179 f2_at20=0.187500 brier=0.007696 ece10=0.003084 rho=0.000000 donor_rows=0 effective_n=71769.000000 seconds=7.545751 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan in_sample_bce=0.023147 local_bce=0.023147 donor_bce=nan in_sample_local_ap=0.877393 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1192} positive_ess=1192.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_anh model=committee_reference ap=0.449701 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_anh models=13 positive_rows=68 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_bo_dao_nha fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=4 origin=6064 ap=0.167090 rows=780 failures=26 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.019003 ap=0.148087 base_ap=0.167090 donor_failures=119 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.045141 ap=0.212231 base_ap=0.167090 donor_failures=96 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=0.007116 ap=0.174206 base_ap=0.167090 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6064 delta_ap=0.018047 ap=0.185137 base_ap=0.167090 donor_failures=59 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6064 delta_ap=0.042028 ap=0.209118 base_ap=0.167090 donor_failures=60 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.005822 ap=0.161268 base_ap=0.167090 donor_failures=15 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6064 delta_ap=0.003338 ap=0.170429 base_ap=0.167090 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6064 delta_ap=0.005772 ap=0.172863 base_ap=0.167090 donor_failures=155 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6064 delta_ap=0.012542 ap=0.179633 base_ap=0.167090 donor_failures=55 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6064 delta_ap=0.012932 ap=0.180022 base_ap=0.167090 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.013650 ap=0.153440 base_ap=0.167090 donor_failures=77 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.040970 ap=0.208060 base_ap=0.167090 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.011581 ap=0.178671 base_ap=0.167090 donor_failures=272 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=4 origin=6066 ap=0.108080 rows=768 failures=30 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.023693 ap=0.131774 base_ap=0.108080 donor_failures=119 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.062304 ap=0.170384 base_ap=0.108080 donor_failures=114 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.033099 ap=0.141180 base_ap=0.108080 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.001192 ap=0.106888 base_ap=0.108080 donor_failures=63 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.009860 ap=0.098221 base_ap=0.108080 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=0.011117 ap=0.119198 base_ap=0.108080 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6066 delta_ap=0.035276 ap=0.143356 base_ap=0.108080 donor_failures=126 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6066 delta_ap=0.080030 ap=0.188110 base_ap=0.108080 donor_failures=178 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6066 delta_ap=0.005463 ap=0.113543 base_ap=0.108080 donor_failures=54 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6066 delta_ap=0.046493 ap=0.154573 base_ap=0.108080 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.057609 ap=0.165689 base_ap=0.108080 donor_failures=103 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.008135 ap=0.099946 base_ap=0.108080 donor_failures=26 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=0.029565 ap=0.137645 base_ap=0.108080 donor_failures=259 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=4 origin=6067 ap=0.411685 rows=716 failures=30 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.033110 ap=0.444796 base_ap=0.411685 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.000837 ap=0.410848 base_ap=0.411685 donor_failures=114 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6067 delta_ap=0.052972 ap=0.464657 base_ap=0.411685 donor_failures=40 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6067 delta_ap=0.020961 ap=0.432646 base_ap=0.411685 donor_failures=78 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6067 delta_ap=0.064584 ap=0.476269 base_ap=0.411685 donor_failures=69 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.020040 ap=0.391645 base_ap=0.411685 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6067 delta_ap=-0.050600 ap=0.361085 base_ap=0.411685 donor_failures=141 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6067 delta_ap=-0.011220 ap=0.400465 base_ap=0.411685 donor_failures=212 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6067 delta_ap=-0.003970 ap=0.407715 base_ap=0.411685 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6067 delta_ap=0.045005 ap=0.456690 base_ap=0.411685 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.081497 ap=0.493182 base_ap=0.411685 donor_failures=106 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.016821 ap=0.428506 base_ap=0.411685 donor_failures=18 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.005422 ap=0.406263 base_ap=0.411685 donor_failures=252 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=4 origin=6069 ap=0.493156 rows=1032 failures=50 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.193538 ap=0.299618 base_ap=0.493156 donor_failures=126 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.125821 ap=0.367335 base_ap=0.493156 donor_failures=135 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.144519 ap=0.348637 base_ap=0.493156 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6069 delta_ap=-0.157955 ap=0.335201 base_ap=0.493156 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6069 delta_ap=-0.044735 ap=0.448421 base_ap=0.493156 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.157736 ap=0.335420 base_ap=0.493156 donor_failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6069 delta_ap=-0.065127 ap=0.428029 base_ap=0.493156 donor_failures=149 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6069 delta_ap=-0.118090 ap=0.375066 base_ap=0.493156 donor_failures=185 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6069 delta_ap=-0.102612 ap=0.390544 base_ap=0.493156 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6069 delta_ap=-0.011199 ap=0.481957 base_ap=0.493156 donor_failures=64 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.174625 ap=0.318531 base_ap=0.493156 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.181359 ap=0.311797 base_ap=0.493156 donor_failures=18 [TRANSFER_EDGE] fold=4 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.107266 ap=0.385890 base_ap=0.493156 donor_failures=264 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_bo_dao_nha origin=6067 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_nga,khoa_tieng_phap policy_donor_count=3 base_ap=0.411685 policy_ap=0.437465 delta_ap=0.025779 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_bo_dao_nha origin=6069 policy_sources=khoa_tieng_tay_ban_nha,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_phap policy_donor_count=3 base_ap=0.493156 policy_ap=0.421760 delta_ap=-0.071396 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_bo_dao_nha selected=[] reason=prequential_policy_delta_-0.022808_below_threshold replay_mean=-0.022808 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common rows=869 failures=64 prevalence=0.073648 ap=0.556293 auc=0.947748 f2_at20=0.697674 brier=0.048659 ece10=0.034015 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=0.354673 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common in_sample_bce=0.034955 local_bce=0.034955 donor_bce=nan in_sample_local_ap=0.945503 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native rows=869 failures=64 prevalence=0.073648 ap=0.604551 auc=0.956483 f2_at20=0.720930 brier=0.045830 ece10=0.033185 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=0.666198 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native in_sample_bce=0.034512 local_bce=0.034512 donor_bce=nan in_sample_local_ap=0.950268 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all rows=869 failures=64 prevalence=0.073648 ap=0.644721 auc=0.956347 f2_at20=0.720930 brier=0.043538 ece10=0.028490 rho=0.250000 donor_rows=4095 effective_n=9384.388838 seconds=0.442876 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all in_sample_bce=0.042409 local_bce=0.042195 donor_bce=0.058349 in_sample_local_ap=0.916740 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.02375150526776216, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0351881652203496, 'khoa_quoc_te_hoc': 0.011854489882980618, 'khoa_tieng_anh': 0.04844464664683043, 'khoa_tieng_duc': 0.011195007101083792, 'khoa_tieng_han_quoc': 0.017940361696268017, 'khoa_tieng_italia': 0.011465010287121389, 'khoa_tieng_nga': 0.007937418661540206, 'khoa_tieng_nhat': 0.018170539412365065, 'khoa_tieng_phap': 0.01536858134925992, 'khoa_tieng_tay_ban_nha': 0.008157471258160846, 'khoa_tieng_trung_quoc': 0.035558069585221105, 'khoa_viet_nam_hoc': 0.004968733631056847} source_failures={'khoa_tieng_bo_dao_nha': 233, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 19} positive_ess=294.740597 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal rows=869 failures=64 prevalence=0.073648 ap=0.630488 auc=0.953998 f2_at20=0.674419 brier=0.043329 ece10=0.025850 rho=0.250000 donor_rows=4095 effective_n=9993.427152 seconds=0.443524 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal in_sample_bce=0.046247 local_bce=0.043154 donor_bce=0.055528 in_sample_local_ap=0.907767 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_bo_dao_nha': 233, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 12, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 19} positive_ess=315.020041 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior rows=869 failures=64 prevalence=0.073648 ap=0.614179 auc=0.951941 f2_at20=0.697674 brier=0.044362 ece10=0.029206 rho=0.250000 donor_rows=4095 effective_n=9993.427152 seconds=0.439712 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior in_sample_bce=0.040426 local_bce=0.041717 donor_bce=0.036552 in_sample_local_ap=0.916377 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_bo_dao_nha': 233, 'khoa_tieng_anh': 8, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 18, 'khoa_tieng_nga': 19, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 3} positive_ess=291.841449 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor rows=869 failures=64 prevalence=0.073648 ap=0.592255 auc=0.952989 f2_at20=0.686047 brier=0.046195 ece10=0.031250 rho=0.250000 donor_rows=4096 effective_n=9993.799297 seconds=0.443426 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor in_sample_bce=0.045505 local_bce=0.042176 donor_bce=0.055492 in_sample_local_ap=0.912452 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor mass={'khoa_tieng_bo_dao_nha': 0.75, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.25} source_failures={'khoa_tieng_bo_dao_nha': 233, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 118} positive_ess=329.222527 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common rows=869 failures=64 prevalence=0.073648 ap=0.556293 auc=0.947748 f2_at20=0.697674 brier=0.048659 ece10=0.034015 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=0.397426 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common in_sample_bce=0.034955 local_bce=0.034955 donor_bce=nan in_sample_local_ap=0.945503 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native rows=869 failures=64 prevalence=0.073648 ap=0.604551 auc=0.956483 f2_at20=0.720930 brier=0.045830 ece10=0.033185 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=0.871543 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native in_sample_bce=0.034512 local_bce=0.034512 donor_bce=nan in_sample_local_ap=0.950268 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator rows=869 failures=64 prevalence=0.073648 ap=0.604551 auc=0.956483 f2_at20=0.720930 brier=0.045830 ece10=0.033185 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=1.275996 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator in_sample_bce=0.034512 local_bce=0.034512 donor_bce=nan in_sample_local_ap=0.950268 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator rows=869 failures=64 prevalence=0.073648 ap=0.583292 auc=0.953882 f2_at20=0.720930 brier=0.046124 ece10=0.031570 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=1.434891 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator in_sample_bce=0.034387 local_bce=0.034387 donor_bce=nan in_sample_local_ap=0.950788 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator rows=869 failures=64 prevalence=0.073648 ap=0.604551 auc=0.956483 f2_at20=0.720930 brier=0.045830 ece10=0.033185 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=1.237356 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator in_sample_bce=0.034512 local_bce=0.034512 donor_bce=nan in_sample_local_ap=0.950268 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan rows=869 failures=64 prevalence=0.073648 ap=0.604551 auc=0.956483 f2_at20=0.720930 brier=0.045830 ece10=0.033185 rho=0.000000 donor_rows=0 effective_n=6633.000000 seconds=1.557560 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan in_sample_bce=0.034512 local_bce=0.034512 donor_bce=nan in_sample_local_ap=0.950268 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 233} positive_ess=233.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_bo_dao_nha model=committee_reference ap=0.656224 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_bo_dao_nha models=13 positive_rows=64 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_duc fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_duc fold=4 origin=6064 ap=0.357053 rows=1984 failures=53 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.007164 ap=0.349889 base_ap=0.357053 donor_failures=146 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.011107 ap=0.368160 base_ap=0.357053 donor_failures=90 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.014651 ap=0.342403 base_ap=0.357053 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_anh origin=6064 delta_ap=0.068687 ap=0.425740 base_ap=0.357053 donor_failures=65 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.043577 ap=0.400631 base_ap=0.357053 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6064 delta_ap=0.013572 ap=0.370626 base_ap=0.357053 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_italia origin=6064 delta_ap=0.018901 ap=0.375955 base_ap=0.357053 donor_failures=126 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nga origin=6064 delta_ap=0.025618 ap=0.382671 base_ap=0.357053 donor_failures=149 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6064 delta_ap=0.030313 ap=0.387366 base_ap=0.357053 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_phap origin=6064 delta_ap=0.021226 ap=0.378279 base_ap=0.357053 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.042455 ap=0.314598 base_ap=0.357053 donor_failures=81 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.000449 ap=0.356604 base_ap=0.357053 donor_failures=24 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6064 delta_ap=0.020355 ap=0.377408 base_ap=0.357053 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_duc fold=4 origin=6066 ap=0.155992 rows=1900 failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.063722 ap=0.219713 base_ap=0.155992 donor_failures=127 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.020616 ap=0.176607 base_ap=0.155992 donor_failures=105 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6066 delta_ap=0.021237 ap=0.177228 base_ap=0.155992 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_anh origin=6066 delta_ap=0.008051 ap=0.164043 base_ap=0.155992 donor_failures=75 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.032199 ap=0.188191 base_ap=0.155992 donor_failures=121 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6066 delta_ap=0.027971 ap=0.183963 base_ap=0.155992 donor_failures=25 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_italia origin=6066 delta_ap=0.013101 ap=0.169093 base_ap=0.155992 donor_failures=128 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nga origin=6066 delta_ap=0.025986 ap=0.181978 base_ap=0.155992 donor_failures=164 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6066 delta_ap=0.024588 ap=0.180580 base_ap=0.155992 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_phap origin=6066 delta_ap=0.022802 ap=0.178794 base_ap=0.155992 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.047825 ap=0.203817 base_ap=0.155992 donor_failures=88 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.026197 ap=0.129795 base_ap=0.155992 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.027459 ap=0.128533 base_ap=0.155992 donor_failures=263 [SCREEN_BASE] target=khoa_tieng_duc fold=4 origin=6067 ap=0.321965 rows=1893 failures=53 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.088986 ap=0.410951 base_ap=0.321965 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.063470 ap=0.385435 base_ap=0.321965 donor_failures=105 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6067 delta_ap=0.022648 ap=0.344613 base_ap=0.321965 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_anh origin=6067 delta_ap=0.066369 ap=0.388334 base_ap=0.321965 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.028994 ap=0.292971 base_ap=0.321965 donor_failures=127 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6067 delta_ap=0.036511 ap=0.358476 base_ap=0.321965 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_italia origin=6067 delta_ap=0.074069 ap=0.396034 base_ap=0.321965 donor_failures=138 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nga origin=6067 delta_ap=0.103952 ap=0.425917 base_ap=0.321965 donor_failures=198 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6067 delta_ap=0.025830 ap=0.347794 base_ap=0.321965 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_phap origin=6067 delta_ap=-0.043409 ap=0.278556 base_ap=0.321965 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.088631 ap=0.410596 base_ap=0.321965 donor_failures=103 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.008149 ap=0.330114 base_ap=0.321965 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.001675 ap=0.320290 base_ap=0.321965 donor_failures=255 [SCREEN_BASE] target=khoa_tieng_duc fold=4 origin=6069 ap=0.321459 rows=1909 failures=47 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.039415 ap=0.360874 base_ap=0.321459 donor_failures=139 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.036113 ap=0.357572 base_ap=0.321459 donor_failures=135 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.033907 ap=0.287552 base_ap=0.321459 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_anh origin=6069 delta_ap=0.068569 ap=0.390028 base_ap=0.321459 donor_failures=75 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.020801 ap=0.342260 base_ap=0.321459 donor_failures=135 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.043688 ap=0.277771 base_ap=0.321459 donor_failures=24 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_italia origin=6069 delta_ap=-0.019666 ap=0.301793 base_ap=0.321459 donor_failures=141 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nga origin=6069 delta_ap=0.042503 ap=0.363962 base_ap=0.321459 donor_failures=186 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6069 delta_ap=-0.069100 ap=0.252359 base_ap=0.321459 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_phap origin=6069 delta_ap=-0.041680 ap=0.279779 base_ap=0.321459 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.091822 ap=0.229636 base_ap=0.321459 donor_failures=105 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.049983 ap=0.371442 base_ap=0.321459 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.025793 ap=0.295666 base_ap=0.321459 donor_failures=253 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_duc origin=6067 policy_sources=khoa_tieng_anh,khoa_tieng_bo_dao_nha,khoa_cong_nghe_thong_tin policy_donor_count=3 base_ap=0.321965 policy_ap=0.338271 delta_ap=0.016306 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_duc origin=6069 policy_sources=khoa_tieng_nga,khoa_cong_nghe_thong_tin,khoa_tieng_anh policy_donor_count=3 base_ap=0.321459 policy_ap=0.265499 delta_ap=-0.055960 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_duc selected=[] reason=prequential_policy_delta_-0.019827_below_threshold replay_mean=-0.019827 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=hgb_local_common rows=1746 failures=44 prevalence=0.025200 ap=0.436248 auc=0.929415 f2_at20=0.380228 brier=0.018000 ece10=0.003794 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=0.472249 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=hgb_local_common in_sample_bce=0.023415 local_bce=0.023415 donor_bce=nan in_sample_local_ap=0.909451 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=hgb_local_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=hgb_local_native rows=1746 failures=44 prevalence=0.025200 ap=0.489551 auc=0.933207 f2_at20=0.370722 brier=0.017463 ece10=0.006769 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=0.999726 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=hgb_local_native in_sample_bce=0.023360 local_bce=0.023360 donor_bce=nan in_sample_local_ap=0.911409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=hgb_local_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=hgb_pool_all rows=1746 failures=44 prevalence=0.025200 ap=0.432618 auc=0.927445 f2_at20=0.370722 brier=0.018488 ece10=0.005022 rho=0.250000 donor_rows=4095 effective_n=17687.202886 seconds=0.611588 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=hgb_pool_all in_sample_bce=0.028071 local_bce=0.028311 donor_bce=0.045195 in_sample_local_ap=0.867287 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=hgb_pool_all mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.024407348216218015, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03615980511300893, 'khoa_quoc_te_hoc': 0.012181824235462551, 'khoa_tieng_anh': 0.04978233364963625, 'khoa_tieng_bo_dao_nha': 0.004600958897268141, 'khoa_tieng_han_quoc': 0.0184357433396039, 'khoa_tieng_italia': 0.011781590060319522, 'khoa_tieng_nga': 0.008156592141259774, 'khoa_tieng_nhat': 0.018672276881586177, 'khoa_tieng_phap': 0.015792949219508676, 'khoa_tieng_tay_ban_nha': 0.008382720981981832, 'khoa_tieng_trung_quoc': 0.0365399235323822, 'khoa_viet_nam_hoc': 0.005105933731764027} source_failures={'khoa_tieng_duc': 318, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 12} positive_ess=349.285062 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=hgb_pool_equal rows=1746 failures=44 prevalence=0.025200 ap=0.416415 auc=0.920762 f2_at20=0.370722 brier=0.018851 ece10=0.006399 rho=0.250000 donor_rows=4095 effective_n=20334.004491 seconds=0.589826 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=hgb_pool_equal in_sample_bce=0.030943 local_bce=0.029430 donor_bce=0.035482 in_sample_local_ap=0.853882 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=hgb_pool_equal mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_duc': 318, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 13, 'khoa_tieng_nhat': 0, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 12} positive_ess=400.725851 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=hgb_language_prior rows=1746 failures=44 prevalence=0.025200 ap=0.407499 auc=0.930530 f2_at20=0.389734 brier=0.019079 ece10=0.005466 rho=0.250000 donor_rows=4095 effective_n=20334.004491 seconds=0.615906 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=hgb_language_prior in_sample_bce=0.029487 local_bce=0.028963 donor_bce=0.031060 in_sample_local_ap=0.856957 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=hgb_language_prior mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_duc': 318, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 20, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 14, 'khoa_tieng_trung_quoc': 2} positive_ess=394.109240 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=hgb_random_donor rows=1746 failures=44 prevalence=0.025200 ap=0.444548 auc=0.922665 f2_at20=0.370722 brier=0.017640 ece10=0.006411 rho=0.250000 donor_rows=4096 effective_n=20335.545286 seconds=0.587890 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=hgb_random_donor in_sample_bce=0.021705 local_bce=0.025635 donor_bce=0.009916 in_sample_local_ap=0.890923 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=hgb_random_donor mass={'khoa_tieng_duc': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_tieng_duc': 318, np.str_('khoa_tieng_han_quoc'): 22} positive_ess=337.610857 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=suite32_selective_common rows=1746 failures=44 prevalence=0.025200 ap=0.436248 auc=0.929415 f2_at20=0.380228 brier=0.018000 ece10=0.003794 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=0.541465 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=suite32_selective_common in_sample_bce=0.023415 local_bce=0.023415 donor_bce=nan in_sample_local_ap=0.909451 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=suite32_selective_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=suite32_selective_native rows=1746 failures=44 prevalence=0.025200 ap=0.489551 auc=0.933207 f2_at20=0.370722 brier=0.017463 ece10=0.006769 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=1.037559 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=suite32_selective_native in_sample_bce=0.023360 local_bce=0.023360 donor_bce=nan in_sample_local_ap=0.911409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=suite32_selective_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=native_cache_nan_indicator rows=1746 failures=44 prevalence=0.025200 ap=0.489551 auc=0.933207 f2_at20=0.370722 brier=0.017463 ece10=0.006769 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=1.897229 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=native_cache_nan_indicator in_sample_bce=0.023360 local_bce=0.023360 donor_bce=nan in_sample_local_ap=0.911409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=native_cache_nan_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=native_naive_zero_indicator rows=1746 failures=44 prevalence=0.025200 ap=0.489551 auc=0.933207 f2_at20=0.370722 brier=0.017463 ece10=0.006769 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=1.941788 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=native_naive_zero_indicator in_sample_bce=0.023360 local_bce=0.023360 donor_bce=nan in_sample_local_ap=0.911409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=native_naive_zero_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=native_placebo_indicator rows=1746 failures=44 prevalence=0.025200 ap=0.489551 auc=0.933207 f2_at20=0.370722 brier=0.017463 ece10=0.006769 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=1.974176 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=native_placebo_indicator in_sample_bce=0.023360 local_bce=0.023360 donor_bce=nan in_sample_local_ap=0.911409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=native_placebo_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan rows=1746 failures=44 prevalence=0.025200 ap=0.489551 auc=0.933207 f2_at20=0.370722 brier=0.017463 ece10=0.006769 rho=0.000000 donor_rows=0 effective_n=16585.000000 seconds=1.943569 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan in_sample_bce=0.023360 local_bce=0.023360 donor_bce=nan in_sample_local_ap=0.911409 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 318} positive_ess=318.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_duc model=committee_reference ap=0.446284 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_duc models=13 positive_rows=44 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_han_quoc fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_han_quoc fold=4 origin=6064 ap=0.069112 rows=3265 failures=12 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.026996 ap=0.042116 base_ap=0.069112 donor_failures=134 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.039678 ap=0.029434 base_ap=0.069112 donor_failures=104 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.037649 ap=0.031462 base_ap=0.069112 donor_failures=28 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6064 delta_ap=0.001031 ap=0.070143 base_ap=0.069112 donor_failures=59 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.037359 ap=0.031753 base_ap=0.069112 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6064 delta_ap=0.001191 ap=0.070303 base_ap=0.069112 donor_failures=56 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6064 delta_ap=-0.022020 ap=0.047092 base_ap=0.069112 donor_failures=121 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.019010 ap=0.050102 base_ap=0.069112 donor_failures=146 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.023628 ap=0.092739 base_ap=0.069112 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.040230 ap=0.028882 base_ap=0.069112 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.034784 ap=0.034328 base_ap=0.069112 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.129541 ap=0.198652 base_ap=0.069112 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.040747 ap=0.028365 base_ap=0.069112 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=4 origin=6066 ap=0.031590 rows=2904 failures=20 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.011541 ap=0.043131 base_ap=0.031590 donor_failures=126 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.008192 ap=0.039782 base_ap=0.031590 donor_failures=98 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.001300 ap=0.030290 base_ap=0.031590 donor_failures=36 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6066 delta_ap=0.010081 ap=0.041671 base_ap=0.031590 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.056230 ap=0.087820 base_ap=0.031590 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6066 delta_ap=0.014538 ap=0.046128 base_ap=0.031590 donor_failures=65 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6066 delta_ap=0.010752 ap=0.042342 base_ap=0.031590 donor_failures=137 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6066 delta_ap=0.015969 ap=0.047559 base_ap=0.031590 donor_failures=177 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6066 delta_ap=0.007647 ap=0.039237 base_ap=0.031590 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6066 delta_ap=0.010847 ap=0.042437 base_ap=0.031590 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.002120 ap=0.033710 base_ap=0.031590 donor_failures=90 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.017437 ap=0.049027 base_ap=0.031590 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.005825 ap=0.037415 base_ap=0.031590 donor_failures=263 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=4 origin=6067 ap=0.327482 rows=3337 failures=29 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.021922 ap=0.349404 base_ap=0.327482 donor_failures=143 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.015327 ap=0.342809 base_ap=0.327482 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.062119 ap=0.265363 base_ap=0.327482 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6067 delta_ap=-0.077822 ap=0.249660 base_ap=0.327482 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.095074 ap=0.422556 base_ap=0.327482 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6067 delta_ap=-0.041959 ap=0.285523 base_ap=0.327482 donor_failures=61 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6067 delta_ap=0.053471 ap=0.380953 base_ap=0.327482 donor_failures=148 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6067 delta_ap=-0.015973 ap=0.311509 base_ap=0.327482 donor_failures=182 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.060605 ap=0.266877 base_ap=0.327482 donor_failures=45 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6067 delta_ap=0.009968 ap=0.337450 base_ap=0.327482 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.027364 ap=0.354846 base_ap=0.327482 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.009085 ap=0.336567 base_ap=0.327482 donor_failures=31 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.039892 ap=0.287590 base_ap=0.327482 donor_failures=250 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=4 origin=6069 ap=0.251187 rows=3235 failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.080393 ap=0.170794 base_ap=0.251187 donor_failures=135 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.019317 ap=0.270505 base_ap=0.251187 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.110291 ap=0.140896 base_ap=0.251187 donor_failures=45 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6069 delta_ap=0.103239 ap=0.354426 base_ap=0.251187 donor_failures=79 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.051504 ap=0.199684 base_ap=0.251187 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6069 delta_ap=0.095498 ap=0.346686 base_ap=0.251187 donor_failures=80 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6069 delta_ap=0.071396 ap=0.322584 base_ap=0.251187 donor_failures=148 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6069 delta_ap=-0.060224 ap=0.190963 base_ap=0.251187 donor_failures=182 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6069 delta_ap=0.071311 ap=0.322499 base_ap=0.251187 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6069 delta_ap=-0.119023 ap=0.132164 base_ap=0.251187 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.011571 ap=0.239616 base_ap=0.251187 donor_failures=108 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.043432 ap=0.294620 base_ap=0.251187 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.042076 ap=0.209111 base_ap=0.251187 donor_failures=257 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_han_quoc origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_nhat,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.327482 policy_ap=0.326826 delta_ap=-0.000656 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_han_quoc origin=6069 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_bo_dao_nha,khoa_tieng_italia policy_donor_count=3 base_ap=0.251187 policy_ap=0.228541 delta_ap=-0.022646 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_han_quoc selected=[] reason=prequential_policy_delta_-0.011651_below_threshold replay_mean=-0.011651 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_local_common rows=3351 failures=38 prevalence=0.011340 ap=0.178304 auc=0.950546 f2_at20=0.212637 brier=0.010641 ece10=0.007506 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=0.657991 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=hgb_local_common in_sample_bce=0.007460 local_bce=0.007460 donor_bce=nan in_sample_local_ap=0.932719 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_local_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_local_native rows=3351 failures=38 prevalence=0.011340 ap=0.190218 auc=0.949128 f2_at20=0.212637 brier=0.010514 ece10=0.007161 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=1.223010 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=hgb_local_native in_sample_bce=0.007437 local_bce=0.007437 donor_bce=nan in_sample_local_ap=0.932099 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_local_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_pool_all rows=3351 failures=38 prevalence=0.011340 ap=0.209213 auc=0.922467 f2_at20=0.182260 brier=0.010024 ece10=0.004618 rho=0.250000 donor_rows=4095 effective_n=22523.709997 seconds=0.740998 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=hgb_pool_all in_sample_bce=0.014102 local_bce=0.011834 donor_bce=0.038989 in_sample_local_ap=0.786420 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_pool_all mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.025103375653856362, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03719097885115333, 'khoa_quoc_te_hoc': 0.012529214858698536, 'khoa_tieng_anh': 0.05120198275788266, 'khoa_tieng_bo_dao_nha': 0.004732165024356417, 'khoa_tieng_duc': 0.011832196129798156, 'khoa_tieng_italia': 0.01211756715493649, 'khoa_tieng_nga': 0.008389194711504162, 'khoa_tieng_nhat': 0.019204756564247007, 'khoa_tieng_phap': 0.01624331875087395, 'khoa_tieng_tay_ban_nha': 0.008621772096991904, 'khoa_tieng_trung_quoc': 0.03758193715559285, 'khoa_viet_nam_hoc': 0.005251540290108184} source_failures={'khoa_tieng_han_quoc': 166, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 4, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 20} positive_ess=176.084090 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_pool_equal rows=3351 failures=38 prevalence=0.011340 ap=0.274920 auc=0.925584 f2_at20=0.200486 brier=0.009492 ece10=0.004701 rho=0.250000 donor_rows=4095 effective_n=27452.438951 seconds=0.681963 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=hgb_pool_equal in_sample_bce=0.016601 local_bce=0.012986 donor_bce=0.027448 in_sample_local_ap=0.748032 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_pool_equal mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_han_quoc': 166, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 4, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 20} positive_ess=237.860291 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_language_prior rows=3351 failures=38 prevalence=0.011340 ap=0.311605 auc=0.922895 f2_at20=0.200486 brier=0.009203 ece10=0.003347 rho=0.250000 donor_rows=4095 effective_n=27452.438951 seconds=0.686664 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=hgb_language_prior in_sample_bce=0.015159 local_bce=0.012694 donor_bce=0.022553 in_sample_local_ap=0.761664 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_language_prior mass={'khoa_tieng_han_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_han_quoc': 166, 'khoa_tieng_anh': 11, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 10, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 24, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 4} positive_ess=221.562923 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_random_donor rows=3351 failures=38 prevalence=0.011340 ap=0.243171 auc=0.937261 f2_at20=0.206561 brier=0.009898 ece10=0.004745 rho=0.250000 donor_rows=4096 effective_n=27455.247439 seconds=0.679321 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=hgb_random_donor in_sample_bce=0.020596 local_bce=0.013459 donor_bce=0.042006 in_sample_local_ap=0.732830 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=hgb_random_donor mass={'khoa_tieng_han_quoc': 0.75, np.str_('khoa_tieng_italia'): 0.25} source_failures={'khoa_tieng_han_quoc': 166, np.str_('khoa_tieng_italia'): 143} positive_ess=270.552684 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_common rows=3351 failures=38 prevalence=0.011340 ap=0.178304 auc=0.950546 f2_at20=0.212637 brier=0.010641 ece10=0.007506 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=0.676554 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_common in_sample_bce=0.007460 local_bce=0.007460 donor_bce=nan in_sample_local_ap=0.932719 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_native rows=3351 failures=38 prevalence=0.011340 ap=0.190218 auc=0.949128 f2_at20=0.212637 brier=0.010514 ece10=0.007161 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=1.177103 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_native in_sample_bce=0.007437 local_bce=0.007437 donor_bce=nan in_sample_local_ap=0.932099 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator rows=3351 failures=38 prevalence=0.011340 ap=0.190218 auc=0.949128 f2_at20=0.212637 brier=0.010514 ece10=0.007161 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=2.148105 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator in_sample_bce=0.007437 local_bce=0.007437 donor_bce=nan in_sample_local_ap=0.932099 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator rows=3351 failures=38 prevalence=0.011340 ap=0.181759 auc=0.948822 f2_at20=0.218712 brier=0.010560 ece10=0.006991 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=2.205298 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator in_sample_bce=0.007457 local_bce=0.007457 donor_bce=nan in_sample_local_ap=0.930433 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=native_placebo_indicator rows=3351 failures=38 prevalence=0.011340 ap=0.190218 auc=0.949128 f2_at20=0.212637 brier=0.010514 ece10=0.007161 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=2.269064 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=native_placebo_indicator in_sample_bce=0.007437 local_bce=0.007437 donor_bce=nan in_sample_local_ap=0.932099 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=native_placebo_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan rows=3351 failures=38 prevalence=0.011340 ap=0.190218 auc=0.949128 f2_at20=0.212637 brier=0.010514 ece10=0.007161 rho=0.000000 donor_rows=0 effective_n=26578.000000 seconds=2.169855 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.007437 local_bce=0.007437 donor_bce=nan in_sample_local_ap=0.932099 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 166} positive_ess=166.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_han_quoc model=committee_reference ap=0.439642 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_han_quoc models=13 positive_rows=38 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_italia fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_italia fold=4 origin=6064 ap=0.288117 rows=2017 failures=65 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.055511 ap=0.343628 base_ap=0.288117 donor_failures=113 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.032828 ap=0.320945 base_ap=0.288117 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6064 delta_ap=0.037381 ap=0.325498 base_ap=0.288117 donor_failures=31 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_anh origin=6064 delta_ap=-0.025296 ap=0.262820 base_ap=0.288117 donor_failures=59 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.035802 ap=0.323919 base_ap=0.288117 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_duc origin=6064 delta_ap=0.019072 ap=0.307188 base_ap=0.288117 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6064 delta_ap=0.047644 ap=0.335761 base_ap=0.288117 donor_failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nga origin=6064 delta_ap=0.099629 ap=0.387746 base_ap=0.288117 donor_failures=149 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6064 delta_ap=0.000694 ap=0.288811 base_ap=0.288117 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_phap origin=6064 delta_ap=-0.039022 ap=0.249095 base_ap=0.288117 donor_failures=36 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.054211 ap=0.342328 base_ap=0.288117 donor_failures=69 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.040991 ap=0.329108 base_ap=0.288117 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6064 delta_ap=0.031925 ap=0.320041 base_ap=0.288117 donor_failures=275 [SCREEN_BASE] target=khoa_tieng_italia fold=4 origin=6066 ap=0.215273 rows=2166 failures=111 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.051770 ap=0.267044 base_ap=0.215273 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.018177 ap=0.233450 base_ap=0.215273 donor_failures=101 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6066 delta_ap=0.048541 ap=0.263814 base_ap=0.215273 donor_failures=34 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_anh origin=6066 delta_ap=0.037792 ap=0.253065 base_ap=0.215273 donor_failures=73 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.073881 ap=0.289154 base_ap=0.215273 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_duc origin=6066 delta_ap=0.009203 ap=0.224476 base_ap=0.215273 donor_failures=62 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6066 delta_ap=0.040038 ap=0.255311 base_ap=0.215273 donor_failures=20 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nga origin=6066 delta_ap=0.020122 ap=0.235395 base_ap=0.215273 donor_failures=165 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6066 delta_ap=0.027799 ap=0.243072 base_ap=0.215273 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_phap origin=6066 delta_ap=0.072316 ap=0.287589 base_ap=0.215273 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.003479 ap=0.218752 base_ap=0.215273 donor_failures=94 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.087409 ap=0.302682 base_ap=0.215273 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6066 delta_ap=0.011803 ap=0.227076 base_ap=0.215273 donor_failures=256 [SCREEN_BASE] target=khoa_tieng_italia fold=4 origin=6067 ap=0.524350 rows=2024 failures=71 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.019617 ap=0.504733 base_ap=0.524350 donor_failures=137 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.073224 ap=0.597574 base_ap=0.524350 donor_failures=102 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.039738 ap=0.484611 base_ap=0.524350 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_anh origin=6067 delta_ap=-0.010598 ap=0.513751 base_ap=0.524350 donor_failures=76 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.008840 ap=0.533189 base_ap=0.524350 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_duc origin=6067 delta_ap=-0.056988 ap=0.467361 base_ap=0.524350 donor_failures=73 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6067 delta_ap=0.004012 ap=0.528361 base_ap=0.524350 donor_failures=12 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nga origin=6067 delta_ap=-0.060690 ap=0.463660 base_ap=0.524350 donor_failures=190 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6067 delta_ap=-0.040101 ap=0.484249 base_ap=0.524350 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_phap origin=6067 delta_ap=-0.027364 ap=0.496986 base_ap=0.524350 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.008117 ap=0.516233 base_ap=0.524350 donor_failures=100 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.001496 ap=0.525845 base_ap=0.524350 donor_failures=25 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6067 delta_ap=0.072924 ap=0.597273 base_ap=0.524350 donor_failures=240 [SCREEN_BASE] target=khoa_tieng_italia fold=4 origin=6069 ap=0.222159 rows=2145 failures=68 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.050862 ap=0.273021 base_ap=0.222159 donor_failures=117 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.024704 ap=0.246863 base_ap=0.222159 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6069 delta_ap=0.026416 ap=0.248575 base_ap=0.222159 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_anh origin=6069 delta_ap=-0.004968 ap=0.217191 base_ap=0.222159 donor_failures=64 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.017692 ap=0.239850 base_ap=0.222159 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_duc origin=6069 delta_ap=-0.031984 ap=0.190175 base_ap=0.222159 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6069 delta_ap=0.002539 ap=0.224698 base_ap=0.222159 donor_failures=29 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nga origin=6069 delta_ap=0.036697 ap=0.258856 base_ap=0.222159 donor_failures=207 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6069 delta_ap=0.012875 ap=0.235034 base_ap=0.222159 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_phap origin=6069 delta_ap=0.093263 ap=0.315422 base_ap=0.222159 donor_failures=57 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.006194 ap=0.215965 base_ap=0.222159 donor_failures=109 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.011505 ap=0.210654 base_ap=0.222159 donor_failures=24 [TRANSFER_EDGE] fold=4 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6069 delta_ap=0.041148 ap=0.263307 base_ap=0.222159 donor_failures=242 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_italia origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_nga,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.524350 policy_ap=0.517918 delta_ap=-0.006432 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_italia origin=6069 policy_sources=khoa_tieng_trung_quoc,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.222159 policy_ap=0.243750 delta_ap=0.021591 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_italia selected=['khoa_viet_nam_hoc', 'khoa_quan_tri_kinh_doanh_va_du_lich', 'khoa_cong_nghe_thong_tin'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.007580 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=hgb_local_common rows=1795 failures=34 prevalence=0.018942 ap=0.396888 auc=0.911063 f2_at20=0.272727 brier=0.014954 ece10=0.003418 rho=0.000000 donor_rows=0 effective_n=16985.000000 seconds=0.489712 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=hgb_local_common in_sample_bce=0.043599 local_bce=0.043599 donor_bce=nan in_sample_local_ap=0.882560 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=hgb_local_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=hgb_local_native rows=1795 failures=34 prevalence=0.018942 ap=0.409088 auc=0.914136 f2_at20=0.292929 brier=0.014651 ece10=0.005131 rho=0.000000 donor_rows=0 effective_n=16985.000000 seconds=1.132646 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=hgb_local_native in_sample_bce=0.043316 local_bce=0.043316 donor_bce=nan in_sample_local_ap=0.883862 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=hgb_local_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=hgb_pool_all rows=1795 failures=34 prevalence=0.018942 ap=0.436188 auc=0.900850 f2_at20=0.272727 brier=0.014543 ece10=0.007617 rho=0.250000 donor_rows=4095 effective_n=17931.089368 seconds=0.552193 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=hgb_pool_all in_sample_bce=0.045029 local_bce=0.048452 donor_bce=0.052739 in_sample_local_ap=0.861178 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=hgb_pool_all mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.024434466437416324, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03619998111184565, 'khoa_quoc_te_hoc': 0.012195359069369525, 'khoa_tieng_anh': 0.04983764520268656, 'khoa_tieng_bo_dao_nha': 0.004606070875021527, 'khoa_tieng_duc': 0.011516913231152122, 'khoa_tieng_han_quoc': 0.018456226702294912, 'khoa_tieng_nga': 0.008165654668985094, 'khoa_tieng_nhat': 0.018693023049103644, 'khoa_tieng_phap': 0.015810496258478836, 'khoa_tieng_tay_ban_nha': 0.008392034754204003, 'khoa_tieng_trung_quoc': 0.036580521868594, 'khoa_viet_nam_hoc': 0.0051116067708478} source_failures={'khoa_tieng_italia': 576, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 17} positive_ess=588.765576 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=hgb_pool_equal rows=1795 failures=34 prevalence=0.018942 ap=0.416097 auc=0.904015 f2_at20=0.282828 brier=0.014456 ece10=0.007796 rho=0.250000 donor_rows=4095 effective_n=20669.710253 seconds=0.530418 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=hgb_pool_equal in_sample_bce=0.047957 local_bce=0.049464 donor_bce=0.043436 in_sample_local_ap=0.851264 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=hgb_pool_equal mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_italia': 576, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 10, 'khoa_tieng_duc': 3, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 17} positive_ess=661.343674 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=hgb_language_prior rows=1795 failures=34 prevalence=0.018942 ap=0.492559 auc=0.914479 f2_at20=0.272727 brier=0.013719 ece10=0.011005 rho=0.250000 donor_rows=4095 effective_n=20669.710253 seconds=0.577459 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=hgb_language_prior in_sample_bce=0.046344 local_bce=0.048920 donor_bce=0.038615 in_sample_local_ap=0.853877 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=hgb_language_prior mass={'khoa_tieng_italia': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_italia': 576, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 18, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_nga': 23, 'khoa_tieng_nhat': 8, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2} positive_ess=652.207244 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=hgb_random_donor rows=1795 failures=34 prevalence=0.018942 ap=0.465678 auc=0.905101 f2_at20=0.282828 brier=0.014238 ece10=0.013024 rho=0.250000 donor_rows=4095 effective_n=20669.710253 seconds=0.643999 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=hgb_random_donor in_sample_bce=0.046736 local_bce=0.048418 donor_bce=0.041691 in_sample_local_ap=0.859656 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=hgb_random_donor mass={'khoa_tieng_italia': 0.75, np.str_('khoa_tieng_nhat'): 0.08333333333333333, np.str_('khoa_tieng_anh'): 0.08333333333333333, np.str_('khoa_tieng_bo_dao_nha'): 0.08333333333333333} source_failures={'khoa_tieng_italia': 576, np.str_('khoa_tieng_nhat'): 15, np.str_('khoa_tieng_anh'): 26, np.str_('khoa_tieng_bo_dao_nha'): 41} positive_ess=648.565288 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=suite32_selective_common rows=1795 failures=34 prevalence=0.018942 ap=0.450048 auc=0.906011 f2_at20=0.282828 brier=0.014294 ece10=0.007797 rho=0.250000 donor_rows=4095 effective_n=20651.364658 seconds=0.595539 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=suite32_selective_common in_sample_bce=0.060532 local_bce=0.053216 donor_bce=0.081706 in_sample_local_ap=0.830081 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=suite32_selective_common mass={'khoa_tieng_italia': 0.75, 'khoa_viet_nam_hoc': 0.08859938820270633, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0836216325850764, 'khoa_cong_nghe_thong_tin': 0.07777897921221727} source_failures={'khoa_tieng_italia': 576, 'khoa_viet_nam_hoc': 92, 'khoa_quan_tri_kinh_doanh_va_du_lich': 48, 'khoa_cong_nghe_thong_tin': 46} positive_ess=742.637224 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=suite32_selective_native rows=1795 failures=34 prevalence=0.018942 ap=0.416998 auc=0.904508 f2_at20=0.303030 brier=0.014693 ece10=0.009386 rho=0.250000 donor_rows=4095 effective_n=20651.364658 seconds=1.322285 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=suite32_selective_native in_sample_bce=0.059842 local_bce=0.052568 donor_bce=0.080912 in_sample_local_ap=0.835129 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=suite32_selective_native mass={'khoa_tieng_italia': 0.75, 'khoa_viet_nam_hoc': 0.08859938820270633, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0836216325850764, 'khoa_cong_nghe_thong_tin': 0.07777897921221727} source_failures={'khoa_tieng_italia': 576, 'khoa_viet_nam_hoc': 92, 'khoa_quan_tri_kinh_doanh_va_du_lich': 48, 'khoa_cong_nghe_thong_tin': 46} positive_ess=742.637224 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=native_cache_nan_indicator rows=1795 failures=34 prevalence=0.018942 ap=0.409088 auc=0.914136 f2_at20=0.292929 brier=0.014651 ece10=0.005131 rho=0.000000 donor_rows=0 effective_n=16985.000000 seconds=2.171944 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=native_cache_nan_indicator in_sample_bce=0.043316 local_bce=0.043316 donor_bce=nan in_sample_local_ap=0.883862 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=native_cache_nan_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=native_naive_zero_indicator rows=1795 failures=34 prevalence=0.018942 ap=0.435766 auc=0.915523 f2_at20=0.292929 brier=0.014547 ece10=0.004515 rho=0.000000 donor_rows=0 effective_n=16985.000000 seconds=2.154434 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=native_naive_zero_indicator in_sample_bce=0.043215 local_bce=0.043215 donor_bce=nan in_sample_local_ap=0.886438 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=native_naive_zero_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=native_placebo_indicator rows=1795 failures=34 prevalence=0.018942 ap=0.409088 auc=0.914136 f2_at20=0.292929 brier=0.014651 ece10=0.005131 rho=0.000000 donor_rows=0 effective_n=16985.000000 seconds=2.085948 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=native_placebo_indicator in_sample_bce=0.043316 local_bce=0.043316 donor_bce=nan in_sample_local_ap=0.883862 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=native_placebo_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 576} positive_ess=576.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan rows=1795 failures=34 prevalence=0.018942 ap=0.416998 auc=0.904508 f2_at20=0.303030 brier=0.014693 ece10=0.009386 rho=0.250000 donor_rows=4095 effective_n=20651.364658 seconds=2.344142 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan in_sample_bce=0.059842 local_bce=0.052568 donor_bce=0.080912 in_sample_local_ap=0.835129 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan mass={'khoa_tieng_italia': 0.75, 'khoa_viet_nam_hoc': 0.08859938820270633, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0836216325850764, 'khoa_cong_nghe_thong_tin': 0.07777897921221727} source_failures={'khoa_tieng_italia': 576, 'khoa_viet_nam_hoc': 92, 'khoa_quan_tri_kinh_doanh_va_du_lich': 48, 'khoa_cong_nghe_thong_tin': 46} positive_ess=742.637224 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_italia model=committee_reference ap=0.397905 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_italia models=13 positive_rows=34 num_edges=52 num_replays=2 chosen=['khoa_viet_nam_hoc', 'khoa_quan_tri_kinh_doanh_va_du_lich', 'khoa_cong_nghe_thong_tin'] [SELECTION_ORIGINS] target=khoa_tieng_nga fold=4 origins=[6065, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nga fold=4 origin=6065 ap=0.297258 rows=45 failures=3 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6065 delta_ap=0.072186 ap=0.369444 base_ap=0.297258 donor_failures=114 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6065 delta_ap=0.089646 ap=0.386905 base_ap=0.297258 donor_failures=112 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6065 delta_ap=0.049964 ap=0.347222 base_ap=0.297258 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_anh origin=6065 delta_ap=0.020406 ap=0.317664 base_ap=0.297258 donor_failures=59 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6065 delta_ap=-0.012073 ap=0.285185 base_ap=0.297258 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_duc origin=6065 delta_ap=0.008297 ap=0.305556 base_ap=0.297258 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6065 delta_ap=0.230519 ap=0.527778 base_ap=0.297258 donor_failures=16 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_italia origin=6065 delta_ap=0.036075 ap=0.333333 base_ap=0.297258 donor_failures=121 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6065 delta_ap=0.075758 ap=0.373016 base_ap=0.297258 donor_failures=45 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_phap origin=6065 delta_ap=0.010317 ap=0.307576 base_ap=0.297258 donor_failures=49 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6065 delta_ap=0.064646 ap=0.361905 base_ap=0.297258 donor_failures=81 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6065 delta_ap=0.091631 ap=0.388889 base_ap=0.297258 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6065 delta_ap=0.091631 ap=0.388889 base_ap=0.297258 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_nga fold=4 origin=6066 ap=0.314411 rows=1588 failures=102 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.011204 ap=0.303208 base_ap=0.314411 donor_failures=116 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.024020 ap=0.338431 base_ap=0.314411 donor_failures=108 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6066 delta_ap=0.003377 ap=0.317788 base_ap=0.314411 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_anh origin=6066 delta_ap=0.029985 ap=0.344396 base_ap=0.314411 donor_failures=69 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.016882 ap=0.331293 base_ap=0.314411 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_duc origin=6066 delta_ap=-0.001804 ap=0.312607 base_ap=0.314411 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.016814 ap=0.297598 base_ap=0.314411 donor_failures=18 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_italia origin=6066 delta_ap=-0.030640 ap=0.283772 base_ap=0.314411 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6066 delta_ap=-0.003677 ap=0.310734 base_ap=0.314411 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_phap origin=6066 delta_ap=0.034000 ap=0.348411 base_ap=0.314411 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.017278 ap=0.297133 base_ap=0.314411 donor_failures=89 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.000896 ap=0.313516 base_ap=0.314411 donor_failures=26 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6066 delta_ap=0.004014 ap=0.318425 base_ap=0.314411 donor_failures=262 [SCREEN_BASE] target=khoa_tieng_nga fold=4 origin=6067 ap=0.792868 rows=1567 failures=83 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.007863 ap=0.800731 base_ap=0.792868 donor_failures=110 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.010473 ap=0.782395 base_ap=0.792868 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.007374 ap=0.785494 base_ap=0.792868 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_anh origin=6067 delta_ap=0.018654 ap=0.811522 base_ap=0.792868 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.000693 ap=0.792175 base_ap=0.792868 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_duc origin=6067 delta_ap=-0.041672 ap=0.751196 base_ap=0.792868 donor_failures=60 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6067 delta_ap=0.019004 ap=0.811872 base_ap=0.792868 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_italia origin=6067 delta_ap=-0.009315 ap=0.783553 base_ap=0.792868 donor_failures=148 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6067 delta_ap=0.009486 ap=0.802354 base_ap=0.792868 donor_failures=36 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_phap origin=6067 delta_ap=0.069250 ap=0.862118 base_ap=0.792868 donor_failures=56 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.009532 ap=0.802400 base_ap=0.792868 donor_failures=91 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.026500 ap=0.766368 base_ap=0.792868 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.022265 ap=0.770603 base_ap=0.792868 donor_failures=254 [SCREEN_BASE] target=khoa_tieng_nga fold=4 origin=6069 ap=0.439684 rows=1996 failures=92 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.013985 ap=0.453669 base_ap=0.439684 donor_failures=151 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.053520 ap=0.493204 base_ap=0.439684 donor_failures=110 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6069 delta_ap=0.015473 ap=0.455157 base_ap=0.439684 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_anh origin=6069 delta_ap=0.034053 ap=0.473737 base_ap=0.439684 donor_failures=75 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.012224 ap=0.451908 base_ap=0.439684 donor_failures=143 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_duc origin=6069 delta_ap=0.010492 ap=0.450176 base_ap=0.439684 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.001968 ap=0.437716 base_ap=0.439684 donor_failures=26 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_italia origin=6069 delta_ap=-0.003166 ap=0.436517 base_ap=0.439684 donor_failures=149 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6069 delta_ap=-0.012811 ap=0.426873 base_ap=0.439684 donor_failures=30 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_phap origin=6069 delta_ap=-0.034245 ap=0.405439 base_ap=0.439684 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.021263 ap=0.460946 base_ap=0.439684 donor_failures=102 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.034513 ap=0.474197 base_ap=0.439684 donor_failures=26 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6069 delta_ap=0.021347 ap=0.461031 base_ap=0.439684 donor_failures=255 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_nga origin=6067 policy_sources=khoa_tieng_han_quoc,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.792868 policy_ap=0.776572 delta_ap=-0.016296 policy_donor_rows=4095 selected_using_origins=6065,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_nga origin=6069 policy_sources=khoa_tieng_han_quoc,khoa_tieng_phap,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.439684 policy_ap=0.452411 delta_ap=0.012727 policy_donor_rows=4095 selected_using_origins=6065,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_nga selected=[] reason=prequential_policy_delta_-0.001784_below_threshold replay_mean=-0.001784 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=hgb_local_common rows=1659 failures=114 prevalence=0.068716 ap=0.599190 auc=0.922177 f2_at20=0.583756 brier=0.041555 ece10=0.021252 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=0.505587 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=hgb_local_common in_sample_bce=0.052724 local_bce=0.052724 donor_bce=nan in_sample_local_ap=0.902483 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=hgb_local_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=hgb_local_native rows=1659 failures=114 prevalence=0.068716 ap=0.617303 auc=0.929118 f2_at20=0.602792 brier=0.041116 ece10=0.022753 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=1.127088 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=hgb_local_native in_sample_bce=0.052554 local_bce=0.052554 donor_bce=nan in_sample_local_ap=0.904220 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=hgb_local_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=hgb_pool_all rows=1659 failures=114 prevalence=0.068716 ap=0.594258 auc=0.924630 f2_at20=0.590102 brier=0.041669 ece10=0.016566 rho=0.250000 donor_rows=4095 effective_n=14272.023612 seconds=0.578076 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=hgb_pool_all in_sample_bce=0.053898 local_bce=0.058421 donor_bce=0.056166 in_sample_local_ap=0.877989 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=hgb_pool_all mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.024084848318913592, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03568201730369072, 'khoa_quoc_te_hoc': 0.012020862994195598, 'khoa_tieng_anh': 0.04912454824225167, 'khoa_tieng_bo_dao_nha': 0.004540165370715147, 'khoa_tieng_duc': 0.011352124630380024, 'khoa_tieng_han_quoc': 0.018192147628956302, 'khoa_tieng_italia': 0.011625917205125397, 'khoa_tieng_nhat': 0.018425555798926734, 'khoa_tieng_phap': 0.015584273354506626, 'khoa_tieng_tay_ban_nha': 0.008271958164494579, 'khoa_tieng_trung_quoc': 0.03605711313109188, 'khoa_viet_nam_hoc': 0.005038467856751725} source_failures={'khoa_tieng_nga': 547, 'khoa_cong_nghe_thong_tin': 15, 'khoa_quan_tri_kinh_doanh_va_du_lich': 18, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 10, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 14} positive_ess=606.222539 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=hgb_pool_equal rows=1659 failures=114 prevalence=0.068716 ap=0.599135 auc=0.929336 f2_at20=0.590102 brier=0.042113 ece10=0.018288 rho=0.250000 donor_rows=4095 effective_n=15848.308718 seconds=0.571501 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=hgb_pool_equal in_sample_bce=0.057364 local_bce=0.059239 donor_bce=0.051739 in_sample_local_ap=0.875683 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=hgb_pool_equal mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nga': 547, 'khoa_cong_nghe_thong_tin': 15, 'khoa_quan_tri_kinh_doanh_va_du_lich': 18, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 10, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 14} positive_ess=645.844323 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=hgb_language_prior rows=1659 failures=114 prevalence=0.068716 ap=0.565082 auc=0.918026 f2_at20=0.609137 brier=0.043229 ece10=0.019412 rho=0.250000 donor_rows=4095 effective_n=15848.308718 seconds=0.575528 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=hgb_language_prior in_sample_bce=0.056004 local_bce=0.058577 donor_bce=0.048286 in_sample_local_ap=0.881746 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=hgb_language_prior mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nga': 547, 'khoa_tieng_anh': 12, 'khoa_tieng_bo_dao_nha': 21, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 11, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 12, 'khoa_tieng_tay_ban_nha': 14, 'khoa_tieng_trung_quoc': 4} positive_ess=634.859398 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=hgb_random_donor rows=1659 failures=114 prevalence=0.068716 ap=0.595494 auc=0.922742 f2_at20=0.602792 brier=0.042518 ece10=0.024045 rho=0.250000 donor_rows=4096 effective_n=15849.244678 seconds=0.569220 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=hgb_random_donor in_sample_bce=0.046923 local_bce=0.055320 donor_bce=0.021733 in_sample_local_ap=0.892343 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=hgb_random_donor mass={'khoa_tieng_nga': 0.75, np.str_('khoa_quoc_te_hoc'): 0.25} source_failures={'khoa_tieng_nga': 547, np.str_('khoa_quoc_te_hoc'): 28} positive_ess=574.950431 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=suite32_selective_common rows=1659 failures=114 prevalence=0.068716 ap=0.599190 auc=0.922177 f2_at20=0.583756 brier=0.041555 ece10=0.021252 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=0.529014 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=suite32_selective_common in_sample_bce=0.052724 local_bce=0.052724 donor_bce=nan in_sample_local_ap=0.902483 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=suite32_selective_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=suite32_selective_native rows=1659 failures=114 prevalence=0.068716 ap=0.617303 auc=0.929118 f2_at20=0.602792 brier=0.041116 ece10=0.022753 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=1.108703 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=suite32_selective_native in_sample_bce=0.052554 local_bce=0.052554 donor_bce=nan in_sample_local_ap=0.904220 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=suite32_selective_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=native_cache_nan_indicator rows=1659 failures=114 prevalence=0.068716 ap=0.617303 auc=0.929118 f2_at20=0.602792 brier=0.041116 ece10=0.022753 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=2.109772 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=native_cache_nan_indicator in_sample_bce=0.052554 local_bce=0.052554 donor_bce=nan in_sample_local_ap=0.904220 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=native_cache_nan_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=native_naive_zero_indicator rows=1659 failures=114 prevalence=0.068716 ap=0.620052 auc=0.929844 f2_at20=0.596447 brier=0.040782 ece10=0.021678 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=2.065857 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=native_naive_zero_indicator in_sample_bce=0.052490 local_bce=0.052490 donor_bce=nan in_sample_local_ap=0.902895 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=native_naive_zero_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=native_placebo_indicator rows=1659 failures=114 prevalence=0.068716 ap=0.617303 auc=0.929118 f2_at20=0.602792 brier=0.041116 ece10=0.022753 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=2.085533 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=native_placebo_indicator in_sample_bce=0.052554 local_bce=0.052554 donor_bce=nan in_sample_local_ap=0.904220 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=native_placebo_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan rows=1659 failures=114 prevalence=0.068716 ap=0.617303 auc=0.929118 f2_at20=0.602792 brier=0.041116 ece10=0.022753 rho=0.000000 donor_rows=0 effective_n=11759.000000 seconds=2.114969 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan in_sample_bce=0.052554 local_bce=0.052554 donor_bce=nan in_sample_local_ap=0.904220 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 547} positive_ess=547.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nga model=committee_reference ap=0.617681 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_nga models=13 positive_rows=114 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nhat fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nhat fold=4 origin=6064 ap=0.196698 rows=2945 failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.098292 ap=0.098406 base_ap=0.196698 donor_failures=126 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.073157 ap=0.123542 base_ap=0.196698 donor_failures=82 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.118469 ap=0.078229 base_ap=0.196698 donor_failures=28 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6064 delta_ap=-0.150499 ap=0.046199 base_ap=0.196698 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.009778 ap=0.186920 base_ap=0.196698 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6064 delta_ap=0.025081 ap=0.221779 base_ap=0.196698 donor_failures=61 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.117658 ap=0.079040 base_ap=0.196698 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6064 delta_ap=-0.117696 ap=0.079002 base_ap=0.196698 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6064 delta_ap=-0.085966 ap=0.110733 base_ap=0.196698 donor_failures=142 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6064 delta_ap=-0.074097 ap=0.122601 base_ap=0.196698 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.104434 ap=0.092265 base_ap=0.196698 donor_failures=77 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.011903 ap=0.184796 base_ap=0.196698 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.144772 ap=0.051926 base_ap=0.196698 donor_failures=271 [SCREEN_BASE] target=khoa_tieng_nhat fold=4 origin=6066 ap=0.330934 rows=2840 failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.198600 ap=0.529534 base_ap=0.330934 donor_failures=109 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.038696 ap=0.292238 base_ap=0.330934 donor_failures=104 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.221204 ap=0.109730 base_ap=0.330934 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6066 delta_ap=-0.162830 ap=0.168104 base_ap=0.330934 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.129840 ap=0.201094 base_ap=0.330934 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6066 delta_ap=0.025491 ap=0.356425 base_ap=0.330934 donor_failures=72 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6066 delta_ap=0.181147 ap=0.512081 base_ap=0.330934 donor_failures=11 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6066 delta_ap=0.130642 ap=0.461576 base_ap=0.330934 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6066 delta_ap=-0.235644 ap=0.095290 base_ap=0.330934 donor_failures=161 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6066 delta_ap=-0.024193 ap=0.306741 base_ap=0.330934 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.064127 ap=0.395061 base_ap=0.330934 donor_failures=83 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.216131 ap=0.547065 base_ap=0.330934 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6066 delta_ap=0.194752 ap=0.525686 base_ap=0.330934 donor_failures=270 [SCREEN_BASE] target=khoa_tieng_nhat fold=4 origin=6067 ap=0.444125 rows=2448 failures=11 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.020571 ap=0.464696 base_ap=0.444125 donor_failures=138 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.079415 ap=0.523540 base_ap=0.444125 donor_failures=116 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6067 delta_ap=0.072904 ap=0.517029 base_ap=0.444125 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6067 delta_ap=-0.127780 ap=0.316345 base_ap=0.444125 donor_failures=62 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.023486 ap=0.420639 base_ap=0.444125 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6067 delta_ap=-0.161515 ap=0.282610 base_ap=0.444125 donor_failures=68 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6067 delta_ap=0.009740 ap=0.453865 base_ap=0.444125 donor_failures=16 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6067 delta_ap=-0.007489 ap=0.436636 base_ap=0.444125 donor_failures=144 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6067 delta_ap=0.160782 ap=0.604907 base_ap=0.444125 donor_failures=192 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6067 delta_ap=-0.124546 ap=0.319579 base_ap=0.444125 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.052172 ap=0.496297 base_ap=0.444125 donor_failures=107 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.191806 ap=0.252319 base_ap=0.444125 donor_failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6067 delta_ap=0.002618 ap=0.446743 base_ap=0.444125 donor_failures=253 [SCREEN_BASE] target=khoa_tieng_nhat fold=4 origin=6069 ap=0.200086 rows=2588 failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.070793 ap=0.270879 base_ap=0.200086 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.119241 ap=0.319328 base_ap=0.200086 donor_failures=115 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.015898 ap=0.184189 base_ap=0.200086 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6069 delta_ap=-0.068759 ap=0.131327 base_ap=0.200086 donor_failures=58 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.026513 ap=0.226599 base_ap=0.200086 donor_failures=117 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6069 delta_ap=0.074464 ap=0.274550 base_ap=0.200086 donor_failures=85 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.104598 ap=0.095488 base_ap=0.200086 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6069 delta_ap=0.051483 ap=0.251570 base_ap=0.200086 donor_failures=152 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6069 delta_ap=-0.004774 ap=0.195313 base_ap=0.200086 donor_failures=199 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6069 delta_ap=0.114286 ap=0.314373 base_ap=0.200086 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.050964 ap=0.251050 base_ap=0.200086 donor_failures=104 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.008229 ap=0.191857 base_ap=0.200086 donor_failures=30 [TRANSFER_EDGE] fold=4 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6069 delta_ap=0.028506 ap=0.228592 base_ap=0.200086 donor_failures=235 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_nhat origin=6067 policy_sources=khoa_tieng_trung_quoc,khoa_cong_nghe_thong_tin,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.444125 policy_ap=0.304029 delta_ap=-0.140096 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_nhat origin=6069 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_han_quoc,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.200086 policy_ap=0.398042 delta_ap=0.197956 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_nhat selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.028930 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=hgb_local_common rows=2413 failures=38 prevalence=0.015748 ap=0.214668 auc=0.862504 f2_at20=0.236220 brier=0.013906 ece10=0.010723 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=0.575879 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=hgb_local_common in_sample_bce=0.007992 local_bce=0.007992 donor_bce=nan in_sample_local_ap=0.942798 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=hgb_local_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=hgb_local_native rows=2413 failures=38 prevalence=0.015748 ap=0.221784 auc=0.869507 f2_at20=0.236220 brier=0.013928 ece10=0.010833 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=1.259719 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=hgb_local_native in_sample_bce=0.007748 local_bce=0.007748 donor_bce=nan in_sample_local_ap=0.948244 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=hgb_local_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=hgb_pool_all rows=2413 failures=38 prevalence=0.015748 ap=0.217318 auc=0.857662 f2_at20=0.196850 brier=0.013994 ece10=0.010085 rho=0.250000 donor_rows=4095 effective_n=22652.462952 seconds=0.657999 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=hgb_pool_all in_sample_bce=0.014148 local_bce=0.011756 donor_bce=0.039671 in_sample_local_ap=0.876177 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=hgb_pool_all mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.025127827925045705, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03722720521023766, 'khoa_quoc_te_hoc': 0.012541419104204754, 'khoa_tieng_anh': 0.05125185671846435, 'khoa_tieng_bo_dao_nha': 0.0047367744515539306, 'khoa_tieng_duc': 0.011843721435100549, 'khoa_tieng_han_quoc': 0.01897994744058501, 'khoa_tieng_italia': 0.012129370429616088, 'khoa_tieng_nga': 0.008397366316270567, 'khoa_tieng_phap': 0.016259140767824497, 'khoa_tieng_tay_ban_nha': 0.00863017024680073, 'khoa_tieng_trung_quoc': 0.03761854433272395, 'khoa_viet_nam_hoc': 0.005256655621572212} source_failures={'khoa_tieng_nhat': 244, 'khoa_cong_nghe_thong_tin': 2, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 13, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 19} positive_ess=225.707804 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=hgb_pool_equal rows=2413 failures=38 prevalence=0.015748 ap=0.189529 auc=0.862172 f2_at20=0.212598 brier=0.014261 ece10=0.012513 rho=0.250000 donor_rows=4095 effective_n=27655.986452 seconds=0.738126 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=hgb_pool_equal in_sample_bce=0.016836 local_bce=0.012593 donor_bce=0.029563 in_sample_local_ap=0.864827 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=hgb_pool_equal mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nhat': 244, 'khoa_cong_nghe_thong_tin': 2, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 13, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 19} positive_ess=298.713405 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=hgb_language_prior rows=2413 failures=38 prevalence=0.015748 ap=0.199986 auc=0.904488 f2_at20=0.275591 brier=0.014177 ece10=0.011034 rho=0.250000 donor_rows=4095 effective_n=27655.986452 seconds=0.660973 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=hgb_language_prior in_sample_bce=0.015624 local_bce=0.012183 donor_bce=0.025947 in_sample_local_ap=0.874736 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=hgb_language_prior mass={'khoa_tieng_nhat': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nhat': 244, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 25, 'khoa_tieng_nga': 17, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 13, 'khoa_tieng_trung_quoc': 3} positive_ess=294.513837 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=hgb_random_donor rows=2413 failures=38 prevalence=0.015748 ap=0.247271 auc=0.878332 f2_at20=0.228346 brier=0.013737 ece10=0.010865 rho=0.250000 donor_rows=4096 effective_n=27658.836743 seconds=0.598895 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=hgb_random_donor in_sample_bce=0.019890 local_bce=0.012651 donor_bce=0.041605 in_sample_local_ap=0.858852 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=hgb_random_donor mass={'khoa_tieng_nhat': 0.75, np.str_('khoa_tieng_bo_dao_nha'): 0.25} source_failures={'khoa_tieng_nhat': 244, np.str_('khoa_tieng_bo_dao_nha'): 151} positive_ess=341.075957 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_common rows=2413 failures=38 prevalence=0.015748 ap=0.214668 auc=0.862504 f2_at20=0.236220 brier=0.013906 ece10=0.010723 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=0.688830 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_common in_sample_bce=0.007992 local_bce=0.007992 donor_bce=nan in_sample_local_ap=0.942798 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_native rows=2413 failures=38 prevalence=0.015748 ap=0.221784 auc=0.869507 f2_at20=0.236220 brier=0.013928 ece10=0.010833 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=1.350038 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_native in_sample_bce=0.007748 local_bce=0.007748 donor_bce=nan in_sample_local_ap=0.948244 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=native_cache_nan_indicator rows=2413 failures=38 prevalence=0.015748 ap=0.221784 auc=0.869507 f2_at20=0.236220 brier=0.013928 ece10=0.010833 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=2.420552 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=native_cache_nan_indicator in_sample_bce=0.007748 local_bce=0.007748 donor_bce=nan in_sample_local_ap=0.948244 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=native_cache_nan_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=native_naive_zero_indicator rows=2413 failures=38 prevalence=0.015748 ap=0.221784 auc=0.869507 f2_at20=0.236220 brier=0.013928 ece10=0.010833 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=2.318694 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=native_naive_zero_indicator in_sample_bce=0.007748 local_bce=0.007748 donor_bce=nan in_sample_local_ap=0.948244 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=native_naive_zero_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=native_placebo_indicator rows=2413 failures=38 prevalence=0.015748 ap=0.221784 auc=0.869507 f2_at20=0.236220 brier=0.013928 ece10=0.010833 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=2.239484 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=native_placebo_indicator in_sample_bce=0.007748 local_bce=0.007748 donor_bce=nan in_sample_local_ap=0.948244 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=native_placebo_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan rows=2413 failures=38 prevalence=0.015748 ap=0.221784 auc=0.869507 f2_at20=0.236220 brier=0.013928 ece10=0.010833 rho=0.000000 donor_rows=0 effective_n=26919.000000 seconds=2.304309 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan in_sample_bce=0.007748 local_bce=0.007748 donor_bce=nan in_sample_local_ap=0.948244 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 244} positive_ess=244.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_nhat model=committee_reference ap=0.166735 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_nhat models=13 positive_rows=38 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_phap fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_phap fold=4 origin=6064 ap=0.458296 rows=2471 failures=32 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.099466 ap=0.358830 base_ap=0.458296 donor_failures=125 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.069044 ap=0.389252 base_ap=0.458296 donor_failures=98 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.058760 ap=0.399536 base_ap=0.458296 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_anh origin=6064 delta_ap=-0.052639 ap=0.405657 base_ap=0.458296 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.010815 ap=0.469111 base_ap=0.458296 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_duc origin=6064 delta_ap=-0.051188 ap=0.407108 base_ap=0.458296 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6064 delta_ap=0.023845 ap=0.482141 base_ap=0.458296 donor_failures=13 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_italia origin=6064 delta_ap=-0.008256 ap=0.450039 base_ap=0.458296 donor_failures=112 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nga origin=6064 delta_ap=-0.008978 ap=0.449317 base_ap=0.458296 donor_failures=148 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6064 delta_ap=-0.007483 ap=0.450812 base_ap=0.458296 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.046901 ap=0.411395 base_ap=0.458296 donor_failures=85 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.077135 ap=0.381160 base_ap=0.458296 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.081207 ap=0.377089 base_ap=0.458296 donor_failures=269 [SCREEN_BASE] target=khoa_tieng_phap fold=4 origin=6066 ap=0.237344 rows=2606 failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.033692 ap=0.271036 base_ap=0.237344 donor_failures=136 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.006688 ap=0.244033 base_ap=0.237344 donor_failures=109 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6066 delta_ap=0.057273 ap=0.294617 base_ap=0.237344 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_anh origin=6066 delta_ap=-0.025445 ap=0.211899 base_ap=0.237344 donor_failures=78 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.032789 ap=0.270134 base_ap=0.237344 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_duc origin=6066 delta_ap=0.021161 ap=0.258506 base_ap=0.237344 donor_failures=62 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6066 delta_ap=0.061391 ap=0.298736 base_ap=0.237344 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_italia origin=6066 delta_ap=-0.000836 ap=0.236508 base_ap=0.237344 donor_failures=117 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nga origin=6066 delta_ap=0.028371 ap=0.265716 base_ap=0.237344 donor_failures=175 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6066 delta_ap=0.013830 ap=0.251174 base_ap=0.237344 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.007215 ap=0.244559 base_ap=0.237344 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.066412 ap=0.303757 base_ap=0.237344 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6066 delta_ap=0.030567 ap=0.267911 base_ap=0.237344 donor_failures=250 [SCREEN_BASE] target=khoa_tieng_phap fold=4 origin=6067 ap=0.499981 rows=2204 failures=40 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.004262 ap=0.495719 base_ap=0.499981 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.032010 ap=0.531991 base_ap=0.499981 donor_failures=125 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6067 delta_ap=0.091804 ap=0.591785 base_ap=0.499981 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_anh origin=6067 delta_ap=-0.017620 ap=0.482361 base_ap=0.499981 donor_failures=68 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.015904 ap=0.484077 base_ap=0.499981 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_duc origin=6067 delta_ap=-0.001038 ap=0.498943 base_ap=0.499981 donor_failures=74 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.023678 ap=0.476303 base_ap=0.499981 donor_failures=19 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_italia origin=6067 delta_ap=0.099192 ap=0.599173 base_ap=0.499981 donor_failures=125 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nga origin=6067 delta_ap=-0.048233 ap=0.451748 base_ap=0.499981 donor_failures=194 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6067 delta_ap=0.077737 ap=0.577718 base_ap=0.499981 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.063517 ap=0.563498 base_ap=0.499981 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.080865 ap=0.580845 base_ap=0.499981 donor_failures=26 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.012604 ap=0.487376 base_ap=0.499981 donor_failures=238 [SCREEN_BASE] target=khoa_tieng_phap fold=4 origin=6069 ap=0.121402 rows=2693 failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.006259 ap=0.115143 base_ap=0.121402 donor_failures=158 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.053386 ap=0.174788 base_ap=0.121402 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.022582 ap=0.098820 base_ap=0.121402 donor_failures=45 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_anh origin=6069 delta_ap=-0.033513 ap=0.087889 base_ap=0.121402 donor_failures=64 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.003496 ap=0.124898 base_ap=0.121402 donor_failures=121 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_duc origin=6069 delta_ap=0.053595 ap=0.174997 base_ap=0.121402 donor_failures=74 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.028434 ap=0.092968 base_ap=0.121402 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_italia origin=6069 delta_ap=-0.022573 ap=0.098829 base_ap=0.121402 donor_failures=151 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nga origin=6069 delta_ap=0.063302 ap=0.184704 base_ap=0.121402 donor_failures=188 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6069 delta_ap=-0.028561 ap=0.092840 base_ap=0.121402 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.029712 ap=0.151114 base_ap=0.121402 donor_failures=98 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.014227 ap=0.135629 base_ap=0.121402 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6069 delta_ap=0.075602 ap=0.197004 base_ap=0.121402 donor_failures=242 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_phap origin=6067 policy_sources=khoa_tieng_han_quoc,khoa_tieng_bo_dao_nha,khoa_tieng_nga policy_donor_count=3 base_ap=0.499981 policy_ap=0.511083 delta_ap=0.011102 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_phap origin=6069 policy_sources=khoa_quoc_te_hoc,khoa_tieng_nhat,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.121402 policy_ap=0.099414 delta_ap=-0.021988 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_phap selected=[] reason=prequential_policy_delta_-0.005443_below_threshold replay_mean=-0.005443 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=hgb_local_common rows=2275 failures=19 prevalence=0.008352 ap=0.297011 auc=0.864513 f2_at20=0.122411 brier=0.007366 ece10=0.007134 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=0.620187 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=hgb_local_common in_sample_bce=0.014834 local_bce=0.014834 donor_bce=nan in_sample_local_ap=0.914995 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=hgb_local_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=hgb_local_native rows=2275 failures=19 prevalence=0.008352 ap=0.313534 auc=0.871396 f2_at20=0.141243 brier=0.007275 ece10=0.006315 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=1.653392 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=hgb_local_native in_sample_bce=0.014632 local_bce=0.014632 donor_bce=nan in_sample_local_ap=0.917259 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=hgb_local_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=hgb_pool_all rows=2275 failures=19 prevalence=0.008352 ap=0.287002 auc=0.866216 f2_at20=0.131827 brier=0.007825 ece10=0.008681 rho=0.250000 donor_rows=4095 effective_n=20947.035087 seconds=0.670515 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=hgb_pool_all in_sample_bce=0.019386 local_bce=0.018618 donor_bce=0.041190 in_sample_local_ap=0.868456 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=hgb_pool_all mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.02483337144405769, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03679096408840559, 'khoa_quoc_te_hoc': 0.012394454466153441, 'khoa_tieng_anh': 0.0506512699340261, 'khoa_tieng_bo_dao_nha': 0.004681267308620646, 'khoa_tieng_duc': 0.011704932656938552, 'khoa_tieng_han_quoc': 0.01875753392560222, 'khoa_tieng_italia': 0.011987234318848435, 'khoa_tieng_nga': 0.008298963105995805, 'khoa_tieng_nhat': 0.018998196092380397, 'khoa_tieng_tay_ban_nha': 0.008529038960452361, 'khoa_tieng_trung_quoc': 0.03717771736522213, 'khoa_viet_nam_hoc': 0.005195056333296634} source_failures={'khoa_tieng_phap': 303, 'khoa_cong_nghe_thong_tin': 2, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 5, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 21} positive_ess=318.802925 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=hgb_pool_equal rows=2275 failures=19 prevalence=0.008352 ap=0.260356 auc=0.849536 f2_at20=0.122411 brier=0.008513 ece10=0.010714 rho=0.250000 donor_rows=4095 effective_n=25019.864146 seconds=0.632269 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=hgb_pool_equal in_sample_bce=0.022434 local_bce=0.019800 donor_bce=0.030337 in_sample_local_ap=0.851329 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=hgb_pool_equal mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_phap': 303, 'khoa_cong_nghe_thong_tin': 2, 'khoa_quan_tri_kinh_doanh_va_du_lich': 9, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 15, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 5, 'khoa_tieng_tay_ban_nha': 10, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 21} positive_ess=376.498210 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=hgb_language_prior rows=2275 failures=19 prevalence=0.008352 ap=0.228410 auc=0.869693 f2_at20=0.141243 brier=0.008293 ece10=0.007464 rho=0.250000 donor_rows=4095 effective_n=25019.864146 seconds=0.636810 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=hgb_language_prior in_sample_bce=0.021304 local_bce=0.019797 donor_bce=0.025825 in_sample_local_ap=0.852033 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=hgb_language_prior mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_phap': 303, 'khoa_tieng_anh': 10, 'khoa_tieng_bo_dao_nha': 18, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 22, 'khoa_tieng_nhat': 5, 'khoa_tieng_tay_ban_nha': 14, 'khoa_tieng_trung_quoc': 1} positive_ess=361.384000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=hgb_random_donor rows=2275 failures=19 prevalence=0.008352 ap=0.300875 auc=0.868549 f2_at20=0.131827 brier=0.007443 ece10=0.009452 rho=0.250000 donor_rows=4096 effective_n=25022.196941 seconds=0.651217 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=hgb_random_donor in_sample_bce=0.014566 local_bce=0.017172 donor_bce=0.006747 in_sample_local_ap=0.889573 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=hgb_random_donor mass={'khoa_tieng_phap': 0.75, np.str_('khoa_tieng_trung_quoc'): 0.25} source_failures={'khoa_tieng_phap': 303, np.str_('khoa_tieng_trung_quoc'): 24} positive_ess=313.275124 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=suite32_selective_common rows=2275 failures=19 prevalence=0.008352 ap=0.297011 auc=0.864513 f2_at20=0.122411 brier=0.007366 ece10=0.007134 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=0.605706 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=suite32_selective_common in_sample_bce=0.014834 local_bce=0.014834 donor_bce=nan in_sample_local_ap=0.914995 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=suite32_selective_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=suite32_selective_native rows=2275 failures=19 prevalence=0.008352 ap=0.313534 auc=0.871396 f2_at20=0.141243 brier=0.007275 ece10=0.006315 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=1.680171 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=suite32_selective_native in_sample_bce=0.014632 local_bce=0.014632 donor_bce=nan in_sample_local_ap=0.917259 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=suite32_selective_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=native_cache_nan_indicator rows=2275 failures=19 prevalence=0.008352 ap=0.313534 auc=0.871396 f2_at20=0.141243 brier=0.007275 ece10=0.006315 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=3.838334 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=native_cache_nan_indicator in_sample_bce=0.014632 local_bce=0.014632 donor_bce=nan in_sample_local_ap=0.917259 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=native_cache_nan_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=native_naive_zero_indicator rows=2275 failures=19 prevalence=0.008352 ap=0.308363 auc=0.872492 f2_at20=0.131827 brier=0.007318 ece10=0.006643 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=3.791423 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=native_naive_zero_indicator in_sample_bce=0.014588 local_bce=0.014588 donor_bce=nan in_sample_local_ap=0.918774 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=native_naive_zero_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=native_placebo_indicator rows=2275 failures=19 prevalence=0.008352 ap=0.313534 auc=0.871396 f2_at20=0.141243 brier=0.007275 ece10=0.006315 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=3.728855 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=native_placebo_indicator in_sample_bce=0.014632 local_bce=0.014632 donor_bce=nan in_sample_local_ap=0.917259 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=native_placebo_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan rows=2275 failures=19 prevalence=0.008352 ap=0.313534 auc=0.871396 f2_at20=0.141243 brier=0.007275 ece10=0.006315 rho=0.000000 donor_rows=0 effective_n=22768.000000 seconds=3.700677 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan in_sample_bce=0.014632 local_bce=0.014632 donor_bce=nan in_sample_local_ap=0.917259 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 303} positive_ess=303.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_phap model=committee_reference ap=0.294413 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_phap models=13 positive_rows=19 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_tay_ban_nha fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=4 origin=6064 ap=0.314541 rows=1388 failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=0.026403 ap=0.340944 base_ap=0.314541 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.045382 ap=0.359923 base_ap=0.314541 donor_failures=87 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.002037 ap=0.312504 base_ap=0.314541 donor_failures=31 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6064 delta_ap=0.053770 ap=0.368311 base_ap=0.314541 donor_failures=68 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=0.036488 ap=0.351029 base_ap=0.314541 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6064 delta_ap=-0.001602 ap=0.312939 base_ap=0.314541 donor_failures=52 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6064 delta_ap=0.067370 ap=0.381911 base_ap=0.314541 donor_failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6064 delta_ap=-0.017767 ap=0.296774 base_ap=0.314541 donor_failures=136 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6064 delta_ap=0.014051 ap=0.328592 base_ap=0.314541 donor_failures=151 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6064 delta_ap=0.031255 ap=0.345796 base_ap=0.314541 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6064 delta_ap=0.039373 ap=0.353914 base_ap=0.314541 donor_failures=55 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6064 delta_ap=0.028000 ap=0.342540 base_ap=0.314541 donor_failures=27 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6064 delta_ap=0.057876 ap=0.372417 base_ap=0.314541 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=4 origin=6066 ap=0.248558 rows=1401 failures=63 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.007094 ap=0.241465 base_ap=0.248558 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.001609 ap=0.250168 base_ap=0.248558 donor_failures=125 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.028573 ap=0.277131 base_ap=0.248558 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6066 delta_ap=0.003051 ap=0.251610 base_ap=0.248558 donor_failures=77 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.047348 ap=0.201210 base_ap=0.248558 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.070903 ap=0.177656 base_ap=0.248558 donor_failures=64 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.021239 ap=0.227320 base_ap=0.248558 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6066 delta_ap=-0.015854 ap=0.232704 base_ap=0.248558 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6066 delta_ap=-0.048078 ap=0.200480 base_ap=0.248558 donor_failures=162 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6066 delta_ap=-0.011674 ap=0.236885 base_ap=0.248558 donor_failures=46 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6066 delta_ap=0.032363 ap=0.280921 base_ap=0.248558 donor_failures=63 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.010902 ap=0.237656 base_ap=0.248558 donor_failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.006237 ap=0.242322 base_ap=0.248558 donor_failures=283 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=4 origin=6067 ap=0.397952 rows=1242 failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.044064 ap=0.353889 base_ap=0.397952 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.023764 ap=0.374188 base_ap=0.397952 donor_failures=117 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6067 delta_ap=0.005621 ap=0.403574 base_ap=0.397952 donor_failures=48 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6067 delta_ap=-0.071575 ap=0.326378 base_ap=0.397952 donor_failures=67 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.006778 ap=0.404730 base_ap=0.397952 donor_failures=129 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6067 delta_ap=0.040037 ap=0.437989 base_ap=0.397952 donor_failures=82 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6067 delta_ap=0.012488 ap=0.410440 base_ap=0.397952 donor_failures=17 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6067 delta_ap=-0.041070 ap=0.356882 base_ap=0.397952 donor_failures=148 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6067 delta_ap=-0.011071 ap=0.386881 base_ap=0.397952 donor_failures=180 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6067 delta_ap=0.017597 ap=0.415549 base_ap=0.397952 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6067 delta_ap=-0.004078 ap=0.393875 base_ap=0.397952 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.037757 ap=0.360196 base_ap=0.397952 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.015000 ap=0.382952 base_ap=0.397952 donor_failures=253 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=4 origin=6069 ap=0.312816 rows=1417 failures=54 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.009328 ap=0.303488 base_ap=0.312816 donor_failures=134 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.009940 ap=0.302876 base_ap=0.312816 donor_failures=120 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6069 delta_ap=0.018147 ap=0.330963 base_ap=0.312816 donor_failures=40 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6069 delta_ap=0.012887 ap=0.325704 base_ap=0.312816 donor_failures=70 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.076108 ap=0.388924 base_ap=0.312816 donor_failures=134 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6069 delta_ap=0.067912 ap=0.380729 base_ap=0.312816 donor_failures=71 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.016580 ap=0.296236 base_ap=0.312816 donor_failures=22 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6069 delta_ap=0.043927 ap=0.356744 base_ap=0.312816 donor_failures=158 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6069 delta_ap=0.121995 ap=0.434811 base_ap=0.312816 donor_failures=197 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6069 delta_ap=0.008453 ap=0.321269 base_ap=0.312816 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6069 delta_ap=0.023995 ap=0.336811 base_ap=0.312816 donor_failures=45 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.012595 ap=0.325411 base_ap=0.312816 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6069 delta_ap=0.216752 ap=0.529568 base_ap=0.312816 donor_failures=245 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_tay_ban_nha origin=6067 policy_sources=khoa_tieng_phap,khoa_tieng_anh,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.397952 policy_ap=0.376155 delta_ap=-0.021798 policy_donor_rows=4095 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_tay_ban_nha origin=6069 policy_sources=khoa_tieng_phap,khoa_tieng_han_quoc,khoa_tieng_nhat policy_donor_count=3 base_ap=0.312816 policy_ap=0.337494 delta_ap=0.024678 policy_donor_rows=4095 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_tay_ban_nha selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.001440 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common rows=1306 failures=39 prevalence=0.029862 ap=0.576896 auc=0.911218 f2_at20=0.382775 brier=0.019134 ece10=0.012405 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=0.557812 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common in_sample_bce=0.026256 local_bce=0.026256 donor_bce=nan in_sample_local_ap=0.937383 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native rows=1306 failures=39 prevalence=0.029862 ap=0.544140 auc=0.896728 f2_at20=0.382775 brier=0.019220 ece10=0.014478 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=0.921948 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native in_sample_bce=0.026307 local_bce=0.026307 donor_bce=nan in_sample_local_ap=0.937153 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all rows=1306 failures=39 prevalence=0.029862 ap=0.578071 auc=0.901160 f2_at20=0.382775 brier=0.018812 ece10=0.007742 rho=0.250000 donor_rows=4095 effective_n=14533.846587 seconds=0.553502 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all in_sample_bce=0.031707 local_bce=0.031640 donor_bce=0.053026 in_sample_local_ap=0.906909 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.024106364787319753, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03571389423261371, 'khoa_quoc_te_hoc': 0.012031601966490734, 'khoa_tieng_anh': 0.049168434206415756, 'khoa_tieng_bo_dao_nha': 0.004544221378187737, 'khoa_tieng_duc': 0.011362266177784354, 'khoa_tieng_han_quoc': 0.018208399787347156, 'khoa_tieng_italia': 0.011636303348186147, 'khoa_tieng_nga': 0.008056007716886719, 'khoa_tieng_nhat': 0.018442016475114686, 'khoa_tieng_phap': 0.015598195739270074, 'khoa_tieng_trung_quoc': 0.03608932515606417, 'khoa_viet_nam_hoc': 0.005042969028319001} source_failures={'khoa_tieng_tay_ban_nha': 329, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 1, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 27} positive_ess=389.725330 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal rows=1306 failures=39 prevalence=0.029862 ap=0.579276 auc=0.902030 f2_at20=0.370813 brier=0.018531 ece10=0.008082 rho=0.250000 donor_rows=4095 effective_n=16179.182673 seconds=0.556919 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal in_sample_bce=0.035575 local_bce=0.033243 donor_bce=0.042570 in_sample_local_ap=0.898233 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_tay_ban_nha': 329, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 1, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 27} positive_ess=439.977822 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior rows=1306 failures=39 prevalence=0.029862 ap=0.513834 auc=0.903366 f2_at20=0.370813 brier=0.020718 ece10=0.009931 rho=0.250000 donor_rows=4095 effective_n=16179.182673 seconds=0.564841 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior in_sample_bce=0.033101 local_bce=0.032145 donor_bce=0.035968 in_sample_local_ap=0.909934 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_tay_ban_nha': 329, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 8, 'khoa_tieng_trung_quoc': 4} positive_ess=415.981640 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor rows=1306 failures=39 prevalence=0.029862 ap=0.548633 auc=0.907029 f2_at20=0.382775 brier=0.019489 ece10=0.010117 rho=0.250000 donor_rows=4096 effective_n=16180.158124 seconds=0.570240 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor in_sample_bce=0.024564 local_bce=0.029754 donor_bce=0.008991 in_sample_local_ap=0.919713 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor mass={'khoa_tieng_tay_ban_nha': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_tieng_tay_ban_nha': 329, np.str_('khoa_tieng_han_quoc'): 29} positive_ess=357.992707 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common rows=1306 failures=39 prevalence=0.029862 ap=0.576896 auc=0.911218 f2_at20=0.382775 brier=0.019134 ece10=0.012405 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=0.528147 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common in_sample_bce=0.026256 local_bce=0.026256 donor_bce=nan in_sample_local_ap=0.937383 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native rows=1306 failures=39 prevalence=0.029862 ap=0.544140 auc=0.896728 f2_at20=0.382775 brier=0.019220 ece10=0.014478 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=0.973781 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native in_sample_bce=0.026307 local_bce=0.026307 donor_bce=nan in_sample_local_ap=0.937153 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator rows=1306 failures=39 prevalence=0.029862 ap=0.544140 auc=0.896728 f2_at20=0.382775 brier=0.019220 ece10=0.014478 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=1.763246 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator in_sample_bce=0.026307 local_bce=0.026307 donor_bce=nan in_sample_local_ap=0.937153 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator rows=1306 failures=39 prevalence=0.029862 ap=0.561348 auc=0.900674 f2_at20=0.382775 brier=0.019233 ece10=0.013918 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=1.830074 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator in_sample_bce=0.026262 local_bce=0.026262 donor_bce=nan in_sample_local_ap=0.938005 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator rows=1306 failures=39 prevalence=0.029862 ap=0.544140 auc=0.896728 f2_at20=0.382775 brier=0.019220 ece10=0.014478 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=1.792382 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator in_sample_bce=0.026307 local_bce=0.026307 donor_bce=nan in_sample_local_ap=0.937153 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan rows=1306 failures=39 prevalence=0.029862 ap=0.544140 auc=0.896728 f2_at20=0.382775 brier=0.019220 ece10=0.014478 rho=0.000000 donor_rows=0 effective_n=12085.000000 seconds=1.741826 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan in_sample_bce=0.026307 local_bce=0.026307 donor_bce=nan in_sample_local_ap=0.937153 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 329} positive_ess=329.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_tay_ban_nha model=committee_reference ap=0.598452 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_tay_ban_nha models=13 positive_rows=39 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_trung_quoc fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=4 origin=6064 ap=0.233873 rows=5347 failures=29 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.024415 ap=0.209458 base_ap=0.233873 donor_failures=135 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=0.003812 ap=0.237685 base_ap=0.233873 donor_failures=90 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.032756 ap=0.201117 base_ap=0.233873 donor_failures=36 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6064 delta_ap=0.015672 ap=0.249545 base_ap=0.233873 donor_failures=74 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.118463 ap=0.115410 base_ap=0.233873 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6064 delta_ap=-0.015885 ap=0.217988 base_ap=0.233873 donor_failures=52 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6064 delta_ap=0.024412 ap=0.258285 base_ap=0.233873 donor_failures=16 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6064 delta_ap=0.049261 ap=0.283134 base_ap=0.233873 donor_failures=118 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6064 delta_ap=-0.033483 ap=0.200390 base_ap=0.233873 donor_failures=150 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6064 delta_ap=0.125827 ap=0.359700 base_ap=0.233873 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6064 delta_ap=-0.074606 ap=0.159267 base_ap=0.233873 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=0.063203 ap=0.297076 base_ap=0.233873 donor_failures=83 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6064 delta_ap=-0.113697 ap=0.120176 base_ap=0.233873 donor_failures=270 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=4 origin=6066 ap=0.242829 rows=7393 failures=35 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.008059 ap=0.234770 base_ap=0.242829 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.130463 ap=0.112366 base_ap=0.242829 donor_failures=119 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.051900 ap=0.190929 base_ap=0.242829 donor_failures=33 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6066 delta_ap=-0.168762 ap=0.074067 base_ap=0.242829 donor_failures=58 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.088993 ap=0.153836 base_ap=0.242829 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6066 delta_ap=-0.050209 ap=0.192620 base_ap=0.242829 donor_failures=66 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.132307 ap=0.110522 base_ap=0.242829 donor_failures=12 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6066 delta_ap=-0.009961 ap=0.232868 base_ap=0.242829 donor_failures=125 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6066 delta_ap=-0.173332 ap=0.069497 base_ap=0.242829 donor_failures=171 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.205306 ap=0.037523 base_ap=0.242829 donor_failures=41 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6066 delta_ap=-0.107563 ap=0.135266 base_ap=0.242829 donor_failures=43 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.111102 ap=0.131727 base_ap=0.242829 donor_failures=82 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.147041 ap=0.095788 base_ap=0.242829 donor_failures=265 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=4 origin=6067 ap=0.115075 rows=5584 failures=76 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.032259 ap=0.147335 base_ap=0.115075 donor_failures=127 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.035485 ap=0.150560 base_ap=0.115075 donor_failures=112 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6067 delta_ap=0.018025 ap=0.133100 base_ap=0.115075 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6067 delta_ap=0.054318 ap=0.169393 base_ap=0.115075 donor_failures=83 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.065246 ap=0.180321 base_ap=0.115075 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6067 delta_ap=0.024914 ap=0.139989 base_ap=0.115075 donor_failures=64 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.023932 ap=0.091144 base_ap=0.115075 donor_failures=24 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6067 delta_ap=-0.031680 ap=0.083395 base_ap=0.115075 donor_failures=159 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6067 delta_ap=-0.015075 ap=0.100000 base_ap=0.115075 donor_failures=183 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.009787 ap=0.105288 base_ap=0.115075 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6067 delta_ap=0.019456 ap=0.134532 base_ap=0.115075 donor_failures=53 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.002117 ap=0.112958 base_ap=0.115075 donor_failures=90 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.004069 ap=0.111006 base_ap=0.115075 donor_failures=254 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=4 origin=6069 ap=0.134722 rows=6920 failures=103 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.007102 ap=0.127620 base_ap=0.134722 donor_failures=128 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.019805 ap=0.114917 base_ap=0.134722 donor_failures=108 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6069 delta_ap=0.009644 ap=0.144366 base_ap=0.134722 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6069 delta_ap=0.014935 ap=0.149657 base_ap=0.134722 donor_failures=69 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.004538 ap=0.130184 base_ap=0.134722 donor_failures=139 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6069 delta_ap=-0.018593 ap=0.116129 base_ap=0.134722 donor_failures=83 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.040569 ap=0.094153 base_ap=0.134722 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6069 delta_ap=-0.023925 ap=0.110797 base_ap=0.134722 donor_failures=133 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6069 delta_ap=-0.013922 ap=0.120800 base_ap=0.134722 donor_failures=209 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6069 delta_ap=-0.008907 ap=0.125815 base_ap=0.134722 donor_failures=38 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6069 delta_ap=0.061159 ap=0.195882 base_ap=0.134722 donor_failures=55 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.058941 ap=0.193663 base_ap=0.134722 donor_failures=124 [TRANSFER_EDGE] fold=4 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6069 delta_ap=0.022162 ap=0.156884 base_ap=0.134722 donor_failures=258 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_trung_quoc origin=6067 policy_sources=khoa_tieng_italia policy_donor_count=1 base_ap=0.115075 policy_ap=0.083395 delta_ap=-0.031680 policy_donor_rows=4096 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_tieng_trung_quoc origin=6069 policy_sources= policy_donor_count=0 base_ap=0.134722 policy_ap=0.134722 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_tieng_trung_quoc selected=[] reason=prequential_policy_delta_-0.015840_below_threshold replay_mean=-0.015840 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_local_common rows=5107 failures=104 prevalence=0.020364 ap=0.490106 auc=0.957912 f2_at20=0.344228 brier=0.014090 ece10=0.005378 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=0.915004 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_local_common in_sample_bce=0.012106 local_bce=0.012106 donor_bce=nan in_sample_local_ap=0.843861 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_local_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_local_native rows=5107 failures=104 prevalence=0.020364 ap=0.343240 auc=0.947798 f2_at20=0.344228 brier=0.015696 ece10=0.004263 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=2.438824 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_local_native in_sample_bce=0.011915 local_bce=0.011915 donor_bce=nan in_sample_local_ap=0.847059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_local_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_pool_all rows=5107 failures=104 prevalence=0.020364 ap=0.273681 auc=0.931239 f2_at20=0.330320 brier=0.017139 ece10=0.006062 rho=0.250000 donor_rows=4095 effective_n=29498.940042 seconds=0.931173 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_pool_all in_sample_bce=0.018494 local_bce=0.016788 donor_bce=0.041119 in_sample_local_ap=0.698208 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_pool_all mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.027123590516802798, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04018395355219058, 'khoa_quoc_te_hoc': 0.013537513759516034, 'khoa_tieng_anh': 0.0553225045556717, 'khoa_tieng_bo_dao_nha': 0.005112989908146559, 'khoa_tieng_duc': 0.012784401873452537, 'khoa_tieng_han_quoc': 0.020487418329371208, 'khoa_tieng_italia': 0.013092738367234931, 'khoa_tieng_nga': 0.009064322075967945, 'khoa_tieng_nhat': 0.020750275190320702, 'khoa_tieng_phap': 0.0175505132260939, 'khoa_tieng_tay_ban_nha': 0.009315616318400596, 'khoa_viet_nam_hoc': 0.005674162326830517} source_failures={'khoa_tieng_trung_quoc': 399, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 6, 'khoa_viet_nam_hoc': 23} positive_ess=210.922616 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal rows=5107 failures=104 prevalence=0.020364 ap=0.367301 auc=0.943847 f2_at20=0.333797 brier=0.015777 ece10=0.004796 rho=0.250000 donor_rows=4095 effective_n=38549.613662 seconds=0.895524 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal in_sample_bce=0.020212 local_bce=0.017323 donor_bce=0.028880 in_sample_local_ap=0.684684 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_trung_quoc': 399, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 1, 'khoa_tieng_tay_ban_nha': 6, 'khoa_viet_nam_hoc': 23} positive_ess=306.891007 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_language_prior rows=5107 failures=104 prevalence=0.020364 ap=0.340965 auc=0.943745 f2_at20=0.333797 brier=0.015991 ece10=0.004984 rho=0.250000 donor_rows=4095 effective_n=38549.613662 seconds=0.949532 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_language_prior in_sample_bce=0.018865 local_bce=0.016830 donor_bce=0.024969 in_sample_local_ap=0.694644 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_language_prior mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776} source_failures={'khoa_tieng_trung_quoc': 399, 'khoa_tieng_anh': 10, 'khoa_tieng_bo_dao_nha': 16, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 20, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 11} positive_ess=298.807796 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_random_donor rows=5107 failures=104 prevalence=0.020364 ap=0.335430 auc=0.948280 f2_at20=0.340751 brier=0.016284 ece10=0.006396 rho=0.250000 donor_rows=4096 effective_n=38555.151862 seconds=0.966526 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_random_donor in_sample_bce=0.019036 local_bce=0.016976 donor_bce=0.025217 in_sample_local_ap=0.704753 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=hgb_random_donor mass={'khoa_tieng_trung_quoc': 0.75, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.25} source_failures={'khoa_tieng_trung_quoc': 399, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 108} positive_ess=311.696732 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_common rows=5107 failures=104 prevalence=0.020364 ap=0.490106 auc=0.957912 f2_at20=0.344228 brier=0.014090 ece10=0.005378 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=0.897067 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_common in_sample_bce=0.012106 local_bce=0.012106 donor_bce=nan in_sample_local_ap=0.843861 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_native rows=5107 failures=104 prevalence=0.020364 ap=0.343240 auc=0.947798 f2_at20=0.344228 brier=0.015696 ece10=0.004263 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=2.392903 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_native in_sample_bce=0.011915 local_bce=0.011915 donor_bce=nan in_sample_local_ap=0.847059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator rows=5107 failures=104 prevalence=0.020364 ap=0.343240 auc=0.947798 f2_at20=0.344228 brier=0.015696 ece10=0.004263 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=4.450312 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator in_sample_bce=0.011915 local_bce=0.011915 donor_bce=nan in_sample_local_ap=0.847059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator rows=5107 failures=104 prevalence=0.020364 ap=0.343240 auc=0.947798 f2_at20=0.344228 brier=0.015696 ece10=0.004263 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=4.563001 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator in_sample_bce=0.011915 local_bce=0.011915 donor_bce=nan in_sample_local_ap=0.847059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator rows=5107 failures=104 prevalence=0.020364 ap=0.343240 auc=0.947798 f2_at20=0.344228 brier=0.015696 ece10=0.004263 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=4.402730 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator in_sample_bce=0.011915 local_bce=0.011915 donor_bce=nan in_sample_local_ap=0.847059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan rows=5107 failures=104 prevalence=0.020364 ap=0.343240 auc=0.947798 f2_at20=0.344228 brier=0.015696 ece10=0.004263 rho=0.000000 donor_rows=0 effective_n=52678.000000 seconds=4.483308 [TRAINING_INSIGHT] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.011915 local_bce=0.011915 donor_bce=nan in_sample_local_ap=0.847059 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 399} positive_ess=399.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_tieng_trung_quoc model=committee_reference ap=0.471306 [FACULTY_COMPLETE] fold=4 faculty=khoa_tieng_trung_quoc models=13 positive_rows=104 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_viet_nam_hoc fold=4 origins=[6064, 6066, 6067, 6069] availability=semester_proxy [SCREEN_BASE] target=khoa_viet_nam_hoc fold=4 origin=6064 ap=0.581246 rows=818 failures=49 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6064 delta_ap=-0.033848 ap=0.547399 base_ap=0.581246 donor_failures=107 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6064 delta_ap=-0.041498 ap=0.539748 base_ap=0.581246 donor_failures=96 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6064 delta_ap=-0.030120 ap=0.551126 base_ap=0.581246 donor_failures=30 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6064 delta_ap=-0.037117 ap=0.544129 base_ap=0.581246 donor_failures=79 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6064 delta_ap=-0.002691 ap=0.578555 base_ap=0.581246 donor_failures=97 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6064 delta_ap=-0.035826 ap=0.545420 base_ap=0.581246 donor_failures=54 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6064 delta_ap=-0.002537 ap=0.578709 base_ap=0.581246 donor_failures=24 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6064 delta_ap=-0.040346 ap=0.540900 base_ap=0.581246 donor_failures=122 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6064 delta_ap=-0.039626 ap=0.541620 base_ap=0.581246 donor_failures=152 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6064 delta_ap=-0.024577 ap=0.556670 base_ap=0.581246 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6064 delta_ap=-0.014631 ap=0.566616 base_ap=0.581246 donor_failures=44 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6064 delta_ap=-0.005209 ap=0.576037 base_ap=0.581246 donor_failures=81 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6064 delta_ap=-0.026787 ap=0.554459 base_ap=0.581246 donor_failures=15 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=4 origin=6066 ap=0.184946 rows=813 failures=30 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.019292 ap=0.204238 base_ap=0.184946 donor_failures=131 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.007090 ap=0.177856 base_ap=0.184946 donor_failures=104 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.029731 ap=0.155215 base_ap=0.184946 donor_failures=37 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6066 delta_ap=-0.018578 ap=0.166368 base_ap=0.184946 donor_failures=61 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.021980 ap=0.162966 base_ap=0.184946 donor_failures=123 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.042282 ap=0.227228 base_ap=0.184946 donor_failures=74 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.041851 ap=0.143094 base_ap=0.184946 donor_failures=23 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6066 delta_ap=-0.026704 ap=0.158241 base_ap=0.184946 donor_failures=113 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.015320 ap=0.200266 base_ap=0.184946 donor_failures=167 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.021405 ap=0.163541 base_ap=0.184946 donor_failures=39 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6066 delta_ap=-0.020763 ap=0.164183 base_ap=0.184946 donor_failures=58 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.066077 ap=0.118869 base_ap=0.184946 donor_failures=91 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.001217 ap=0.183728 base_ap=0.184946 donor_failures=16 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=4 origin=6067 ap=0.428154 rows=556 failures=39 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.103920 ap=0.324235 base_ap=0.428154 donor_failures=132 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.002656 ap=0.430810 base_ap=0.428154 donor_failures=113 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.033151 ap=0.395003 base_ap=0.428154 donor_failures=47 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6067 delta_ap=-0.019662 ap=0.408492 base_ap=0.428154 donor_failures=53 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.057396 ap=0.370758 base_ap=0.428154 donor_failures=130 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6067 delta_ap=-0.095401 ap=0.332753 base_ap=0.428154 donor_failures=82 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.017170 ap=0.410984 base_ap=0.428154 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6067 delta_ap=-0.125193 ap=0.302961 base_ap=0.428154 donor_failures=152 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6067 delta_ap=-0.009122 ap=0.419032 base_ap=0.428154 donor_failures=200 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.010259 ap=0.417895 base_ap=0.428154 donor_failures=42 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6067 delta_ap=-0.017200 ap=0.410954 base_ap=0.428154 donor_failures=51 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.072297 ap=0.355857 base_ap=0.428154 donor_failures=108 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.024321 ap=0.403833 base_ap=0.428154 donor_failures=23 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=4 origin=6069 ap=0.391156 rows=773 failures=33 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.047913 ap=0.439068 base_ap=0.391156 donor_failures=138 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.074829 ap=0.316327 base_ap=0.391156 donor_failures=115 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.021813 ap=0.369343 base_ap=0.391156 donor_failures=35 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6069 delta_ap=-0.004877 ap=0.386279 base_ap=0.391156 donor_failures=62 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.072627 ap=0.463783 base_ap=0.391156 donor_failures=134 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6069 delta_ap=0.068642 ap=0.459798 base_ap=0.391156 donor_failures=87 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.098646 ap=0.292509 base_ap=0.391156 donor_failures=21 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6069 delta_ap=0.030972 ap=0.422127 base_ap=0.391156 donor_failures=163 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6069 delta_ap=-0.026873 ap=0.364283 base_ap=0.391156 donor_failures=188 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6069 delta_ap=-0.045438 ap=0.345718 base_ap=0.391156 donor_failures=40 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6069 delta_ap=0.059599 ap=0.450755 base_ap=0.391156 donor_failures=50 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.060890 ap=0.330266 base_ap=0.391156 donor_failures=98 [TRANSFER_EDGE] fold=4 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.012470 ap=0.403625 base_ap=0.391156 donor_failures=38 [POLICY_REPLAY] fold=4 faculty=khoa_viet_nam_hoc origin=6067 policy_sources=khoa_tieng_duc policy_donor_count=1 base_ap=0.428154 policy_ap=0.332753 delta_ap=-0.095401 policy_donor_rows=4096 selected_using_origins=6064,6066 [POLICY_REPLAY] fold=4 faculty=khoa_viet_nam_hoc origin=6069 policy_sources= policy_donor_count=0 base_ap=0.391156 policy_ap=0.391156 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6064,6066,6067 [SELECTION_FINAL] fold=4 faculty=khoa_viet_nam_hoc selected=[] reason=no_stable_positive_directed_edge replay_mean=-0.047700 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_local_common rows=505 failures=18 prevalence=0.035644 ap=0.575939 auc=0.961271 f2_at20=0.491329 brier=0.022300 ece10=0.027315 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.397042 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=hgb_local_common in_sample_bce=0.059939 local_bce=0.059939 donor_bce=nan in_sample_local_ap=0.919147 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_local_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_local_native rows=505 failures=18 prevalence=0.035644 ap=0.568273 auc=0.965891 f2_at20=0.491329 brier=0.022426 ece10=0.021592 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.455311 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=hgb_local_native in_sample_bce=0.059441 local_bce=0.059441 donor_bce=nan in_sample_local_ap=0.916813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_local_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_pool_all rows=505 failures=18 prevalence=0.035644 ap=0.512079 auc=0.970283 f2_at20=0.491329 brier=0.022864 ece10=0.023877 rho=0.250000 donor_rows=4095 effective_n=10178.745861 seconds=0.549068 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=hgb_pool_all in_sample_bce=0.059912 local_bce=0.066013 donor_bce=0.057356 in_sample_local_ap=0.897188 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_pool_all mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.023798283726240266, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.035257468117455455, 'khoa_quoc_te_hoc': 0.01187783723534918, 'khoa_tieng_anh': 0.04854005811090851, 'khoa_tieng_bo_dao_nha': 0.004486145904912374, 'khoa_tieng_duc': 0.01121705560575482, 'khoa_tieng_han_quoc': 0.017975695139568985, 'khoa_tieng_italia': 0.011487590561576461, 'khoa_tieng_nga': 0.007953051363766712, 'khoa_tieng_nhat': 0.018206326189406933, 'khoa_tieng_phap': 0.015398849685367847, 'khoa_tieng_tay_ban_nha': 0.00817353735276135, 'khoa_tieng_trung_quoc': 0.035628101006931104} source_failures={'khoa_viet_nam_hoc': 439, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 2, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 11, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1} positive_ess=498.613268 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_pool_equal rows=505 failures=18 prevalence=0.035644 ap=0.520135 auc=0.957734 f2_at20=0.491329 brier=0.024138 ece10=0.020176 rho=0.250000 donor_rows=4095 effective_n=10907.651529 seconds=0.524761 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=hgb_pool_equal in_sample_bce=0.063289 local_bce=0.067802 donor_bce=0.049753 in_sample_local_ap=0.891092 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_pool_equal mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232} source_failures={'khoa_viet_nam_hoc': 439, 'khoa_cong_nghe_thong_tin': 10, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 2, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 11, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 6, 'khoa_tieng_trung_quoc': 1} positive_ess=519.838336 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_language_prior rows=505 failures=18 prevalence=0.035644 ap=0.575939 auc=0.961271 f2_at20=0.491329 brier=0.022300 ece10=0.027315 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.400730 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=hgb_language_prior in_sample_bce=0.059939 local_bce=0.059939 donor_bce=nan in_sample_local_ap=0.919147 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_language_prior mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_random_donor rows=505 failures=18 prevalence=0.035644 ap=0.493661 auc=0.953057 f2_at20=0.433526 brier=0.022863 ece10=0.028595 rho=0.250000 donor_rows=4096 effective_n=10908.094878 seconds=0.487410 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=hgb_random_donor in_sample_bce=0.062586 local_bce=0.067702 donor_bce=0.047236 in_sample_local_ap=0.889857 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=hgb_random_donor mass={'khoa_viet_nam_hoc': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.25} source_failures={'khoa_viet_nam_hoc': 439, np.str_('khoa_tieng_tay_ban_nha'): 98} positive_ess=522.413139 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_common rows=505 failures=18 prevalence=0.035644 ap=0.575939 auc=0.961271 f2_at20=0.491329 brier=0.022300 ece10=0.027315 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.410846 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_common in_sample_bce=0.059939 local_bce=0.059939 donor_bce=nan in_sample_local_ap=0.919147 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_native rows=505 failures=18 prevalence=0.035644 ap=0.568273 auc=0.965891 f2_at20=0.491329 brier=0.022426 ece10=0.021592 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.457778 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_native in_sample_bce=0.059441 local_bce=0.059441 donor_bce=nan in_sample_local_ap=0.916813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator rows=505 failures=18 prevalence=0.035644 ap=0.568273 auc=0.965891 f2_at20=0.491329 brier=0.022426 ece10=0.021592 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.725772 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator in_sample_bce=0.059441 local_bce=0.059441 donor_bce=nan in_sample_local_ap=0.916813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator rows=505 failures=18 prevalence=0.035644 ap=0.553972 auc=0.964465 f2_at20=0.491329 brier=0.022606 ece10=0.020817 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.726528 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator in_sample_bce=0.059981 local_bce=0.059981 donor_bce=nan in_sample_local_ap=0.915021 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=native_placebo_indicator rows=505 failures=18 prevalence=0.035644 ap=0.568273 auc=0.965891 f2_at20=0.491329 brier=0.022426 ece10=0.021592 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.761436 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=native_placebo_indicator in_sample_bce=0.059441 local_bce=0.059441 donor_bce=nan in_sample_local_ap=0.916813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=native_placebo_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan rows=505 failures=18 prevalence=0.035644 ap=0.568273 auc=0.965891 f2_at20=0.491329 brier=0.022426 ece10=0.021592 rho=0.000000 donor_rows=0 effective_n=7361.000000 seconds=0.731336 [TRAINING_INSIGHT] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.059441 local_bce=0.059441 donor_bce=nan in_sample_local_ap=0.916813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=4 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 439} positive_ess=439.000000 [FACULTY_RESULTS] fold=4 faculty=khoa_viet_nam_hoc model=committee_reference ap=0.519234 [FACULTY_COMPLETE] fold=4 faculty=khoa_viet_nam_hoc models=13 positive_rows=18 num_edges=52 num_replays=2 chosen=[] [FOLD_START] fold=5 stage=development [DATA] fold=5 common_raw=136 course_features=17 common_total=153 faculties=14 history_rows=419467 target_rows=50698 target_failures=1797 history_last=6071 target_first=6072 [OBSERVATION_AUDIT] fold=5 faculty=khoa_cong_nghe_thong_tin raw_columns=275 common_raw_columns=136 native_raw_columns=139 history_rows=40261 history_failures=1379 target_rows=6398 target_failures=351 target_prevalence=0.054861 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.651800 history_latest_semester=6071 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich raw_columns=301 common_raw_columns=136 native_raw_columns=165 history_rows=59575 history_failures=1814 target_rows=7604 target_failures=407 target_prevalence=0.053524 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.690069 history_latest_semester=6071 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_quoc_te_hoc raw_columns=254 common_raw_columns=136 native_raw_columns=118 history_rows=19764 history_failures=209 target_rows=2674 target_failures=58 target_prevalence=0.021690 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.619340 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_anh raw_columns=511 common_raw_columns=136 native_raw_columns=375 history_rows=78407 history_failures=1260 target_rows=8345 target_failures=132 target_prevalence=0.015818 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.803769 history_latest_semester=6071 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_bo_dao_nha raw_columns=400 common_raw_columns=136 native_raw_columns=264 history_rows=7502 history_failures=297 target_rows=1138 target_failures=80 target_prevalence=0.070299 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.750673 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_duc raw_columns=381 common_raw_columns=136 native_raw_columns=245 history_rows=18332 history_failures=362 target_rows=1975 target_failures=59 target_prevalence=0.029873 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.744918 history_latest_semester=6071 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_han_quoc raw_columns=333 common_raw_columns=136 native_raw_columns=197 history_rows=29929 history_failures=204 target_rows=3573 target_failures=67 target_prevalence=0.018752 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.680547 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_italia raw_columns=420 common_raw_columns=136 native_raw_columns=284 history_rows=18780 history_failures=610 target_rows=2129 target_failures=83 target_prevalence=0.038985 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.740438 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_nga raw_columns=469 common_raw_columns=136 native_raw_columns=333 history_rows=13418 history_failures=661 target_rows=2047 target_failures=134 target_prevalence=0.065462 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.790459 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_nhat raw_columns=362 common_raw_columns=136 native_raw_columns=226 history_rows=29332 history_failures=282 target_rows=2593 target_failures=69 target_prevalence=0.026610 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.739892 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_phap raw_columns=495 common_raw_columns=136 native_raw_columns=359 history_rows=25043 history_failures=322 target_rows=2983 target_failures=57 target_prevalence=0.019108 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.786876 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_tay_ban_nha raw_columns=396 common_raw_columns=136 native_raw_columns=260 history_rows=13391 history_failures=368 target_rows=1606 target_failures=84 target_prevalence=0.052304 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.745315 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_tieng_trung_quoc raw_columns=436 common_raw_columns=136 native_raw_columns=300 history_rows=57867 history_failures=511 target_rows=6995 target_failures=155 target_prevalence=0.022159 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.758655 history_latest_semester=6071 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [OBSERVATION_AUDIT] fold=5 faculty=khoa_viet_nam_hoc raw_columns=181 common_raw_columns=136 native_raw_columns=45 history_rows=7866 history_failures=457 target_rows=638 target_failures=61 target_prevalence=0.095611 raw_cache_nonfinite_rate=0.000000 raw_exact_zero_rate=0.495005 history_latest_semester=6070 target_first_semester=6072 availability_provenance=semester_proxy; source release timestamps not verified [SELECTION_ORIGINS] target=khoa_cong_nghe_thong_tin fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=5 origin=6066 ap=0.247822 rows=5222 failures=189 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.032305 ap=0.280127 base_ap=0.247822 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6066 delta_ap=0.045278 ap=0.293099 base_ap=0.247822 donor_failures=29 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6066 delta_ap=-0.015673 ap=0.232149 base_ap=0.247822 donor_failures=71 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.011610 ap=0.259432 base_ap=0.247822 donor_failures=122 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6066 delta_ap=-0.031387 ap=0.216435 base_ap=0.247822 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6066 delta_ap=0.031780 ap=0.279601 base_ap=0.247822 donor_failures=18 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6066 delta_ap=0.016631 ap=0.264453 base_ap=0.247822 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6066 delta_ap=0.056037 ap=0.303859 base_ap=0.247822 donor_failures=156 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6066 delta_ap=-0.073003 ap=0.174819 base_ap=0.247822 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6066 delta_ap=0.018617 ap=0.266439 base_ap=0.247822 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.012575 ap=0.260397 base_ap=0.247822 donor_failures=78 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.002789 ap=0.250611 base_ap=0.247822 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6066 delta_ap=0.044822 ap=0.292644 base_ap=0.247822 donor_failures=257 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=5 origin=6067 ap=0.371559 rows=4598 failures=152 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.025673 ap=0.397232 base_ap=0.371559 donor_failures=118 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6067 delta_ap=0.016514 ap=0.388073 base_ap=0.371559 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6067 delta_ap=-0.035835 ap=0.335723 base_ap=0.371559 donor_failures=63 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.036319 ap=0.335240 base_ap=0.371559 donor_failures=136 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6067 delta_ap=-0.009951 ap=0.361608 base_ap=0.371559 donor_failures=71 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6067 delta_ap=0.010128 ap=0.381687 base_ap=0.371559 donor_failures=15 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6067 delta_ap=-0.036175 ap=0.335384 base_ap=0.371559 donor_failures=143 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6067 delta_ap=0.005698 ap=0.377257 base_ap=0.371559 donor_failures=200 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6067 delta_ap=-0.010573 ap=0.360986 base_ap=0.371559 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6067 delta_ap=0.018267 ap=0.389826 base_ap=0.371559 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.004682 ap=0.366877 base_ap=0.371559 donor_failures=106 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.005897 ap=0.365662 base_ap=0.371559 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.005272 ap=0.366286 base_ap=0.371559 donor_failures=244 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=5 origin=6069 ap=0.239657 rows=6277 failures=218 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.037117 ap=0.276775 base_ap=0.239657 donor_failures=117 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6069 delta_ap=0.015243 ap=0.254901 base_ap=0.239657 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6069 delta_ap=0.053509 ap=0.293166 base_ap=0.239657 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.019852 ap=0.259510 base_ap=0.239657 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6069 delta_ap=0.042381 ap=0.282038 base_ap=0.239657 donor_failures=70 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6069 delta_ap=0.049414 ap=0.289071 base_ap=0.239657 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6069 delta_ap=-0.034460 ap=0.205197 base_ap=0.239657 donor_failures=145 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6069 delta_ap=0.054600 ap=0.294257 base_ap=0.239657 donor_failures=185 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6069 delta_ap=0.026771 ap=0.266429 base_ap=0.239657 donor_failures=36 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6069 delta_ap=0.025436 ap=0.265094 base_ap=0.239657 donor_failures=55 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.079734 ap=0.319391 base_ap=0.239657 donor_failures=110 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.040748 ap=0.280406 base_ap=0.239657 donor_failures=31 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6069 delta_ap=0.058940 ap=0.298598 base_ap=0.239657 donor_failures=256 [SCREEN_BASE] target=khoa_cong_nghe_thong_tin fold=5 origin=6070 ap=0.342071 rows=5072 failures=244 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=0.026055 ap=0.368127 base_ap=0.342071 donor_failures=110 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_quoc_te_hoc origin=6070 delta_ap=0.009145 ap=0.351216 base_ap=0.342071 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_anh origin=6070 delta_ap=0.002488 ap=0.344560 base_ap=0.342071 donor_failures=58 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.011531 ap=0.353603 base_ap=0.342071 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_duc origin=6070 delta_ap=0.004746 ap=0.346817 base_ap=0.342071 donor_failures=81 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_han_quoc origin=6070 delta_ap=-0.002227 ap=0.339844 base_ap=0.342071 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_italia origin=6070 delta_ap=0.010980 ap=0.353052 base_ap=0.342071 donor_failures=149 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nga origin=6070 delta_ap=0.007022 ap=0.349093 base_ap=0.342071 donor_failures=180 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_nhat origin=6070 delta_ap=-0.002681 ap=0.339390 base_ap=0.342071 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_phap origin=6070 delta_ap=-0.005505 ap=0.336566 base_ap=0.342071 donor_failures=61 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=0.001848 ap=0.343919 base_ap=0.342071 donor_failures=100 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_tieng_trung_quoc origin=6070 delta_ap=0.011451 ap=0.353522 base_ap=0.342071 donor_failures=32 [TRANSFER_EDGE] fold=5 target=khoa_cong_nghe_thong_tin source=khoa_viet_nam_hoc origin=6070 delta_ap=0.004662 ap=0.346734 base_ap=0.342071 donor_failures=247 [POLICY_REPLAY] fold=5 faculty=khoa_cong_nghe_thong_tin origin=6069 policy_sources=khoa_quoc_te_hoc,khoa_tieng_nga,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.239657 policy_ap=0.287661 delta_ap=0.048004 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_cong_nghe_thong_tin origin=6070 policy_sources=khoa_tieng_nga,khoa_viet_nam_hoc,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.342071 policy_ap=0.330524 delta_ap=-0.011547 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_cong_nghe_thong_tin selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.018228 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common rows=6398 failures=351 prevalence=0.054861 ap=0.241487 auc=0.814929 f2_at20=0.422876 brier=0.047365 ece10=0.021777 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=0.807832 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common in_sample_bce=0.071289 local_bce=0.071289 donor_bce=nan in_sample_local_ap=0.709555 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_local_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native rows=6398 failures=351 prevalence=0.054861 ap=0.266354 auc=0.821347 f2_at20=0.417288 brier=0.046441 ece10=0.021927 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=1.298964 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native in_sample_bce=0.070826 local_bce=0.070826 donor_bce=nan in_sample_local_ap=0.713199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_local_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all rows=6398 failures=351 prevalence=0.054861 ap=0.256778 auc=0.826133 f2_at20=0.435917 brier=0.046644 ece10=0.018575 rho=0.250000 donor_rows=4095 effective_n=26732.115227 seconds=0.825410 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all in_sample_bce=0.069374 local_bce=0.079013 donor_bce=0.065135 in_sample_local_ap=0.639304 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_all mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03927614541963999, 'khoa_quoc_te_hoc': 0.013029857122513885, 'khoa_tieng_anh': 0.05169156078754028, 'khoa_tieng_bo_dao_nha': 0.004945860561278039, 'khoa_tieng_duc': 0.012085779233450947, 'khoa_tieng_han_quoc': 0.019731359735869158, 'khoa_tieng_italia': 0.012381133209917565, 'khoa_tieng_nga': 0.008846115304082742, 'khoa_tieng_nhat': 0.019337774191336636, 'khoa_tieng_phap': 0.016510155430030114, 'khoa_tieng_tay_ban_nha': 0.008828314952822476, 'khoa_tieng_trung_quoc': 0.03815010838436101, 'khoa_viet_nam_hoc': 0.005185835667157165} source_failures={'khoa_cong_nghe_thong_tin': 1379, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 16, 'khoa_tieng_trung_quoc': 6, 'khoa_viet_nam_hoc': 19} positive_ess=1004.193910 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal rows=6398 failures=351 prevalence=0.054861 ap=0.236667 auc=0.823851 f2_at20=0.417288 brier=0.046954 ece10=0.017344 rho=0.250000 donor_rows=4095 effective_n=34206.918409 seconds=0.838040 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal in_sample_bce=0.072908 local_bce=0.079355 donor_bce=0.053567 in_sample_local_ap=0.646964 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_pool_equal mass={'khoa_cong_nghe_thong_tin': 0.75, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_cong_nghe_thong_tin': 1379, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 16, 'khoa_tieng_trung_quoc': 6, 'khoa_viet_nam_hoc': 19} positive_ess=1178.540796 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior rows=6398 failures=351 prevalence=0.054861 ap=0.241487 auc=0.814929 f2_at20=0.422876 brier=0.047365 ece10=0.021777 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=0.796746 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior in_sample_bce=0.071289 local_bce=0.071289 donor_bce=nan in_sample_local_ap=0.709555 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_language_prior mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor rows=6398 failures=351 prevalence=0.054861 ap=0.243850 auc=0.830053 f2_at20=0.432191 brier=0.047506 ece10=0.022295 rho=0.250000 donor_rows=4096 effective_n=34211.279041 seconds=0.829545 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor in_sample_bce=0.069422 local_bce=0.078544 donor_bce=0.042056 in_sample_local_ap=0.641701 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=hgb_random_donor mass={'khoa_cong_nghe_thong_tin': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.25} source_failures={'khoa_cong_nghe_thong_tin': 1379, np.str_('khoa_tieng_tay_ban_nha'): 118} positive_ess=1178.276412 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common rows=6398 failures=351 prevalence=0.054861 ap=0.241487 auc=0.814929 f2_at20=0.422876 brier=0.047365 ece10=0.021777 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=0.781138 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common in_sample_bce=0.071289 local_bce=0.071289 donor_bce=nan in_sample_local_ap=0.709555 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_common mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native rows=6398 failures=351 prevalence=0.054861 ap=0.266354 auc=0.821347 f2_at20=0.417288 brier=0.046441 ece10=0.021927 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=1.296872 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native in_sample_bce=0.070826 local_bce=0.070826 donor_bce=nan in_sample_local_ap=0.713199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator rows=6398 failures=351 prevalence=0.054861 ap=0.266354 auc=0.821347 f2_at20=0.417288 brier=0.046441 ece10=0.021927 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=9.479388 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator in_sample_bce=0.070826 local_bce=0.070826 donor_bce=nan in_sample_local_ap=0.713199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_cache_nan_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator rows=6398 failures=351 prevalence=0.054861 ap=0.266354 auc=0.821347 f2_at20=0.417288 brier=0.046441 ece10=0.021927 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=2.466014 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator in_sample_bce=0.070826 local_bce=0.070826 donor_bce=nan in_sample_local_ap=0.713199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_naive_zero_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator rows=6398 failures=351 prevalence=0.054861 ap=0.266354 auc=0.821347 f2_at20=0.417288 brier=0.046441 ece10=0.021927 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=2.393151 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator in_sample_bce=0.070826 local_bce=0.070826 donor_bce=nan in_sample_local_ap=0.713199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=native_placebo_indicator mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan rows=6398 failures=351 prevalence=0.054861 ap=0.266354 auc=0.821347 f2_at20=0.417288 brier=0.046441 ece10=0.021927 rho=0.000000 donor_rows=0 effective_n=40261.000000 seconds=2.579991 [TRAINING_INSIGHT] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan in_sample_bce=0.070826 local_bce=0.070826 donor_bce=nan in_sample_local_ap=0.713199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_cong_nghe_thong_tin model=suite32_selective_native_cache_nan mass={'khoa_cong_nghe_thong_tin': 1.0} source_failures={'khoa_cong_nghe_thong_tin': 1379} positive_ess=1379.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_cong_nghe_thong_tin model=committee_reference ap=0.275062 [FACULTY_COMPLETE] fold=5 faculty=khoa_cong_nghe_thong_tin models=13 positive_rows=351 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=5 origins=[6068, 6069, 6070, 6071] availability=semester_proxy [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=5 origin=6068 ap=0.457398 rows=984 failures=34 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6068 delta_ap=0.045280 ap=0.502678 base_ap=0.457398 donor_failures=167 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6068 delta_ap=-0.015719 ap=0.441679 base_ap=0.457398 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6068 delta_ap=-0.028416 ap=0.428981 base_ap=0.457398 donor_failures=50 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6068 delta_ap=-0.008189 ap=0.449209 base_ap=0.457398 donor_failures=130 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6068 delta_ap=-0.067359 ap=0.390039 base_ap=0.457398 donor_failures=84 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6068 delta_ap=-0.033089 ap=0.424309 base_ap=0.457398 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6068 delta_ap=-0.057864 ap=0.399533 base_ap=0.457398 donor_failures=136 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6068 delta_ap=-0.037078 ap=0.420320 base_ap=0.457398 donor_failures=198 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6068 delta_ap=0.025988 ap=0.483386 base_ap=0.457398 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6068 delta_ap=-0.033933 ap=0.423465 base_ap=0.457398 donor_failures=50 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6068 delta_ap=-0.019417 ap=0.437980 base_ap=0.457398 donor_failures=106 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6068 delta_ap=-0.010961 ap=0.446437 base_ap=0.457398 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6068 delta_ap=-0.031508 ap=0.425890 base_ap=0.457398 donor_failures=253 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=5 origin=6069 ap=0.388864 rows=6491 failures=209 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.025085 ap=0.413949 base_ap=0.388864 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6069 delta_ap=0.018821 ap=0.407684 base_ap=0.388864 donor_failures=32 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6069 delta_ap=-0.031648 ap=0.357216 base_ap=0.388864 donor_failures=74 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.019586 ap=0.369278 base_ap=0.388864 donor_failures=140 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6069 delta_ap=0.014819 ap=0.403682 base_ap=0.388864 donor_failures=81 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6069 delta_ap=0.018682 ap=0.407545 base_ap=0.388864 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6069 delta_ap=-0.042688 ap=0.346176 base_ap=0.388864 donor_failures=121 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6069 delta_ap=-0.011265 ap=0.377599 base_ap=0.388864 donor_failures=203 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6069 delta_ap=-0.008285 ap=0.380578 base_ap=0.388864 donor_failures=32 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6069 delta_ap=-0.022512 ap=0.366352 base_ap=0.388864 donor_failures=53 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.036872 ap=0.351991 base_ap=0.388864 donor_failures=102 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.014530 ap=0.403394 base_ap=0.388864 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.003740 ap=0.385123 base_ap=0.388864 donor_failures=251 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=5 origin=6070 ap=0.343269 rows=6440 failures=241 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.001545 ap=0.341724 base_ap=0.343269 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.040717 ap=0.302552 base_ap=0.343269 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6070 delta_ap=0.010814 ap=0.354083 base_ap=0.343269 donor_failures=69 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.015317 ap=0.358586 base_ap=0.343269 donor_failures=137 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6070 delta_ap=-0.014690 ap=0.328579 base_ap=0.343269 donor_failures=89 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6070 delta_ap=0.010319 ap=0.353588 base_ap=0.343269 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6070 delta_ap=-0.014547 ap=0.328722 base_ap=0.343269 donor_failures=141 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6070 delta_ap=0.008118 ap=0.351387 base_ap=0.343269 donor_failures=178 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6070 delta_ap=0.000170 ap=0.343439 base_ap=0.343269 donor_failures=35 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6070 delta_ap=-0.000734 ap=0.342535 base_ap=0.343269 donor_failures=53 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.013113 ap=0.330156 base_ap=0.343269 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.007888 ap=0.335381 base_ap=0.343269 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.022677 ap=0.320592 base_ap=0.343269 donor_failures=238 [SCREEN_BASE] target=khoa_quan_tri_kinh_doanh_va_du_lich fold=5 origin=6071 ap=0.406242 rows=1005 failures=30 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_cong_nghe_thong_tin origin=6071 delta_ap=-0.036544 ap=0.369698 base_ap=0.406242 donor_failures=153 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_quoc_te_hoc origin=6071 delta_ap=-0.006409 ap=0.399833 base_ap=0.406242 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_anh origin=6071 delta_ap=-0.026306 ap=0.379936 base_ap=0.406242 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_bo_dao_nha origin=6071 delta_ap=0.039524 ap=0.445766 base_ap=0.406242 donor_failures=145 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_duc origin=6071 delta_ap=-0.072702 ap=0.333540 base_ap=0.406242 donor_failures=75 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_han_quoc origin=6071 delta_ap=-0.024796 ap=0.381446 base_ap=0.406242 donor_failures=20 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_italia origin=6071 delta_ap=0.026443 ap=0.432685 base_ap=0.406242 donor_failures=131 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nga origin=6071 delta_ap=0.044819 ap=0.451061 base_ap=0.406242 donor_failures=216 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_nhat origin=6071 delta_ap=0.021480 ap=0.427723 base_ap=0.406242 donor_failures=44 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_phap origin=6071 delta_ap=-0.036515 ap=0.369727 base_ap=0.406242 donor_failures=46 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_tay_ban_nha origin=6071 delta_ap=-0.011218 ap=0.395024 base_ap=0.406242 donor_failures=115 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_tieng_trung_quoc origin=6071 delta_ap=0.030733 ap=0.436975 base_ap=0.406242 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_quan_tri_kinh_doanh_va_du_lich source=khoa_viet_nam_hoc origin=6071 delta_ap=-0.059410 ap=0.346832 base_ap=0.406242 donor_failures=240 [POLICY_REPLAY] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nhat,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.343269 policy_ap=0.328586 delta_ap=-0.014683 policy_donor_rows=4095 selected_using_origins=6068,6069 [POLICY_REPLAY] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich origin=6071 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nhat policy_donor_count=2 base_ap=0.406242 policy_ap=0.345177 delta_ap=-0.061065 policy_donor_rows=4096 selected_using_origins=6068,6069,6070 [SELECTION_FINAL] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich selected=[] reason=prequential_policy_delta_-0.037874_below_threshold replay_mean=-0.037874 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common rows=7604 failures=407 prevalence=0.053524 ap=0.382019 auc=0.878473 f2_at20=0.479517 brier=0.041019 ece10=0.017422 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=0.996971 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common in_sample_bce=0.062357 local_bce=0.062357 donor_bce=nan in_sample_local_ap=0.691040 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native rows=7604 failures=407 prevalence=0.053524 ap=0.382596 auc=0.876793 f2_at20=0.476342 brier=0.040889 ece10=0.016996 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=1.962178 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native in_sample_bce=0.062151 local_bce=0.062151 donor_bce=nan in_sample_local_ap=0.692355 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_local_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all rows=7604 failures=407 prevalence=0.053524 ap=0.391739 auc=0.882635 f2_at20=0.490632 brier=0.040682 ece10=0.016420 rho=0.250000 donor_rows=4095 effective_n=30761.718947 seconds=1.053868 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all in_sample_bce=0.059566 local_bce=0.068646 donor_bce=0.051040 in_sample_local_ap=0.630805 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_all mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.027967418003178732, 'khoa_quoc_te_hoc': 0.013729118735620686, 'khoa_tieng_anh': 0.05446564524913029, 'khoa_tieng_bo_dao_nha': 0.005211285607904593, 'khoa_tieng_duc': 0.01273437586831605, 'khoa_tieng_han_quoc': 0.02079026485723495, 'khoa_tieng_italia': 0.013045580340768897, 'khoa_tieng_nga': 0.009320851811098886, 'khoa_tieng_nhat': 0.02037555711157792, 'khoa_tieng_phap': 0.017396191079546087, 'khoa_tieng_tay_ban_nha': 0.009302096184410878, 'khoa_tieng_trung_quoc': 0.0401974759094395, 'khoa_viet_nam_hoc': 0.0054641392417725315} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quoc_te_hoc': 7, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 8, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 21} positive_ess=1106.886609 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal rows=7604 failures=407 prevalence=0.053524 ap=0.388487 auc=0.881080 f2_at20=0.476342 brier=0.040764 ece10=0.017026 rho=0.250000 donor_rows=4095 effective_n=40478.626983 seconds=1.045509 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal in_sample_bce=0.062389 local_bce=0.070061 donor_bce=0.039376 in_sample_local_ap=0.617699 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_pool_equal mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quoc_te_hoc': 7, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 8, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 21} positive_ess=1263.442962 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior rows=7604 failures=407 prevalence=0.053524 ap=0.382019 auc=0.878473 f2_at20=0.479517 brier=0.041019 ece10=0.017422 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=1.006292 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior in_sample_bce=0.062357 local_bce=0.062357 donor_bce=nan in_sample_local_ap=0.691040 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_language_prior mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor rows=7604 failures=407 prevalence=0.053524 ap=0.379358 auc=0.878882 f2_at20=0.474754 brier=0.041295 ece10=0.018734 rho=0.250000 donor_rows=4096 effective_n=40484.733355 seconds=1.036831 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor in_sample_bce=0.050988 local_bce=0.065214 donor_bce=0.008311 in_sample_local_ap=0.657651 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=hgb_random_donor mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814, np.str_('khoa_tieng_han_quoc'): 23} positive_ess=1574.599244 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common rows=7604 failures=407 prevalence=0.053524 ap=0.382019 auc=0.878473 f2_at20=0.479517 brier=0.041019 ece10=0.017422 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=0.963779 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common in_sample_bce=0.062357 local_bce=0.062357 donor_bce=nan in_sample_local_ap=0.691040 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_common mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native rows=7604 failures=407 prevalence=0.053524 ap=0.382596 auc=0.876793 f2_at20=0.476342 brier=0.040889 ece10=0.016996 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=1.833209 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native in_sample_bce=0.062151 local_bce=0.062151 donor_bce=nan in_sample_local_ap=0.692355 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator rows=7604 failures=407 prevalence=0.053524 ap=0.382596 auc=0.876793 f2_at20=0.476342 brier=0.040889 ece10=0.016996 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=3.446722 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator in_sample_bce=0.062151 local_bce=0.062151 donor_bce=nan in_sample_local_ap=0.692355 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_cache_nan_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator rows=7604 failures=407 prevalence=0.053524 ap=0.383433 auc=0.876381 f2_at20=0.476342 brier=0.040855 ece10=0.016396 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=3.516417 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator in_sample_bce=0.062109 local_bce=0.062109 donor_bce=nan in_sample_local_ap=0.693034 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_naive_zero_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator rows=7604 failures=407 prevalence=0.053524 ap=0.382596 auc=0.876793 f2_at20=0.476342 brier=0.040889 ece10=0.016996 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=3.526135 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator in_sample_bce=0.062151 local_bce=0.062151 donor_bce=nan in_sample_local_ap=0.692355 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=native_placebo_indicator mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan rows=7604 failures=407 prevalence=0.053524 ap=0.382596 auc=0.876793 f2_at20=0.476342 brier=0.040889 ece10=0.016996 rho=0.000000 donor_rows=0 effective_n=59575.000000 seconds=3.623726 [TRAINING_INSIGHT] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan in_sample_bce=0.062151 local_bce=0.062151 donor_bce=nan in_sample_local_ap=0.692355 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=suite32_selective_native_cache_nan mass={'khoa_quan_tri_kinh_doanh_va_du_lich': 1.0} source_failures={'khoa_quan_tri_kinh_doanh_va_du_lich': 1814} positive_ess=1814.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich model=committee_reference ap=0.372871 [FACULTY_COMPLETE] fold=5 faculty=khoa_quan_tri_kinh_doanh_va_du_lich models=13 positive_rows=407 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_quoc_te_hoc fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_quoc_te_hoc fold=5 origin=6066 ap=0.035602 rows=2363 failures=31 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.012461 ap=0.048063 base_ap=0.035602 donor_failures=122 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.030105 ap=0.065707 base_ap=0.035602 donor_failures=96 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6066 delta_ap=0.020956 ap=0.056558 base_ap=0.035602 donor_failures=68 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.003725 ap=0.039327 base_ap=0.035602 donor_failures=121 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.017747 ap=0.053348 base_ap=0.035602 donor_failures=73 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=0.001812 ap=0.037414 base_ap=0.035602 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6066 delta_ap=0.018669 ap=0.054271 base_ap=0.035602 donor_failures=118 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.009543 ap=0.045145 base_ap=0.035602 donor_failures=164 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6066 delta_ap=-0.006258 ap=0.029343 base_ap=0.035602 donor_failures=49 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6066 delta_ap=0.014888 ap=0.050490 base_ap=0.035602 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.003563 ap=0.032039 base_ap=0.035602 donor_failures=91 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.048921 ap=0.084523 base_ap=0.035602 donor_failures=25 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.020482 ap=0.056084 base_ap=0.035602 donor_failures=266 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=5 origin=6067 ap=0.412807 rows=2095 failures=19 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.027686 ap=0.385121 base_ap=0.412807 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.026567 ap=0.386240 base_ap=0.412807 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6067 delta_ap=0.038313 ap=0.451120 base_ap=0.412807 donor_failures=76 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.115104 ap=0.297703 base_ap=0.412807 donor_failures=128 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6067 delta_ap=-0.020272 ap=0.392535 base_ap=0.412807 donor_failures=76 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6067 delta_ap=0.034882 ap=0.447689 base_ap=0.412807 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6067 delta_ap=0.000344 ap=0.413151 base_ap=0.412807 donor_failures=141 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6067 delta_ap=-0.054558 ap=0.358249 base_ap=0.412807 donor_failures=204 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.114992 ap=0.297815 base_ap=0.412807 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6067 delta_ap=-0.019339 ap=0.393468 base_ap=0.412807 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.169991 ap=0.242815 base_ap=0.412807 donor_failures=111 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.042107 ap=0.454914 base_ap=0.412807 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.026027 ap=0.386780 base_ap=0.412807 donor_failures=246 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=5 origin=6069 ap=0.238349 rows=2665 failures=26 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.005251 ap=0.233098 base_ap=0.238349 donor_failures=118 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.083356 ap=0.154993 base_ap=0.238349 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6069 delta_ap=-0.146742 ap=0.091606 base_ap=0.238349 donor_failures=70 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.023497 ap=0.214851 base_ap=0.238349 donor_failures=146 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6069 delta_ap=-0.145831 ap=0.092518 base_ap=0.238349 donor_failures=80 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.121229 ap=0.117119 base_ap=0.238349 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6069 delta_ap=-0.083635 ap=0.154714 base_ap=0.238349 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6069 delta_ap=-0.012236 ap=0.226113 base_ap=0.238349 donor_failures=182 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6069 delta_ap=-0.127218 ap=0.111131 base_ap=0.238349 donor_failures=29 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6069 delta_ap=-0.146110 ap=0.092239 base_ap=0.238349 donor_failures=50 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.072753 ap=0.165595 base_ap=0.238349 donor_failures=110 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.152969 ap=0.085380 base_ap=0.238349 donor_failures=20 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.080980 ap=0.157369 base_ap=0.238349 donor_failures=237 [SCREEN_BASE] target=khoa_quoc_te_hoc fold=5 origin=6070 ap=0.122908 rows=2202 failures=35 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=0.032561 ap=0.155469 base_ap=0.122908 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.011308 ap=0.111600 base_ap=0.122908 donor_failures=112 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_anh origin=6070 delta_ap=-0.029908 ap=0.093000 base_ap=0.122908 donor_failures=77 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=-0.049060 ap=0.073848 base_ap=0.122908 donor_failures=144 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_duc origin=6070 delta_ap=-0.019269 ap=0.103638 base_ap=0.122908 donor_failures=75 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_han_quoc origin=6070 delta_ap=0.001514 ap=0.124421 base_ap=0.122908 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_italia origin=6070 delta_ap=0.011635 ap=0.134543 base_ap=0.122908 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nga origin=6070 delta_ap=0.005071 ap=0.127979 base_ap=0.122908 donor_failures=200 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_nhat origin=6070 delta_ap=0.017938 ap=0.140846 base_ap=0.122908 donor_failures=36 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_phap origin=6070 delta_ap=-0.008450 ap=0.114457 base_ap=0.122908 donor_failures=59 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.030231 ap=0.092677 base_ap=0.122908 donor_failures=106 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_tieng_trung_quoc origin=6070 delta_ap=0.009577 ap=0.132485 base_ap=0.122908 donor_failures=28 [TRANSFER_EDGE] fold=5 target=khoa_quoc_te_hoc source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.011949 ap=0.110959 base_ap=0.122908 donor_failures=258 [POLICY_REPLAY] fold=5 faculty=khoa_quoc_te_hoc origin=6069 policy_sources=khoa_tieng_trung_quoc,khoa_tieng_anh,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.238349 policy_ap=0.133983 delta_ap=-0.104366 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_quoc_te_hoc origin=6070 policy_sources= policy_donor_count=0 base_ap=0.122908 policy_ap=0.122908 delta_ap=0.000000 policy_donor_rows=0 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_quoc_te_hoc selected=[] reason=prequential_policy_delta_-0.052183_below_threshold replay_mean=-0.052183 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_local_common rows=2674 failures=58 prevalence=0.021690 ap=0.201403 auc=0.853580 f2_at20=0.273794 brier=0.019353 ece10=0.011671 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=0.607009 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=hgb_local_common in_sample_bce=0.012016 local_bce=0.012016 donor_bce=nan in_sample_local_ap=0.938826 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_local_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_local_native rows=2674 failures=58 prevalence=0.021690 ap=0.197416 auc=0.849174 f2_at20=0.280313 brier=0.020176 ece10=0.013615 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=0.941862 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=hgb_local_native in_sample_bce=0.012028 local_bce=0.012028 donor_bce=nan in_sample_local_ap=0.936617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_local_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_pool_all rows=2674 failures=58 prevalence=0.021690 ap=0.217835 auc=0.858807 f2_at20=0.267275 brier=0.018893 ece10=0.008546 rho=0.250000 donor_rows=4095 effective_n=19575.758628 seconds=0.743436 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=hgb_pool_all in_sample_bce=0.019075 local_bce=0.017729 donor_bce=0.042084 in_sample_local_ap=0.844394 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_pool_all mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.025181822503208634, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03726204206623418, 'khoa_tieng_anh': 0.04904078778493031, 'khoa_tieng_bo_dao_nha': 0.004692233983732922, 'khoa_tieng_duc': 0.011466013515034913, 'khoa_tieng_han_quoc': 0.018719524246753213, 'khoa_tieng_italia': 0.011746221569515365, 'khoa_tieng_nga': 0.008392481417452458, 'khoa_tieng_nhat': 0.018346121995581718, 'khoa_tieng_phap': 0.015663505152575787, 'khoa_tieng_tay_ban_nha': 0.008375593878454752, 'khoa_tieng_trung_quoc': 0.036193748858527455, 'khoa_viet_nam_hoc': 0.004919903027998288} source_failures={'khoa_quoc_te_hoc': 209, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 18} positive_ess=242.170002 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_pool_equal rows=2674 failures=58 prevalence=0.021690 ap=0.234174 auc=0.861624 f2_at20=0.280313 brier=0.019074 ece10=0.010065 rho=0.250000 donor_rows=4095 effective_n=22871.072961 seconds=0.665580 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=hgb_pool_equal in_sample_bce=0.022094 local_bce=0.018657 donor_bce=0.032405 in_sample_local_ap=0.835285 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_pool_equal mass={'khoa_quoc_te_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_quoc_te_hoc': 209, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 1, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 18} positive_ess=299.875473 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_language_prior rows=2674 failures=58 prevalence=0.021690 ap=0.201403 auc=0.853580 f2_at20=0.273794 brier=0.019353 ece10=0.011671 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=0.686950 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=hgb_language_prior in_sample_bce=0.012016 local_bce=0.012016 donor_bce=nan in_sample_local_ap=0.938826 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_language_prior mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_random_donor rows=2674 failures=58 prevalence=0.021690 ap=0.189667 auc=0.830282 f2_at20=0.254237 brier=0.019526 ece10=0.011860 rho=0.250000 donor_rows=4096 effective_n=22873.022250 seconds=0.674584 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=hgb_random_donor in_sample_bce=0.012582 local_bce=0.014547 donor_bce=0.006688 in_sample_local_ap=0.896517 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=hgb_random_donor mass={'khoa_quoc_te_hoc': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_quoc_te_hoc': 209, np.str_('khoa_tieng_han_quoc'): 23} positive_ess=225.373161 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_common rows=2674 failures=58 prevalence=0.021690 ap=0.201403 auc=0.853580 f2_at20=0.273794 brier=0.019353 ece10=0.011671 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=0.697883 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_common in_sample_bce=0.012016 local_bce=0.012016 donor_bce=nan in_sample_local_ap=0.938826 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_common mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_native rows=2674 failures=58 prevalence=0.021690 ap=0.197416 auc=0.849174 f2_at20=0.280313 brier=0.020176 ece10=0.013615 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=0.928535 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_native in_sample_bce=0.012028 local_bce=0.012028 donor_bce=nan in_sample_local_ap=0.936617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_native mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator rows=2674 failures=58 prevalence=0.021690 ap=0.197416 auc=0.849174 f2_at20=0.280313 brier=0.020176 ece10=0.013615 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=1.613233 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator in_sample_bce=0.012028 local_bce=0.012028 donor_bce=nan in_sample_local_ap=0.936617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=native_cache_nan_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator rows=2674 failures=58 prevalence=0.021690 ap=0.197416 auc=0.849174 f2_at20=0.280313 brier=0.020176 ece10=0.013615 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=1.588459 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator in_sample_bce=0.012028 local_bce=0.012028 donor_bce=nan in_sample_local_ap=0.936617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=native_naive_zero_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=native_placebo_indicator rows=2674 failures=58 prevalence=0.021690 ap=0.197416 auc=0.849174 f2_at20=0.280313 brier=0.020176 ece10=0.013615 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=1.616396 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=native_placebo_indicator in_sample_bce=0.012028 local_bce=0.012028 donor_bce=nan in_sample_local_ap=0.936617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=native_placebo_indicator mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan rows=2674 failures=58 prevalence=0.021690 ap=0.197416 auc=0.849174 f2_at20=0.280313 brier=0.020176 ece10=0.013615 rho=0.000000 donor_rows=0 effective_n=19764.000000 seconds=1.487871 [TRAINING_INSIGHT] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.012028 local_bce=0.012028 donor_bce=nan in_sample_local_ap=0.936617 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_quoc_te_hoc model=suite32_selective_native_cache_nan mass={'khoa_quoc_te_hoc': 1.0} source_failures={'khoa_quoc_te_hoc': 209} positive_ess=209.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_quoc_te_hoc model=committee_reference ap=0.163461 [FACULTY_COMPLETE] fold=5 faculty=khoa_quoc_te_hoc models=13 positive_rows=58 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_anh fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_anh fold=5 origin=6066 ap=0.519021 rows=8164 failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.073196 ap=0.592216 base_ap=0.519021 donor_failures=136 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.081830 ap=0.600851 base_ap=0.519021 donor_failures=117 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6066 delta_ap=0.054427 ap=0.573448 base_ap=0.519021 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.019350 ap=0.499670 base_ap=0.519021 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_duc origin=6066 delta_ap=0.056576 ap=0.575597 base_ap=0.519021 donor_failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6066 delta_ap=0.032795 ap=0.551816 base_ap=0.519021 donor_failures=14 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_italia origin=6066 delta_ap=0.052038 ap=0.571059 base_ap=0.519021 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nga origin=6066 delta_ap=0.072213 ap=0.591233 base_ap=0.519021 donor_failures=157 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6066 delta_ap=-0.014389 ap=0.504632 base_ap=0.519021 donor_failures=52 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_phap origin=6066 delta_ap=0.036496 ap=0.555517 base_ap=0.519021 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.073731 ap=0.592752 base_ap=0.519021 donor_failures=84 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.070014 ap=0.589035 base_ap=0.519021 donor_failures=18 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6066 delta_ap=0.044600 ap=0.563621 base_ap=0.519021 donor_failures=264 [SCREEN_BASE] target=khoa_tieng_anh fold=5 origin=6067 ap=0.693816 rows=8328 failures=122 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.075640 ap=0.618175 base_ap=0.693816 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.008486 ap=0.685329 base_ap=0.693816 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.024547 ap=0.669269 base_ap=0.693816 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.006654 ap=0.687162 base_ap=0.693816 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_duc origin=6067 delta_ap=-0.042542 ap=0.651274 base_ap=0.693816 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.046573 ap=0.647242 base_ap=0.693816 donor_failures=16 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_italia origin=6067 delta_ap=-0.015592 ap=0.678223 base_ap=0.693816 donor_failures=147 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nga origin=6067 delta_ap=-0.038212 ap=0.655603 base_ap=0.693816 donor_failures=203 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6067 delta_ap=-0.012139 ap=0.681676 base_ap=0.693816 donor_failures=35 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_phap origin=6067 delta_ap=-0.059197 ap=0.634618 base_ap=0.693816 donor_failures=52 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.030188 ap=0.663628 base_ap=0.693816 donor_failures=98 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.014794 ap=0.679022 base_ap=0.693816 donor_failures=23 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.078373 ap=0.615442 base_ap=0.693816 donor_failures=245 [SCREEN_BASE] target=khoa_tieng_anh fold=5 origin=6069 ap=0.360769 rows=7279 failures=101 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.019418 ap=0.380186 base_ap=0.360769 donor_failures=105 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.013867 ap=0.374635 base_ap=0.360769 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.005944 ap=0.354825 base_ap=0.360769 donor_failures=56 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.006767 ap=0.354001 base_ap=0.360769 donor_failures=144 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_duc origin=6069 delta_ap=0.053192 ap=0.413961 base_ap=0.360769 donor_failures=84 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6069 delta_ap=0.013133 ap=0.373902 base_ap=0.360769 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_italia origin=6069 delta_ap=0.010135 ap=0.370904 base_ap=0.360769 donor_failures=147 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nga origin=6069 delta_ap=-0.021706 ap=0.339063 base_ap=0.360769 donor_failures=180 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6069 delta_ap=-0.022811 ap=0.337958 base_ap=0.360769 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_phap origin=6069 delta_ap=-0.002389 ap=0.358380 base_ap=0.360769 donor_failures=48 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.012425 ap=0.373194 base_ap=0.360769 donor_failures=103 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.013412 ap=0.347357 base_ap=0.360769 donor_failures=35 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6069 delta_ap=0.051513 ap=0.412281 base_ap=0.360769 donor_failures=247 [SCREEN_BASE] target=khoa_tieng_anh fold=5 origin=6070 ap=0.407463 rows=6636 failures=68 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.196797 ap=0.210666 base_ap=0.407463 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.114727 ap=0.292736 base_ap=0.407463 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.105623 ap=0.301840 base_ap=0.407463 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=-0.205965 ap=0.201498 base_ap=0.407463 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_duc origin=6070 delta_ap=-0.130540 ap=0.276923 base_ap=0.407463 donor_failures=89 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_han_quoc origin=6070 delta_ap=-0.055350 ap=0.352113 base_ap=0.407463 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_italia origin=6070 delta_ap=-0.136903 ap=0.270560 base_ap=0.407463 donor_failures=151 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nga origin=6070 delta_ap=-0.066857 ap=0.340606 base_ap=0.407463 donor_failures=174 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_nhat origin=6070 delta_ap=-0.156636 ap=0.250827 base_ap=0.407463 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_phap origin=6070 delta_ap=-0.113634 ap=0.293829 base_ap=0.407463 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.142381 ap=0.265082 base_ap=0.407463 donor_failures=104 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.024476 ap=0.382987 base_ap=0.407463 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_anh source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.137065 ap=0.270398 base_ap=0.407463 donor_failures=229 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_anh origin=6069 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_trung_quoc,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.360769 policy_ap=0.358423 delta_ap=-0.002345 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_anh origin=6070 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_duc,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.407463 policy_ap=0.240314 delta_ap=-0.167149 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_anh selected=[] reason=no_stable_positive_directed_edge replay_mean=-0.084747 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=hgb_local_common rows=8345 failures=132 prevalence=0.015818 ap=0.108452 auc=0.857245 f2_at20=0.200273 brier=0.015109 ece10=0.009663 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=1.183930 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=hgb_local_common in_sample_bce=0.023889 local_bce=0.023889 donor_bce=nan in_sample_local_ap=0.862758 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=hgb_local_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=hgb_local_native rows=8345 failures=132 prevalence=0.015818 ap=0.124114 auc=0.822553 f2_at20=0.200273 brier=0.015027 ece10=0.009877 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=3.618413 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=hgb_local_native in_sample_bce=0.023682 local_bce=0.023682 donor_bce=nan in_sample_local_ap=0.862469 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=hgb_local_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=hgb_pool_all rows=8345 failures=132 prevalence=0.015818 ap=0.145794 auc=0.842812 f2_at20=0.202549 brier=0.014643 ece10=0.007371 rho=0.250000 donor_rows=4095 effective_n=35179.749456 seconds=1.201881 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=hgb_pool_all in_sample_bce=0.029625 local_bce=0.030697 donor_bce=0.041599 in_sample_local_ap=0.762997 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=hgb_pool_all mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.02951166950096757, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04366900252155046, 'khoa_quoc_te_hoc': 0.014487187005219023, 'khoa_tieng_bo_dao_nha': 0.005499032428311734, 'khoa_tieng_duc': 0.013437518325221368, 'khoa_tieng_han_quoc': 0.021938222013721927, 'khoa_tieng_italia': 0.013765906292148009, 'khoa_tieng_nga': 0.00983551281299478, 'khoa_tieng_nhat': 0.02150061572743799, 'khoa_tieng_phap': 0.018356740749428252, 'khoa_tieng_tay_ban_nha': 0.009815721573916613, 'khoa_tieng_trung_quoc': 0.042417023397642646, 'khoa_viet_nam_hoc': 0.005765847651439629} source_failures={'khoa_tieng_anh': 1260, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 4, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 20} positive_ess=540.421994 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=hgb_pool_equal rows=8345 failures=132 prevalence=0.015818 ap=0.183021 auc=0.860715 f2_at20=0.200273 brier=0.014388 ece10=0.008097 rho=0.250000 donor_rows=4095 effective_n=44569.993927 seconds=1.284135 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=hgb_pool_equal in_sample_bce=0.031878 local_bce=0.031162 donor_bce=0.034025 in_sample_local_ap=0.760102 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=hgb_pool_equal mass={'khoa_tieng_anh': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_anh': 1260, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 8, 'khoa_tieng_nga': 17, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 4, 'khoa_tieng_trung_quoc': 3, 'khoa_viet_nam_hoc': 20} positive_ess=672.818960 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=hgb_language_prior rows=8345 failures=132 prevalence=0.015818 ap=0.113428 auc=0.832494 f2_at20=0.191170 brier=0.015100 ece10=0.009921 rho=0.250000 donor_rows=4095 effective_n=44569.993927 seconds=1.206260 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=hgb_language_prior in_sample_bce=0.027443 local_bce=0.029838 donor_bce=0.020257 in_sample_local_ap=0.776693 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=hgb_language_prior mass={'khoa_tieng_anh': 0.75, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_anh': 1260, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 25, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 0} positive_ess=693.773018 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=hgb_random_donor rows=8345 failures=132 prevalence=0.015818 ap=0.115315 auc=0.806291 f2_at20=0.191170 brier=0.015035 ece10=0.008680 rho=0.250000 donor_rows=4096 effective_n=44577.397194 seconds=1.282832 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=hgb_random_donor in_sample_bce=0.022469 local_bce=0.026871 donor_bce=0.009263 in_sample_local_ap=0.818260 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=hgb_random_donor mass={'khoa_tieng_anh': 0.75, np.str_('khoa_tieng_han_quoc'): 0.25} source_failures={'khoa_tieng_anh': 1260, np.str_('khoa_tieng_han_quoc'): 33} positive_ess=830.613757 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=suite32_selective_common rows=8345 failures=132 prevalence=0.015818 ap=0.108452 auc=0.857245 f2_at20=0.200273 brier=0.015109 ece10=0.009663 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=1.190914 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=suite32_selective_common in_sample_bce=0.023889 local_bce=0.023889 donor_bce=nan in_sample_local_ap=0.862758 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=suite32_selective_common mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=suite32_selective_native rows=8345 failures=132 prevalence=0.015818 ap=0.124114 auc=0.822553 f2_at20=0.200273 brier=0.015027 ece10=0.009877 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=3.616839 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=suite32_selective_native in_sample_bce=0.023682 local_bce=0.023682 donor_bce=nan in_sample_local_ap=0.862469 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=suite32_selective_native mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=native_cache_nan_indicator rows=8345 failures=132 prevalence=0.015818 ap=0.124114 auc=0.822553 f2_at20=0.200273 brier=0.015027 ece10=0.009877 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=7.824657 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=native_cache_nan_indicator in_sample_bce=0.023682 local_bce=0.023682 donor_bce=nan in_sample_local_ap=0.862469 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=native_cache_nan_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=native_naive_zero_indicator rows=8345 failures=132 prevalence=0.015818 ap=0.124114 auc=0.822553 f2_at20=0.200273 brier=0.015027 ece10=0.009877 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=12.334225 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=native_naive_zero_indicator in_sample_bce=0.023682 local_bce=0.023682 donor_bce=nan in_sample_local_ap=0.862469 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=native_naive_zero_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=native_placebo_indicator rows=8345 failures=132 prevalence=0.015818 ap=0.124114 auc=0.822553 f2_at20=0.200273 brier=0.015027 ece10=0.009877 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=7.807203 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=native_placebo_indicator in_sample_bce=0.023682 local_bce=0.023682 donor_bce=nan in_sample_local_ap=0.862469 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=native_placebo_indicator mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan rows=8345 failures=132 prevalence=0.015818 ap=0.124114 auc=0.822553 f2_at20=0.200273 brier=0.015027 ece10=0.009877 rho=0.000000 donor_rows=0 effective_n=78407.000000 seconds=7.845709 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan in_sample_bce=0.023682 local_bce=0.023682 donor_bce=nan in_sample_local_ap=0.862469 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_anh model=suite32_selective_native_cache_nan mass={'khoa_tieng_anh': 1.0} source_failures={'khoa_tieng_anh': 1260} positive_ess=1260.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_anh model=committee_reference ap=0.153075 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_anh models=13 positive_rows=132 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_bo_dao_nha fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=5 origin=6066 ap=0.119890 rows=768 failures=30 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.006146 ap=0.126036 base_ap=0.119890 donor_failures=119 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.050770 ap=0.170660 base_ap=0.119890 donor_failures=114 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.082477 ap=0.202368 base_ap=0.119890 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.021383 ap=0.098507 base_ap=0.119890 donor_failures=63 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.011313 ap=0.108577 base_ap=0.119890 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.003622 ap=0.116268 base_ap=0.119890 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6066 delta_ap=0.029016 ap=0.148907 base_ap=0.119890 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6066 delta_ap=0.084776 ap=0.204667 base_ap=0.119890 donor_failures=178 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6066 delta_ap=0.002190 ap=0.122081 base_ap=0.119890 donor_failures=54 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6066 delta_ap=0.049659 ap=0.169549 base_ap=0.119890 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.089194 ap=0.209084 base_ap=0.119890 donor_failures=103 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.014657 ap=0.105234 base_ap=0.119890 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=0.028055 ap=0.147945 base_ap=0.119890 donor_failures=259 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=5 origin=6067 ap=0.411054 rows=716 failures=30 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.061056 ap=0.472111 base_ap=0.411054 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.030515 ap=0.441569 base_ap=0.411054 donor_failures=114 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6067 delta_ap=0.070055 ap=0.481109 base_ap=0.411054 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6067 delta_ap=0.020791 ap=0.431845 base_ap=0.411054 donor_failures=78 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6067 delta_ap=0.052750 ap=0.463804 base_ap=0.411054 donor_failures=69 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6067 delta_ap=0.013278 ap=0.424333 base_ap=0.411054 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6067 delta_ap=-0.038434 ap=0.372620 base_ap=0.411054 donor_failures=141 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6067 delta_ap=-0.003558 ap=0.407496 base_ap=0.411054 donor_failures=212 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6067 delta_ap=-0.022287 ap=0.388767 base_ap=0.411054 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6067 delta_ap=0.037772 ap=0.448826 base_ap=0.411054 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.109593 ap=0.520647 base_ap=0.411054 donor_failures=106 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.051594 ap=0.462649 base_ap=0.411054 donor_failures=18 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6067 delta_ap=0.013685 ap=0.424739 base_ap=0.411054 donor_failures=252 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=5 origin=6069 ap=0.534364 rows=1032 failures=50 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.281416 ap=0.252948 base_ap=0.534364 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.184201 ap=0.350163 base_ap=0.534364 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.197912 ap=0.336451 base_ap=0.534364 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6069 delta_ap=-0.212758 ap=0.321606 base_ap=0.534364 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6069 delta_ap=-0.159103 ap=0.375261 base_ap=0.534364 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.164727 ap=0.369637 base_ap=0.534364 donor_failures=17 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6069 delta_ap=-0.103587 ap=0.430777 base_ap=0.534364 donor_failures=149 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6069 delta_ap=-0.143652 ap=0.390712 base_ap=0.534364 donor_failures=185 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6069 delta_ap=-0.133283 ap=0.401080 base_ap=0.534364 donor_failures=33 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6069 delta_ap=-0.072184 ap=0.462180 base_ap=0.534364 donor_failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.218959 ap=0.315404 base_ap=0.534364 donor_failures=97 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.216517 ap=0.317847 base_ap=0.534364 donor_failures=18 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.178684 ap=0.355680 base_ap=0.534364 donor_failures=264 [SCREEN_BASE] target=khoa_tieng_bo_dao_nha fold=5 origin=6070 ap=0.570080 rows=869 failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=0.080873 ap=0.650952 base_ap=0.570080 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=0.028000 ap=0.598080 base_ap=0.570080 donor_failures=118 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_quoc_te_hoc origin=6070 delta_ap=0.008555 ap=0.578635 base_ap=0.570080 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_anh origin=6070 delta_ap=0.052150 ap=0.622230 base_ap=0.570080 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_duc origin=6070 delta_ap=0.050316 ap=0.620395 base_ap=0.570080 donor_failures=91 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_han_quoc origin=6070 delta_ap=0.006321 ap=0.576401 base_ap=0.570080 donor_failures=32 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_italia origin=6070 delta_ap=0.019920 ap=0.590000 base_ap=0.570080 donor_failures=137 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nga origin=6070 delta_ap=0.058922 ap=0.629002 base_ap=0.570080 donor_failures=169 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_nhat origin=6070 delta_ap=0.047760 ap=0.617839 base_ap=0.570080 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_phap origin=6070 delta_ap=0.015417 ap=0.585496 base_ap=0.570080 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=0.060890 ap=0.630970 base_ap=0.570080 donor_failures=114 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_tieng_trung_quoc origin=6070 delta_ap=0.029137 ap=0.599216 base_ap=0.570080 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_bo_dao_nha source=khoa_viet_nam_hoc origin=6070 delta_ap=0.031379 ap=0.601459 base_ap=0.570080 donor_failures=237 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_bo_dao_nha origin=6069 policy_sources=khoa_tieng_tay_ban_nha,khoa_quoc_te_hoc,khoa_tieng_phap policy_donor_count=3 base_ap=0.534364 policy_ap=0.389577 delta_ap=-0.144786 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_bo_dao_nha origin=6070 policy_sources=khoa_tieng_phap policy_donor_count=1 base_ap=0.570080 policy_ap=0.585496 delta_ap=0.015417 policy_donor_rows=4096 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_bo_dao_nha selected=[] reason=prequential_policy_delta_-0.064685_below_threshold replay_mean=-0.064685 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common rows=1138 failures=80 prevalence=0.070299 ap=0.346110 auc=0.883684 f2_at20=0.574818 brier=0.054532 ece10=0.031146 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=0.440091 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common in_sample_bce=0.040573 local_bce=0.040573 donor_bce=nan in_sample_local_ap=0.937199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_local_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native rows=1138 failures=80 prevalence=0.070299 ap=0.383441 auc=0.879714 f2_at20=0.565693 brier=0.053743 ece10=0.030858 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=0.854239 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native in_sample_bce=0.040189 local_bce=0.040189 donor_bce=nan in_sample_local_ap=0.939711 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_local_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all rows=1138 failures=80 prevalence=0.070299 ap=0.343887 auc=0.883684 f2_at20=0.565693 brier=0.054413 ece10=0.027424 rho=0.250000 donor_rows=4095 effective_n=10344.797216 seconds=0.563339 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all in_sample_bce=0.047589 local_bce=0.048274 donor_bce=0.061545 in_sample_local_ap=0.908835 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_all mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.024432294005558724, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.036152949886519486, 'khoa_quoc_te_hoc': 0.011993737332054907, 'khoa_tieng_anh': 0.047581105190974964, 'khoa_tieng_duc': 0.011124731469906423, 'khoa_tieng_han_quoc': 0.018162343888437123, 'khoa_tieng_italia': 0.011396599225662375, 'khoa_tieng_nga': 0.008142682023958346, 'khoa_tieng_nhat': 0.017800055829985558, 'khoa_tieng_phap': 0.015197286177223794, 'khoa_tieng_tay_ban_nha': 0.008126297136892698, 'khoa_tieng_trung_quoc': 0.03511645406769993, 'khoa_viet_nam_hoc': 0.0047734637651256785} source_failures={'khoa_tieng_bo_dao_nha': 297, 'khoa_cong_nghe_thong_tin': 15, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 6, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 11, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 19} positive_ess=362.148179 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal rows=1138 failures=80 prevalence=0.070299 ap=0.331971 auc=0.886011 f2_at20=0.574818 brier=0.055431 ece10=0.027066 rho=0.250000 donor_rows=4095 effective_n=11081.250761 seconds=0.538523 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal in_sample_bce=0.050254 local_bce=0.048898 donor_bce=0.054322 in_sample_local_ap=0.904820 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_pool_equal mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_bo_dao_nha': 297, 'khoa_cong_nghe_thong_tin': 15, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 6, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 11, 'khoa_tieng_nhat': 6, 'khoa_tieng_phap': 5, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 19} positive_ess=383.532647 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior rows=1138 failures=80 prevalence=0.070299 ap=0.325521 auc=0.887405 f2_at20=0.565693 brier=0.056986 ece10=0.032011 rho=0.250000 donor_rows=4095 effective_n=11081.250761 seconds=0.544133 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior in_sample_bce=0.045188 local_bce=0.046902 donor_bce=0.040044 in_sample_local_ap=0.911339 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_language_prior mass={'khoa_tieng_bo_dao_nha': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_bo_dao_nha': 297, 'khoa_tieng_anh': 10, 'khoa_tieng_duc': 11, 'khoa_tieng_han_quoc': 4, 'khoa_tieng_italia': 6, 'khoa_tieng_nga': 15, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 4} positive_ess=358.118511 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor rows=1138 failures=80 prevalence=0.070299 ap=0.346842 auc=0.886017 f2_at20=0.574818 brier=0.058244 ece10=0.037952 rho=0.250000 donor_rows=4096 effective_n=11081.708335 seconds=0.518943 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor in_sample_bce=0.056288 local_bce=0.049821 donor_bce=0.075692 in_sample_local_ap=0.897307 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=hgb_random_donor mass={'khoa_tieng_bo_dao_nha': 0.75, np.str_('khoa_tieng_nga'): 0.25} source_failures={'khoa_tieng_bo_dao_nha': 297, np.str_('khoa_tieng_nga'): 191} positive_ess=464.627798 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common rows=1138 failures=80 prevalence=0.070299 ap=0.346110 auc=0.883684 f2_at20=0.574818 brier=0.054532 ece10=0.031146 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=0.464449 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common in_sample_bce=0.040573 local_bce=0.040573 donor_bce=nan in_sample_local_ap=0.937199 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_common mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native rows=1138 failures=80 prevalence=0.070299 ap=0.383441 auc=0.879714 f2_at20=0.565693 brier=0.053743 ece10=0.030858 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=0.894351 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native in_sample_bce=0.040189 local_bce=0.040189 donor_bce=nan in_sample_local_ap=0.939711 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator rows=1138 failures=80 prevalence=0.070299 ap=0.383441 auc=0.879714 f2_at20=0.565693 brier=0.053743 ece10=0.030858 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=1.585709 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator in_sample_bce=0.040189 local_bce=0.040189 donor_bce=nan in_sample_local_ap=0.939711 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_cache_nan_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator rows=1138 failures=80 prevalence=0.070299 ap=0.346835 auc=0.879407 f2_at20=0.556569 brier=0.054390 ece10=0.030459 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=1.473884 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator in_sample_bce=0.040595 local_bce=0.040595 donor_bce=nan in_sample_local_ap=0.938510 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_naive_zero_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator rows=1138 failures=80 prevalence=0.070299 ap=0.383441 auc=0.879714 f2_at20=0.565693 brier=0.053743 ece10=0.030858 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=1.502280 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator in_sample_bce=0.040189 local_bce=0.040189 donor_bce=nan in_sample_local_ap=0.939711 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=native_placebo_indicator mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan rows=1138 failures=80 prevalence=0.070299 ap=0.383441 auc=0.879714 f2_at20=0.565693 brier=0.053743 ece10=0.030858 rho=0.000000 donor_rows=0 effective_n=7502.000000 seconds=1.515736 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan in_sample_bce=0.040189 local_bce=0.040189 donor_bce=nan in_sample_local_ap=0.939711 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_bo_dao_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_bo_dao_nha': 1.0} source_failures={'khoa_tieng_bo_dao_nha': 297} positive_ess=297.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_bo_dao_nha model=committee_reference ap=0.312913 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_bo_dao_nha models=13 positive_rows=80 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_duc fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_duc fold=5 origin=6066 ap=0.143416 rows=1900 failures=46 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.051752 ap=0.195168 base_ap=0.143416 donor_failures=127 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.042234 ap=0.185650 base_ap=0.143416 donor_failures=105 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6066 delta_ap=0.037030 ap=0.180446 base_ap=0.143416 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_anh origin=6066 delta_ap=0.023801 ap=0.167217 base_ap=0.143416 donor_failures=75 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.040812 ap=0.184228 base_ap=0.143416 donor_failures=121 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6066 delta_ap=0.041314 ap=0.184730 base_ap=0.143416 donor_failures=25 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_italia origin=6066 delta_ap=0.018144 ap=0.161560 base_ap=0.143416 donor_failures=128 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nga origin=6066 delta_ap=0.038562 ap=0.181978 base_ap=0.143416 donor_failures=164 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6066 delta_ap=0.030050 ap=0.173466 base_ap=0.143416 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_phap origin=6066 delta_ap=0.032530 ap=0.175947 base_ap=0.143416 donor_failures=48 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.056274 ap=0.199690 base_ap=0.143416 donor_failures=88 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.002411 ap=0.141005 base_ap=0.143416 donor_failures=23 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.002012 ap=0.141404 base_ap=0.143416 donor_failures=263 [SCREEN_BASE] target=khoa_tieng_duc fold=5 origin=6067 ap=0.321965 rows=1893 failures=53 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.103454 ap=0.425419 base_ap=0.321965 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.053502 ap=0.375467 base_ap=0.321965 donor_failures=105 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6067 delta_ap=0.019184 ap=0.341149 base_ap=0.321965 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_anh origin=6067 delta_ap=0.051951 ap=0.373916 base_ap=0.321965 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.034826 ap=0.287139 base_ap=0.321965 donor_failures=127 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6067 delta_ap=0.070491 ap=0.392456 base_ap=0.321965 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_italia origin=6067 delta_ap=0.030660 ap=0.352625 base_ap=0.321965 donor_failures=138 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nga origin=6067 delta_ap=0.098620 ap=0.420585 base_ap=0.321965 donor_failures=198 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6067 delta_ap=0.009237 ap=0.331202 base_ap=0.321965 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_phap origin=6067 delta_ap=-0.013528 ap=0.308437 base_ap=0.321965 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.075295 ap=0.397260 base_ap=0.321965 donor_failures=103 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.047865 ap=0.369830 base_ap=0.321965 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6067 delta_ap=0.007850 ap=0.329815 base_ap=0.321965 donor_failures=255 [SCREEN_BASE] target=khoa_tieng_duc fold=5 origin=6069 ap=0.313135 rows=1909 failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.039607 ap=0.352743 base_ap=0.313135 donor_failures=139 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.050719 ap=0.363855 base_ap=0.313135 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.033607 ap=0.279529 base_ap=0.313135 donor_failures=46 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_anh origin=6069 delta_ap=0.014091 ap=0.327226 base_ap=0.313135 donor_failures=75 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.057473 ap=0.370609 base_ap=0.313135 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6069 delta_ap=0.002648 ap=0.315784 base_ap=0.313135 donor_failures=24 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_italia origin=6069 delta_ap=-0.043977 ap=0.269158 base_ap=0.313135 donor_failures=141 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nga origin=6069 delta_ap=0.010167 ap=0.323302 base_ap=0.313135 donor_failures=186 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6069 delta_ap=-0.041629 ap=0.271506 base_ap=0.313135 donor_failures=44 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_phap origin=6069 delta_ap=-0.033122 ap=0.280013 base_ap=0.313135 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.049424 ap=0.263712 base_ap=0.313135 donor_failures=105 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.052497 ap=0.365633 base_ap=0.313135 donor_failures=33 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.087633 ap=0.225502 base_ap=0.313135 donor_failures=253 [SCREEN_BASE] target=khoa_tieng_duc fold=5 origin=6070 ap=0.437746 rows=1746 failures=44 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.034787 ap=0.402959 base_ap=0.437746 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.014877 ap=0.422869 base_ap=0.437746 donor_failures=111 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.035638 ap=0.402108 base_ap=0.437746 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_anh origin=6070 delta_ap=-0.070623 ap=0.367123 base_ap=0.437746 donor_failures=54 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=-0.015081 ap=0.422665 base_ap=0.437746 donor_failures=143 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_han_quoc origin=6070 delta_ap=0.001681 ap=0.439427 base_ap=0.437746 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_italia origin=6070 delta_ap=-0.078142 ap=0.359603 base_ap=0.437746 donor_failures=144 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nga origin=6070 delta_ap=-0.077869 ap=0.359876 base_ap=0.437746 donor_failures=201 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_nhat origin=6070 delta_ap=-0.028091 ap=0.409655 base_ap=0.437746 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_phap origin=6070 delta_ap=0.018308 ap=0.456054 base_ap=0.437746 donor_failures=52 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.005607 ap=0.432139 base_ap=0.437746 donor_failures=115 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.006990 ap=0.430756 base_ap=0.437746 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_duc source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.107659 ap=0.330086 base_ap=0.437746 donor_failures=239 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_duc origin=6069 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nga,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.313135 policy_ap=0.282979 delta_ap=-0.030157 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_duc origin=6070 policy_sources=khoa_cong_nghe_thong_tin,khoa_tieng_nga,khoa_quan_tri_kinh_doanh_va_du_lich policy_donor_count=3 base_ap=0.437746 policy_ap=0.409941 delta_ap=-0.027805 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_duc selected=[] reason=prequential_policy_delta_-0.028981_below_threshold replay_mean=-0.028981 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=hgb_local_common rows=1975 failures=59 prevalence=0.029873 ap=0.122942 auc=0.839142 f2_at20=0.316957 brier=0.028650 ece10=0.020076 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=0.590481 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=hgb_local_common in_sample_bce=0.025762 local_bce=0.025762 donor_bce=nan in_sample_local_ap=0.892413 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=hgb_local_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=hgb_local_native rows=1975 failures=59 prevalence=0.029873 ap=0.134144 auc=0.852367 f2_at20=0.301109 brier=0.028198 ece10=0.018802 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=1.123190 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=hgb_local_native in_sample_bce=0.025526 local_bce=0.025526 donor_bce=nan in_sample_local_ap=0.894120 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=hgb_local_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=hgb_pool_all rows=1975 failures=59 prevalence=0.029873 ap=0.204111 auc=0.855384 f2_at20=0.324881 brier=0.026302 ece10=0.013296 rho=0.250000 donor_rows=4095 effective_n=18792.458088 seconds=0.612505 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=hgb_pool_all in_sample_bce=0.032417 local_bce=0.031909 donor_bce=0.058831 in_sample_local_ap=0.827104 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=hgb_pool_all mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02509192665810762, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03712902140177247, 'khoa_quoc_te_hoc': 0.01231754895484064, 'khoa_tieng_anh': 0.04886571852368903, 'khoa_tieng_bo_dao_nha': 0.004675483316090593, 'khoa_tieng_han_quoc': 0.018652697969511512, 'khoa_tieng_italia': 0.011704289079736249, 'khoa_tieng_nga': 0.008362521345681629, 'khoa_tieng_nhat': 0.018280628716018297, 'khoa_tieng_phap': 0.015607588467722837, 'khoa_tieng_tay_ban_nha': 0.00834569409301108, 'khoa_tieng_trung_quoc': 0.03606454186246526, 'khoa_viet_nam_hoc': 0.0049023396113527865} source_failures={'khoa_tieng_duc': 362, 'khoa_cong_nghe_thong_tin': 14, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 21} positive_ess=397.550900 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=hgb_pool_equal rows=1975 failures=59 prevalence=0.029873 ap=0.198806 auc=0.848139 f2_at20=0.309033 brier=0.026632 ece10=0.012580 rho=0.250000 donor_rows=4095 effective_n=21764.412633 seconds=0.623971 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=hgb_pool_equal in_sample_bce=0.036493 local_bce=0.032878 donor_bce=0.047337 in_sample_local_ap=0.826694 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=hgb_pool_equal mass={'khoa_tieng_duc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_duc': 362, 'khoa_cong_nghe_thong_tin': 14, 'khoa_quan_tri_kinh_doanh_va_du_lich': 7, 'khoa_quoc_te_hoc': 5, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 12, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 1, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 4, 'khoa_viet_nam_hoc': 21} positive_ess=462.515916 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=hgb_language_prior rows=1975 failures=59 prevalence=0.029873 ap=0.193580 auc=0.855733 f2_at20=0.340729 brier=0.027089 ece10=0.014553 rho=0.250000 donor_rows=4095 effective_n=21764.412633 seconds=0.690921 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=hgb_language_prior in_sample_bce=0.032498 local_bce=0.031754 donor_bce=0.034727 in_sample_local_ap=0.832277 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=hgb_language_prior mass={'khoa_tieng_duc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_duc': 362, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 23, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 15, 'khoa_tieng_nga': 22, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 12, 'khoa_tieng_trung_quoc': 2} positive_ess=438.864437 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=hgb_random_donor rows=1975 failures=59 prevalence=0.029873 ap=0.200284 auc=0.869962 f2_at20=0.340729 brier=0.026550 ece10=0.010927 rho=0.250000 donor_rows=4096 effective_n=21766.177839 seconds=0.705748 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=hgb_random_donor in_sample_bce=0.043747 local_bce=0.034780 donor_bce=0.070651 in_sample_local_ap=0.800089 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=hgb_random_donor mass={'khoa_tieng_duc': 0.75, np.str_('khoa_viet_nam_hoc'): 0.25} source_failures={'khoa_tieng_duc': 362, np.str_('khoa_viet_nam_hoc'): 224} positive_ess=563.203459 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=suite32_selective_common rows=1975 failures=59 prevalence=0.029873 ap=0.122942 auc=0.839142 f2_at20=0.316957 brier=0.028650 ece10=0.020076 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=0.666785 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=suite32_selective_common in_sample_bce=0.025762 local_bce=0.025762 donor_bce=nan in_sample_local_ap=0.892413 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=suite32_selective_common mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=suite32_selective_native rows=1975 failures=59 prevalence=0.029873 ap=0.134144 auc=0.852367 f2_at20=0.301109 brier=0.028198 ece10=0.018802 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=1.153027 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=suite32_selective_native in_sample_bce=0.025526 local_bce=0.025526 donor_bce=nan in_sample_local_ap=0.894120 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=suite32_selective_native mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=native_cache_nan_indicator rows=1975 failures=59 prevalence=0.029873 ap=0.134144 auc=0.852367 f2_at20=0.301109 brier=0.028198 ece10=0.018802 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=2.114149 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=native_cache_nan_indicator in_sample_bce=0.025526 local_bce=0.025526 donor_bce=nan in_sample_local_ap=0.894120 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=native_cache_nan_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=native_naive_zero_indicator rows=1975 failures=59 prevalence=0.029873 ap=0.141033 auc=0.852186 f2_at20=0.309033 brier=0.028235 ece10=0.018247 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=2.123407 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=native_naive_zero_indicator in_sample_bce=0.025685 local_bce=0.025685 donor_bce=nan in_sample_local_ap=0.890975 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=native_naive_zero_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=native_placebo_indicator rows=1975 failures=59 prevalence=0.029873 ap=0.134144 auc=0.852367 f2_at20=0.301109 brier=0.028198 ece10=0.018802 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=2.185521 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=native_placebo_indicator in_sample_bce=0.025526 local_bce=0.025526 donor_bce=nan in_sample_local_ap=0.894120 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=native_placebo_indicator mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan rows=1975 failures=59 prevalence=0.029873 ap=0.134144 auc=0.852367 f2_at20=0.301109 brier=0.028198 ece10=0.018802 rho=0.000000 donor_rows=0 effective_n=18332.000000 seconds=2.360031 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan in_sample_bce=0.025526 local_bce=0.025526 donor_bce=nan in_sample_local_ap=0.894120 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_duc model=suite32_selective_native_cache_nan mass={'khoa_tieng_duc': 1.0} source_failures={'khoa_tieng_duc': 362} positive_ess=362.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_duc model=committee_reference ap=0.160231 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_duc models=13 positive_rows=59 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_han_quoc fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_han_quoc fold=5 origin=6066 ap=0.033158 rows=2904 failures=20 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.020767 ap=0.053925 base_ap=0.033158 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.010049 ap=0.043207 base_ap=0.033158 donor_failures=98 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.002054 ap=0.031104 base_ap=0.033158 donor_failures=36 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6066 delta_ap=0.001841 ap=0.034999 base_ap=0.033158 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.035794 ap=0.068951 base_ap=0.033158 donor_failures=122 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6066 delta_ap=0.008498 ap=0.041656 base_ap=0.033158 donor_failures=65 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6066 delta_ap=0.012793 ap=0.045951 base_ap=0.033158 donor_failures=137 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6066 delta_ap=0.018804 ap=0.051962 base_ap=0.033158 donor_failures=177 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6066 delta_ap=0.003468 ap=0.036625 base_ap=0.033158 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6066 delta_ap=0.012382 ap=0.045540 base_ap=0.033158 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.004023 ap=0.037181 base_ap=0.033158 donor_failures=90 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.017330 ap=0.050488 base_ap=0.033158 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6066 delta_ap=0.048662 ap=0.081820 base_ap=0.033158 donor_failures=263 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=5 origin=6067 ap=0.336051 rows=3337 failures=29 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.014819 ap=0.350869 base_ap=0.336051 donor_failures=143 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.009782 ap=0.345832 base_ap=0.336051 donor_failures=118 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.099191 ap=0.236859 base_ap=0.336051 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6067 delta_ap=-0.071640 ap=0.264410 base_ap=0.336051 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.077985 ap=0.414035 base_ap=0.336051 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6067 delta_ap=-0.068932 ap=0.267118 base_ap=0.336051 donor_failures=61 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6067 delta_ap=0.034607 ap=0.370657 base_ap=0.336051 donor_failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6067 delta_ap=-0.043068 ap=0.292983 base_ap=0.336051 donor_failures=182 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.092563 ap=0.243488 base_ap=0.336051 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6067 delta_ap=-0.004151 ap=0.331900 base_ap=0.336051 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.031145 ap=0.304906 base_ap=0.336051 donor_failures=97 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.003580 ap=0.332471 base_ap=0.336051 donor_failures=31 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.010523 ap=0.325528 base_ap=0.336051 donor_failures=250 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=5 origin=6069 ap=0.245244 rows=3235 failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.088758 ap=0.156486 base_ap=0.245244 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.036637 ap=0.281880 base_ap=0.245244 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.128408 ap=0.116836 base_ap=0.245244 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6069 delta_ap=0.106646 ap=0.351890 base_ap=0.245244 donor_failures=79 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.023071 ap=0.222173 base_ap=0.245244 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6069 delta_ap=0.069400 ap=0.314644 base_ap=0.245244 donor_failures=80 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6069 delta_ap=0.095474 ap=0.340717 base_ap=0.245244 donor_failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6069 delta_ap=-0.025244 ap=0.220000 base_ap=0.245244 donor_failures=182 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6069 delta_ap=0.073413 ap=0.318656 base_ap=0.245244 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6069 delta_ap=-0.131247 ap=0.113997 base_ap=0.245244 donor_failures=46 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.002157 ap=0.243087 base_ap=0.245244 donor_failures=108 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.049079 ap=0.294322 base_ap=0.245244 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.036633 ap=0.208610 base_ap=0.245244 donor_failures=257 [SCREEN_BASE] target=khoa_tieng_han_quoc fold=5 origin=6070 ap=0.167472 rows=3351 failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.006189 ap=0.161283 base_ap=0.167472 donor_failures=147 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=0.040962 ap=0.208435 base_ap=0.167472 donor_failures=111 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_quoc_te_hoc origin=6070 delta_ap=0.023984 ap=0.191456 base_ap=0.167472 donor_failures=36 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_anh origin=6070 delta_ap=0.124936 ap=0.292408 base_ap=0.167472 donor_failures=74 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.145595 ap=0.313067 base_ap=0.167472 donor_failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_duc origin=6070 delta_ap=0.140865 ap=0.308338 base_ap=0.167472 donor_failures=76 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_italia origin=6070 delta_ap=0.081703 ap=0.249175 base_ap=0.167472 donor_failures=143 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nga origin=6070 delta_ap=0.065669 ap=0.233141 base_ap=0.167472 donor_failures=212 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_nhat origin=6070 delta_ap=0.054317 ap=0.221790 base_ap=0.167472 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_phap origin=6070 delta_ap=0.116051 ap=0.283523 base_ap=0.167472 donor_failures=61 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=0.124870 ap=0.292342 base_ap=0.167472 donor_failures=106 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_tieng_trung_quoc origin=6070 delta_ap=0.052461 ap=0.219933 base_ap=0.167472 donor_failures=36 [TRANSFER_EDGE] fold=5 target=khoa_tieng_han_quoc source=khoa_viet_nam_hoc origin=6070 delta_ap=0.051848 ap=0.219320 base_ap=0.167472 donor_failures=245 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_han_quoc origin=6069 policy_sources=khoa_tieng_bo_dao_nha,khoa_tieng_italia,khoa_viet_nam_hoc policy_donor_count=3 base_ap=0.245244 policy_ap=0.188480 delta_ap=-0.056764 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_han_quoc origin=6070 policy_sources=khoa_tieng_italia,khoa_tieng_bo_dao_nha,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.167472 policy_ap=0.262616 delta_ap=0.095144 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_han_quoc selected=[] reason=prequential_policy_exceeds_harm_tolerance replay_mean=0.019190 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_local_common rows=3573 failures=67 prevalence=0.018752 ap=0.119312 auc=0.861498 f2_at20=0.259410 brier=0.017813 ece10=0.011128 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=0.668177 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=hgb_local_common in_sample_bce=0.008859 local_bce=0.008859 donor_bce=nan in_sample_local_ap=0.910382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_local_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_local_native rows=3573 failures=67 prevalence=0.018752 ap=0.131508 auc=0.863792 f2_at20=0.244151 brier=0.017762 ece10=0.011835 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=1.257424 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=hgb_local_native in_sample_bce=0.008701 local_bce=0.008701 donor_bce=nan in_sample_local_ap=0.913296 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_local_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_pool_all rows=3573 failures=67 prevalence=0.018752 ap=0.204471 auc=0.875457 f2_at20=0.279756 brier=0.017039 ece10=0.010223 rho=0.250000 donor_rows=4095 effective_n=23836.300710 seconds=0.690802 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=hgb_pool_all in_sample_bce=0.017651 local_bce=0.014528 donor_bce=0.047280 in_sample_local_ap=0.717676 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_pool_all mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.025838942542191004, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03823439561737237, 'khoa_quoc_te_hoc': 0.012684256734901344, 'khoa_tieng_anh': 0.05032050788369812, 'khoa_tieng_bo_dao_nha': 0.004814677900487244, 'khoa_tieng_duc': 0.011765219311081333, 'khoa_tieng_italia': 0.012052739398980331, 'khoa_tieng_nga': 0.008611483346939194, 'khoa_tieng_nhat': 0.018824864326458524, 'khoa_tieng_phap': 0.016072244556371906, 'khoa_tieng_tay_ban_nha': 0.008594155127355996, 'khoa_tieng_trung_quoc': 0.037138225282257444, 'khoa_viet_nam_hoc': 0.00504828797190518} source_failures={'khoa_tieng_han_quoc': 204, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 24, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 8, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 21} positive_ess=197.042098 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_pool_equal rows=3573 failures=67 prevalence=0.018752 ap=0.182622 auc=0.880048 f2_at20=0.269583 brier=0.017296 ece10=0.010285 rho=0.250000 donor_rows=4095 effective_n=29362.543124 seconds=0.682294 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=hgb_pool_equal in_sample_bce=0.019967 local_bce=0.015068 donor_bce=0.034663 in_sample_local_ap=0.711296 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_pool_equal mass={'khoa_tieng_han_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_han_quoc': 204, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 1, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 3, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 24, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 8, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 21} positive_ess=272.488236 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_language_prior rows=3573 failures=67 prevalence=0.018752 ap=0.135508 auc=0.885303 f2_at20=0.269583 brier=0.017880 ece10=0.011743 rho=0.250000 donor_rows=4095 effective_n=29362.543124 seconds=0.698244 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=hgb_language_prior in_sample_bce=0.017490 local_bce=0.014180 donor_bce=0.027422 in_sample_local_ap=0.725037 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_language_prior mass={'khoa_tieng_han_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_han_quoc': 204, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 19, 'khoa_tieng_duc': 9, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 31, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 12, 'khoa_tieng_trung_quoc': 6} positive_ess=253.840481 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_random_donor rows=3573 failures=67 prevalence=0.018752 ap=0.197276 auc=0.880821 f2_at20=0.274669 brier=0.016868 ece10=0.009319 rho=0.250000 donor_rows=4096 effective_n=29365.756052 seconds=0.716215 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=hgb_random_donor in_sample_bce=0.016598 local_bce=0.013295 donor_bce=0.026506 in_sample_local_ap=0.761154 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=hgb_random_donor mass={'khoa_tieng_han_quoc': 0.75, np.str_('khoa_tieng_tay_ban_nha'): 0.25} source_failures={'khoa_tieng_han_quoc': 204, np.str_('khoa_tieng_tay_ban_nha'): 112} positive_ess=261.774395 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_common rows=3573 failures=67 prevalence=0.018752 ap=0.119312 auc=0.861498 f2_at20=0.259410 brier=0.017813 ece10=0.011128 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=0.712180 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_common in_sample_bce=0.008859 local_bce=0.008859 donor_bce=nan in_sample_local_ap=0.910382 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_common mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_native rows=3573 failures=67 prevalence=0.018752 ap=0.131508 auc=0.863792 f2_at20=0.244151 brier=0.017762 ece10=0.011835 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=1.247726 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_native in_sample_bce=0.008701 local_bce=0.008701 donor_bce=nan in_sample_local_ap=0.913296 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_native mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator rows=3573 failures=67 prevalence=0.018752 ap=0.131508 auc=0.863792 f2_at20=0.244151 brier=0.017762 ece10=0.011835 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=2.613320 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator in_sample_bce=0.008701 local_bce=0.008701 donor_bce=nan in_sample_local_ap=0.913296 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=native_cache_nan_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator rows=3573 failures=67 prevalence=0.018752 ap=0.127880 auc=0.863703 f2_at20=0.239064 brier=0.017818 ece10=0.011843 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=2.445120 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator in_sample_bce=0.008712 local_bce=0.008712 donor_bce=nan in_sample_local_ap=0.914092 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=native_naive_zero_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=native_placebo_indicator rows=3573 failures=67 prevalence=0.018752 ap=0.131508 auc=0.863792 f2_at20=0.244151 brier=0.017762 ece10=0.011835 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=2.375251 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=native_placebo_indicator in_sample_bce=0.008701 local_bce=0.008701 donor_bce=nan in_sample_local_ap=0.913296 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=native_placebo_indicator mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan rows=3573 failures=67 prevalence=0.018752 ap=0.131508 auc=0.863792 f2_at20=0.244151 brier=0.017762 ece10=0.011835 rho=0.000000 donor_rows=0 effective_n=29929.000000 seconds=2.378291 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.008701 local_bce=0.008701 donor_bce=nan in_sample_local_ap=0.913296 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_han_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_han_quoc': 1.0} source_failures={'khoa_tieng_han_quoc': 204} positive_ess=204.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_han_quoc model=committee_reference ap=0.191086 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_han_quoc models=13 positive_rows=67 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_italia fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_italia fold=5 origin=6066 ap=0.217946 rows=2166 failures=111 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.064989 ap=0.282935 base_ap=0.217946 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.028344 ap=0.246290 base_ap=0.217946 donor_failures=101 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6066 delta_ap=0.031805 ap=0.249751 base_ap=0.217946 donor_failures=34 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_anh origin=6066 delta_ap=0.000427 ap=0.218373 base_ap=0.217946 donor_failures=73 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.048521 ap=0.266466 base_ap=0.217946 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_duc origin=6066 delta_ap=-0.004358 ap=0.213587 base_ap=0.217946 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6066 delta_ap=0.032405 ap=0.250351 base_ap=0.217946 donor_failures=20 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nga origin=6066 delta_ap=0.010226 ap=0.228172 base_ap=0.217946 donor_failures=165 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6066 delta_ap=-0.002194 ap=0.215751 base_ap=0.217946 donor_failures=44 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_phap origin=6066 delta_ap=0.075836 ap=0.293782 base_ap=0.217946 donor_failures=37 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.002192 ap=0.220137 base_ap=0.217946 donor_failures=94 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.085493 ap=0.303439 base_ap=0.217946 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6066 delta_ap=0.011477 ap=0.229422 base_ap=0.217946 donor_failures=256 [SCREEN_BASE] target=khoa_tieng_italia fold=5 origin=6067 ap=0.511519 rows=2024 failures=71 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.008201 ap=0.503319 base_ap=0.511519 donor_failures=137 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.068725 ap=0.580245 base_ap=0.511519 donor_failures=102 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.055880 ap=0.455639 base_ap=0.511519 donor_failures=44 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_anh origin=6067 delta_ap=-0.025257 ap=0.486262 base_ap=0.511519 donor_failures=76 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.009167 ap=0.520687 base_ap=0.511519 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_duc origin=6067 delta_ap=-0.000616 ap=0.510903 base_ap=0.511519 donor_failures=73 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6067 delta_ap=0.022126 ap=0.533646 base_ap=0.511519 donor_failures=12 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nga origin=6067 delta_ap=-0.034169 ap=0.477350 base_ap=0.511519 donor_failures=190 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6067 delta_ap=-0.023043 ap=0.488476 base_ap=0.511519 donor_failures=37 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_phap origin=6067 delta_ap=-0.019597 ap=0.491923 base_ap=0.511519 donor_failures=46 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.008160 ap=0.503359 base_ap=0.511519 donor_failures=100 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.023075 ap=0.534594 base_ap=0.511519 donor_failures=25 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6067 delta_ap=0.019949 ap=0.531469 base_ap=0.511519 donor_failures=240 [SCREEN_BASE] target=khoa_tieng_italia fold=5 origin=6069 ap=0.246886 rows=2145 failures=68 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.016866 ap=0.263752 base_ap=0.246886 donor_failures=117 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.027625 ap=0.274511 base_ap=0.246886 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6069 delta_ap=0.011442 ap=0.258328 base_ap=0.246886 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_anh origin=6069 delta_ap=-0.017587 ap=0.229299 base_ap=0.246886 donor_failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.011338 ap=0.235548 base_ap=0.246886 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_duc origin=6069 delta_ap=-0.033091 ap=0.213795 base_ap=0.246886 donor_failures=70 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.016363 ap=0.230523 base_ap=0.246886 donor_failures=29 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nga origin=6069 delta_ap=0.008796 ap=0.255682 base_ap=0.246886 donor_failures=207 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6069 delta_ap=0.024532 ap=0.271418 base_ap=0.246886 donor_failures=43 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_phap origin=6069 delta_ap=0.034712 ap=0.281598 base_ap=0.246886 donor_failures=57 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.021786 ap=0.225100 base_ap=0.246886 donor_failures=109 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.032145 ap=0.214741 base_ap=0.246886 donor_failures=24 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.018109 ap=0.228777 base_ap=0.246886 donor_failures=242 [SCREEN_BASE] target=khoa_tieng_italia fold=5 origin=6070 ap=0.394694 rows=1795 failures=34 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.027706 ap=0.366988 base_ap=0.394694 donor_failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=0.004481 ap=0.399175 base_ap=0.394694 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.013572 ap=0.381122 base_ap=0.394694 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_anh origin=6070 delta_ap=0.060800 ap=0.455495 base_ap=0.394694 donor_failures=77 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.036509 ap=0.431203 base_ap=0.394694 donor_failures=147 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_duc origin=6070 delta_ap=-0.077046 ap=0.317648 base_ap=0.394694 donor_failures=86 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_han_quoc origin=6070 delta_ap=0.061962 ap=0.456656 base_ap=0.394694 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nga origin=6070 delta_ap=-0.011040 ap=0.383654 base_ap=0.394694 donor_failures=194 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_nhat origin=6070 delta_ap=0.059637 ap=0.454331 base_ap=0.394694 donor_failures=34 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_phap origin=6070 delta_ap=0.043582 ap=0.438276 base_ap=0.394694 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=0.072626 ap=0.467320 base_ap=0.394694 donor_failures=99 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_tieng_trung_quoc origin=6070 delta_ap=0.076261 ap=0.470955 base_ap=0.394694 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_italia source=khoa_viet_nam_hoc origin=6070 delta_ap=0.006267 ap=0.400961 base_ap=0.394694 donor_failures=229 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_italia origin=6069 policy_sources=khoa_tieng_trung_quoc,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.246886 policy_ap=0.211969 delta_ap=-0.034917 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_italia origin=6070 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_phap,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.394694 policy_ap=0.377499 delta_ap=-0.017195 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_italia selected=[] reason=prequential_policy_delta_-0.026056_below_threshold replay_mean=-0.026056 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=hgb_local_common rows=2129 failures=83 prevalence=0.038985 ap=0.169917 auc=0.782317 f2_at20=0.283641 brier=0.035720 ece10=0.022236 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=0.585229 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=hgb_local_common in_sample_bce=0.044128 local_bce=0.044128 donor_bce=nan in_sample_local_ap=0.870522 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=hgb_local_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=hgb_local_native rows=2129 failures=83 prevalence=0.038985 ap=0.195668 auc=0.798355 f2_at20=0.296834 brier=0.035309 ece10=0.023237 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=1.175254 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=hgb_local_native in_sample_bce=0.043335 local_bce=0.043335 donor_bce=nan in_sample_local_ap=0.876171 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=hgb_local_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=hgb_pool_all rows=2129 failures=83 prevalence=0.038985 ap=0.207998 auc=0.789130 f2_at20=0.283641 brier=0.035003 ece10=0.024478 rho=0.250000 donor_rows=4095 effective_n=19043.729618 seconds=0.613927 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=hgb_pool_all in_sample_bce=0.044739 local_bce=0.049036 donor_bce=0.053466 in_sample_local_ap=0.838606 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=hgb_pool_all mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.025119981431890728, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.0371705346068128, 'khoa_quoc_te_hoc': 0.012331320956257627, 'khoa_tieng_anh': 0.04892035429150434, 'khoa_tieng_bo_dao_nha': 0.004680710879065205, 'khoa_tieng_duc': 0.01143785548320759, 'khoa_tieng_han_quoc': 0.018673553172426357, 'khoa_tieng_nga': 0.008371871311023318, 'khoa_tieng_nhat': 0.01830106791585452, 'khoa_tieng_phap': 0.015625038995525184, 'khoa_tieng_tay_ban_nha': 0.008355025244143183, 'khoa_tieng_trung_quoc': 0.03610486489454362, 'khoa_viet_nam_hoc': 0.004907820817745522} source_failures={'khoa_tieng_italia': 610, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 20} positive_ess=621.416346 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=hgb_pool_equal rows=2129 failures=83 prevalence=0.038985 ap=0.207132 auc=0.811831 f2_at20=0.329815 brier=0.035455 ece10=0.022517 rho=0.250000 donor_rows=4095 effective_n=22116.753842 seconds=0.595574 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=hgb_pool_equal in_sample_bce=0.048999 local_bce=0.049983 donor_bce=0.046047 in_sample_local_ap=0.836145 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=hgb_pool_equal mass={'khoa_tieng_italia': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_italia': 610, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 6, 'khoa_quoc_te_hoc': 3, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 14, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_nga': 16, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 3, 'khoa_tieng_tay_ban_nha': 9, 'khoa_tieng_trung_quoc': 1, 'khoa_viet_nam_hoc': 20} positive_ess=688.084402 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=hgb_language_prior rows=2129 failures=83 prevalence=0.038985 ap=0.183897 auc=0.809870 f2_at20=0.362797 brier=0.035547 ece10=0.020532 rho=0.250000 donor_rows=4095 effective_n=22116.753842 seconds=0.614731 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=hgb_language_prior in_sample_bce=0.047434 local_bce=0.049769 donor_bce=0.040428 in_sample_local_ap=0.843621 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=hgb_language_prior mass={'khoa_tieng_italia': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_italia': 610, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 11, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_nga': 29, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 10, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 5} positive_ess=685.531386 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=hgb_random_donor rows=2129 failures=83 prevalence=0.038985 ap=0.196760 auc=0.806025 f2_at20=0.283641 brier=0.035258 ece10=0.021310 rho=0.250000 donor_rows=4096 effective_n=22118.576666 seconds=0.588956 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=hgb_random_donor in_sample_bce=0.058550 local_bce=0.053397 donor_bce=0.074009 in_sample_local_ap=0.812583 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=hgb_random_donor mass={'khoa_tieng_italia': 0.75, np.str_('khoa_cong_nghe_thong_tin'): 0.25} source_failures={'khoa_tieng_italia': 610, np.str_('khoa_cong_nghe_thong_tin'): 158} positive_ess=740.522550 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=suite32_selective_common rows=2129 failures=83 prevalence=0.038985 ap=0.169917 auc=0.782317 f2_at20=0.283641 brier=0.035720 ece10=0.022236 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=0.592614 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=suite32_selective_common in_sample_bce=0.044128 local_bce=0.044128 donor_bce=nan in_sample_local_ap=0.870522 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=suite32_selective_common mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=suite32_selective_native rows=2129 failures=83 prevalence=0.038985 ap=0.195668 auc=0.798355 f2_at20=0.296834 brier=0.035309 ece10=0.023237 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=1.186190 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=suite32_selective_native in_sample_bce=0.043335 local_bce=0.043335 donor_bce=nan in_sample_local_ap=0.876171 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=suite32_selective_native mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=native_cache_nan_indicator rows=2129 failures=83 prevalence=0.038985 ap=0.195668 auc=0.798355 f2_at20=0.296834 brier=0.035309 ece10=0.023237 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=2.264769 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=native_cache_nan_indicator in_sample_bce=0.043335 local_bce=0.043335 donor_bce=nan in_sample_local_ap=0.876171 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=native_cache_nan_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=native_naive_zero_indicator rows=2129 failures=83 prevalence=0.038985 ap=0.175903 auc=0.793714 f2_at20=0.303430 brier=0.035535 ece10=0.025746 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=2.352834 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=native_naive_zero_indicator in_sample_bce=0.043202 local_bce=0.043202 donor_bce=nan in_sample_local_ap=0.877574 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=native_naive_zero_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=native_placebo_indicator rows=2129 failures=83 prevalence=0.038985 ap=0.195668 auc=0.798355 f2_at20=0.296834 brier=0.035309 ece10=0.023237 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=2.203418 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=native_placebo_indicator in_sample_bce=0.043335 local_bce=0.043335 donor_bce=nan in_sample_local_ap=0.876171 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=native_placebo_indicator mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan rows=2129 failures=83 prevalence=0.038985 ap=0.195668 auc=0.798355 f2_at20=0.296834 brier=0.035309 ece10=0.023237 rho=0.000000 donor_rows=0 effective_n=18780.000000 seconds=2.315211 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan in_sample_bce=0.043335 local_bce=0.043335 donor_bce=nan in_sample_local_ap=0.876171 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_italia model=suite32_selective_native_cache_nan mass={'khoa_tieng_italia': 1.0} source_failures={'khoa_tieng_italia': 610} positive_ess=610.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_italia model=committee_reference ap=0.215484 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_italia models=13 positive_rows=83 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_nga fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nga fold=5 origin=6066 ap=0.304798 rows=1588 failures=102 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.000674 ap=0.305472 base_ap=0.304798 donor_failures=116 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.028134 ap=0.332932 base_ap=0.304798 donor_failures=108 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6066 delta_ap=0.009272 ap=0.314069 base_ap=0.304798 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_anh origin=6066 delta_ap=0.056534 ap=0.361332 base_ap=0.304798 donor_failures=69 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.022130 ap=0.326927 base_ap=0.304798 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_duc origin=6066 delta_ap=0.009070 ap=0.313867 base_ap=0.304798 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.013598 ap=0.291200 base_ap=0.304798 donor_failures=18 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_italia origin=6066 delta_ap=-0.027511 ap=0.277287 base_ap=0.304798 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6066 delta_ap=-0.017919 ap=0.286879 base_ap=0.304798 donor_failures=37 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_phap origin=6066 delta_ap=0.017541 ap=0.322339 base_ap=0.304798 donor_failures=37 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.009865 ap=0.314662 base_ap=0.304798 donor_failures=89 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.004377 ap=0.309175 base_ap=0.304798 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6066 delta_ap=0.012098 ap=0.316896 base_ap=0.304798 donor_failures=262 [SCREEN_BASE] target=khoa_tieng_nga fold=5 origin=6067 ap=0.792868 rows=1567 failures=83 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.015821 ap=0.808689 base_ap=0.792868 donor_failures=110 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.008304 ap=0.784564 base_ap=0.792868 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.009053 ap=0.783815 base_ap=0.792868 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_anh origin=6067 delta_ap=0.004699 ap=0.797567 base_ap=0.792868 donor_failures=66 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.022417 ap=0.770451 base_ap=0.792868 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_duc origin=6067 delta_ap=-0.010040 ap=0.782828 base_ap=0.792868 donor_failures=60 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6067 delta_ap=0.033788 ap=0.826656 base_ap=0.792868 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_italia origin=6067 delta_ap=-0.000991 ap=0.791877 base_ap=0.792868 donor_failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6067 delta_ap=0.002678 ap=0.795546 base_ap=0.792868 donor_failures=36 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_phap origin=6067 delta_ap=0.060886 ap=0.853754 base_ap=0.792868 donor_failures=56 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.012364 ap=0.805232 base_ap=0.792868 donor_failures=91 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.000319 ap=0.793187 base_ap=0.792868 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.025913 ap=0.766955 base_ap=0.792868 donor_failures=254 [SCREEN_BASE] target=khoa_tieng_nga fold=5 origin=6069 ap=0.446899 rows=1996 failures=92 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.014460 ap=0.432439 base_ap=0.446899 donor_failures=151 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.038378 ap=0.485277 base_ap=0.446899 donor_failures=110 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6069 delta_ap=0.008258 ap=0.455157 base_ap=0.446899 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_anh origin=6069 delta_ap=0.024174 ap=0.471072 base_ap=0.446899 donor_failures=75 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.004419 ap=0.451318 base_ap=0.446899 donor_failures=143 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_duc origin=6069 delta_ap=0.018817 ap=0.465716 base_ap=0.446899 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6069 delta_ap=0.009311 ap=0.456210 base_ap=0.446899 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_italia origin=6069 delta_ap=0.005838 ap=0.452736 base_ap=0.446899 donor_failures=149 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6069 delta_ap=-0.023468 ap=0.423430 base_ap=0.446899 donor_failures=30 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_phap origin=6069 delta_ap=-0.031920 ap=0.414979 base_ap=0.446899 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.034903 ap=0.481801 base_ap=0.446899 donor_failures=102 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.026594 ap=0.473492 base_ap=0.446899 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6069 delta_ap=0.021841 ap=0.468740 base_ap=0.446899 donor_failures=255 [SCREEN_BASE] target=khoa_tieng_nga fold=5 origin=6070 ap=0.599173 rows=1659 failures=114 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.010502 ap=0.588671 base_ap=0.599173 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.004357 ap=0.594816 base_ap=0.599173 donor_failures=127 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.000944 ap=0.598229 base_ap=0.599173 donor_failures=28 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_anh origin=6070 delta_ap=-0.009939 ap=0.589234 base_ap=0.599173 donor_failures=58 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=-0.018608 ap=0.580565 base_ap=0.599173 donor_failures=146 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_duc origin=6070 delta_ap=-0.023792 ap=0.575381 base_ap=0.599173 donor_failures=82 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_han_quoc origin=6070 delta_ap=-0.007098 ap=0.592075 base_ap=0.599173 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_italia origin=6070 delta_ap=0.016711 ap=0.615884 base_ap=0.599173 donor_failures=119 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_nhat origin=6070 delta_ap=-0.023958 ap=0.575215 base_ap=0.599173 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_phap origin=6070 delta_ap=-0.009110 ap=0.590062 base_ap=0.599173 donor_failures=63 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.008121 ap=0.591052 base_ap=0.599173 donor_failures=109 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_tieng_trung_quoc origin=6070 delta_ap=0.046314 ap=0.645487 base_ap=0.599173 donor_failures=24 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nga source=khoa_viet_nam_hoc origin=6070 delta_ap=0.023401 ap=0.622574 base_ap=0.599173 donor_failures=239 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_nga origin=6069 policy_sources=khoa_tieng_phap,khoa_tieng_anh,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.446899 policy_ap=0.457184 delta_ap=0.010285 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_nga origin=6070 policy_sources=khoa_tieng_anh,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.599173 policy_ap=0.600948 delta_ap=0.001775 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_nga selected=['khoa_tieng_trung_quoc', 'khoa_tieng_anh', 'khoa_quan_tri_kinh_doanh_va_du_lich'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.006030 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=hgb_local_common rows=2047 failures=134 prevalence=0.065462 ap=0.460201 auc=0.916028 f2_at20=0.623679 brier=0.045946 ece10=0.014528 rho=0.000000 donor_rows=0 effective_n=13418.000000 seconds=0.456595 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=hgb_local_common in_sample_bce=0.059903 local_bce=0.059903 donor_bce=nan in_sample_local_ap=0.888330 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=hgb_local_common mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 661} positive_ess=661.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=hgb_local_native rows=2047 failures=134 prevalence=0.065462 ap=0.459097 auc=0.919040 f2_at20=0.613108 brier=0.045666 ece10=0.012767 rho=0.000000 donor_rows=0 effective_n=13418.000000 seconds=1.195889 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=hgb_local_native in_sample_bce=0.059549 local_bce=0.059549 donor_bce=nan in_sample_local_ap=0.888388 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=hgb_local_native mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 661} positive_ess=661.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=hgb_pool_all rows=2047 failures=134 prevalence=0.065462 ap=0.480118 auc=0.926298 f2_at20=0.623679 brier=0.044429 ece10=0.015502 rho=0.250000 donor_rows=4095 effective_n=15601.632943 seconds=0.600006 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=hgb_pool_all in_sample_bce=0.060371 local_bce=0.065740 donor_bce=0.063906 in_sample_local_ap=0.864234 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=hgb_pool_all mass={'khoa_tieng_nga': 0.75, 'khoa_cong_nghe_thong_tin': 0.024788264470544195, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03667968644178412, 'khoa_quoc_te_hoc': 0.012168482129004135, 'khoa_tieng_anh': 0.0482743461995966, 'khoa_tieng_bo_dao_nha': 0.0046189006745491305, 'khoa_tieng_duc': 0.011286815138074469, 'khoa_tieng_han_quoc': 0.018426963248277917, 'khoa_tieng_italia': 0.011562643917359727, 'khoa_tieng_nhat': 0.01805939677231073, 'khoa_tieng_phap': 0.015418705624198064, 'khoa_tieng_tay_ban_nha': 0.008244694605823435, 'khoa_tieng_trung_quoc': 0.03562808922075907, 'khoa_viet_nam_hoc': 0.004843011557718403} source_failures={'khoa_tieng_nga': 661, 'khoa_cong_nghe_thong_tin': 14, 'khoa_quan_tri_kinh_doanh_va_du_lich': 15, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 12, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 19} positive_ess=710.959518 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=hgb_pool_equal rows=2047 failures=134 prevalence=0.065462 ap=0.501233 auc=0.928874 f2_at20=0.618393 brier=0.043548 ece10=0.013968 rho=0.250000 donor_rows=4095 effective_n=17487.465638 seconds=7.855507 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=hgb_pool_equal in_sample_bce=0.063724 local_bce=0.066286 donor_bce=0.056037 in_sample_local_ap=0.862890 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=hgb_pool_equal mass={'khoa_tieng_nga': 0.7499999999999999, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nga': 661, 'khoa_cong_nghe_thong_tin': 14, 'khoa_quan_tri_kinh_doanh_va_du_lich': 15, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 4, 'khoa_tieng_bo_dao_nha': 9, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 12, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 11, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 19} positive_ess=761.273376 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=hgb_language_prior rows=2047 failures=134 prevalence=0.065462 ap=0.460673 auc=0.922689 f2_at20=0.634249 brier=0.045548 ece10=0.012494 rho=0.250000 donor_rows=4095 effective_n=17487.465638 seconds=0.562410 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=hgb_language_prior in_sample_bce=0.060903 local_bce=0.065302 donor_bce=0.047707 in_sample_local_ap=0.864052 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=hgb_language_prior mass={'khoa_tieng_nga': 0.7499999999999999, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nga': 661, 'khoa_tieng_anh': 8, 'khoa_tieng_bo_dao_nha': 26, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 16, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 6, 'khoa_tieng_tay_ban_nha': 19, 'khoa_tieng_trung_quoc': 5} positive_ess=752.328905 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=hgb_random_donor rows=2047 failures=134 prevalence=0.065462 ap=0.496132 auc=0.929709 f2_at20=0.628964 brier=0.043355 ece10=0.014392 rho=0.250000 donor_rows=4095 effective_n=17487.465638 seconds=0.522165 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=hgb_random_donor in_sample_bce=0.063598 local_bce=0.066846 donor_bce=0.053854 in_sample_local_ap=0.856827 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=hgb_random_donor mass={'khoa_tieng_nga': 0.7499999999999999, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.08333333333333333, np.str_('khoa_tieng_han_quoc'): 0.08333333333333333, np.str_('khoa_tieng_tay_ban_nha'): 0.08333333333333333} source_failures={'khoa_tieng_nga': 661, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 52, np.str_('khoa_tieng_han_quoc'): 7, np.str_('khoa_tieng_tay_ban_nha'): 49} positive_ess=768.231231 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=suite32_selective_common rows=2047 failures=134 prevalence=0.065462 ap=0.470893 auc=0.925984 f2_at20=0.623679 brier=0.045856 ece10=0.015802 rho=0.250000 donor_rows=4095 effective_n=17375.256385 seconds=0.568277 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=suite32_selective_common in_sample_bce=0.059146 local_bce=0.065065 donor_bce=0.045867 in_sample_local_ap=0.863854 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=suite32_selective_common mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_trung_quoc': 0.09375909087131243, 'khoa_tieng_anh': 0.0911787800939345, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.06506212903475307} source_failures={'khoa_tieng_nga': 661, 'khoa_tieng_trung_quoc': 12, 'khoa_tieng_anh': 21, 'khoa_quan_tri_kinh_doanh_va_du_lich': 52} positive_ess=743.449008 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=suite32_selective_native rows=2047 failures=134 prevalence=0.065462 ap=0.485429 auc=0.926091 f2_at20=0.628964 brier=0.044319 ece10=0.010676 rho=0.250000 donor_rows=4095 effective_n=17375.256385 seconds=1.313345 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=suite32_selective_native in_sample_bce=0.058684 local_bce=0.064351 donor_bce=0.046178 in_sample_local_ap=0.870624 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=suite32_selective_native mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_trung_quoc': 0.09375909087131243, 'khoa_tieng_anh': 0.0911787800939345, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.06506212903475307} source_failures={'khoa_tieng_nga': 661, 'khoa_tieng_trung_quoc': 12, 'khoa_tieng_anh': 21, 'khoa_quan_tri_kinh_doanh_va_du_lich': 52} positive_ess=743.449008 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=native_cache_nan_indicator rows=2047 failures=134 prevalence=0.065462 ap=0.459097 auc=0.919040 f2_at20=0.613108 brier=0.045666 ece10=0.012767 rho=0.000000 donor_rows=0 effective_n=13418.000000 seconds=2.135121 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=native_cache_nan_indicator in_sample_bce=0.059549 local_bce=0.059549 donor_bce=nan in_sample_local_ap=0.888388 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=native_cache_nan_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 661} positive_ess=661.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=native_naive_zero_indicator rows=2047 failures=134 prevalence=0.065462 ap=0.459097 auc=0.919040 f2_at20=0.613108 brier=0.045666 ece10=0.012767 rho=0.000000 donor_rows=0 effective_n=13418.000000 seconds=2.105937 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=native_naive_zero_indicator in_sample_bce=0.059549 local_bce=0.059549 donor_bce=nan in_sample_local_ap=0.888388 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=native_naive_zero_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 661} positive_ess=661.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=native_placebo_indicator rows=2047 failures=134 prevalence=0.065462 ap=0.459097 auc=0.919040 f2_at20=0.613108 brier=0.045666 ece10=0.012767 rho=0.000000 donor_rows=0 effective_n=13418.000000 seconds=2.148861 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=native_placebo_indicator in_sample_bce=0.059549 local_bce=0.059549 donor_bce=nan in_sample_local_ap=0.888388 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=native_placebo_indicator mass={'khoa_tieng_nga': 1.0} source_failures={'khoa_tieng_nga': 661} positive_ess=661.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan rows=2047 failures=134 prevalence=0.065462 ap=0.485429 auc=0.926091 f2_at20=0.628964 brier=0.044319 ece10=0.010676 rho=0.250000 donor_rows=4095 effective_n=17375.256385 seconds=2.329704 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan in_sample_bce=0.058684 local_bce=0.064351 donor_bce=0.046178 in_sample_local_ap=0.870624 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nga model=suite32_selective_native_cache_nan mass={'khoa_tieng_nga': 0.75, 'khoa_tieng_trung_quoc': 0.09375909087131243, 'khoa_tieng_anh': 0.0911787800939345, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.06506212903475307} source_failures={'khoa_tieng_nga': 661, 'khoa_tieng_trung_quoc': 12, 'khoa_tieng_anh': 21, 'khoa_quan_tri_kinh_doanh_va_du_lich': 52} positive_ess=743.449008 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nga model=committee_reference ap=0.465978 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_nga models=13 positive_rows=134 num_edges=52 num_replays=2 chosen=['khoa_tieng_trung_quoc', 'khoa_tieng_anh', 'khoa_quan_tri_kinh_doanh_va_du_lich'] [SELECTION_ORIGINS] target=khoa_tieng_nhat fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_nhat fold=5 origin=6066 ap=0.408537 rows=2840 failures=17 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.118392 ap=0.526929 base_ap=0.408537 donor_failures=109 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.132843 ap=0.541381 base_ap=0.408537 donor_failures=104 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.293699 ap=0.114838 base_ap=0.408537 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6066 delta_ap=-0.281828 ap=0.126709 base_ap=0.408537 donor_failures=70 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.224813 ap=0.183724 base_ap=0.408537 donor_failures=122 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6066 delta_ap=-0.226564 ap=0.181974 base_ap=0.408537 donor_failures=72 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6066 delta_ap=0.098801 ap=0.507338 base_ap=0.408537 donor_failures=11 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6066 delta_ap=-0.104096 ap=0.304441 base_ap=0.408537 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6066 delta_ap=-0.321571 ap=0.086966 base_ap=0.408537 donor_failures=161 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6066 delta_ap=0.056137 ap=0.464675 base_ap=0.408537 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.014584 ap=0.393953 base_ap=0.408537 donor_failures=83 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.135907 ap=0.544444 base_ap=0.408537 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6066 delta_ap=0.117930 ap=0.526467 base_ap=0.408537 donor_failures=270 [SCREEN_BASE] target=khoa_tieng_nhat fold=5 origin=6067 ap=0.360864 rows=2448 failures=11 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.127308 ap=0.488172 base_ap=0.360864 donor_failures=138 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.127795 ap=0.488660 base_ap=0.360864 donor_failures=116 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6067 delta_ap=0.172942 ap=0.533807 base_ap=0.360864 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6067 delta_ap=-0.029797 ap=0.331067 base_ap=0.360864 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.061360 ap=0.422224 base_ap=0.360864 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6067 delta_ap=-0.086275 ap=0.274589 base_ap=0.360864 donor_failures=68 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6067 delta_ap=0.063484 ap=0.424349 base_ap=0.360864 donor_failures=16 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6067 delta_ap=0.090203 ap=0.451067 base_ap=0.360864 donor_failures=144 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6067 delta_ap=0.226299 ap=0.587163 base_ap=0.360864 donor_failures=192 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6067 delta_ap=-0.029417 ap=0.331447 base_ap=0.360864 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.131773 ap=0.492637 base_ap=0.360864 donor_failures=107 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.042265 ap=0.318600 base_ap=0.360864 donor_failures=17 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6067 delta_ap=0.002544 ap=0.363408 base_ap=0.360864 donor_failures=253 [SCREEN_BASE] target=khoa_tieng_nhat fold=5 origin=6069 ap=0.213171 rows=2588 failures=17 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.191963 ap=0.405134 base_ap=0.213171 donor_failures=122 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.091184 ap=0.304355 base_ap=0.213171 donor_failures=115 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.009952 ap=0.203219 base_ap=0.213171 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6069 delta_ap=-0.060030 ap=0.153141 base_ap=0.213171 donor_failures=58 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.006680 ap=0.219851 base_ap=0.213171 donor_failures=117 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6069 delta_ap=-0.004685 ap=0.208486 base_ap=0.213171 donor_failures=85 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.095381 ap=0.117790 base_ap=0.213171 donor_failures=23 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6069 delta_ap=0.003735 ap=0.216906 base_ap=0.213171 donor_failures=152 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6069 delta_ap=-0.018172 ap=0.194999 base_ap=0.213171 donor_failures=199 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6069 delta_ap=-0.052040 ap=0.161131 base_ap=0.213171 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.063803 ap=0.276975 base_ap=0.213171 donor_failures=104 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.050698 ap=0.162473 base_ap=0.213171 donor_failures=30 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6069 delta_ap=-0.012501 ap=0.200670 base_ap=0.213171 donor_failures=235 [SCREEN_BASE] target=khoa_tieng_nhat fold=5 origin=6070 ap=0.210716 rows=2413 failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=0.035507 ap=0.246223 base_ap=0.210716 donor_failures=119 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.002549 ap=0.208168 base_ap=0.210716 donor_failures=139 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.058027 ap=0.152690 base_ap=0.210716 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_anh origin=6070 delta_ap=-0.034857 ap=0.175860 base_ap=0.210716 donor_failures=73 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.040014 ap=0.250730 base_ap=0.210716 donor_failures=151 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_duc origin=6070 delta_ap=-0.051089 ap=0.159628 base_ap=0.210716 donor_failures=76 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_han_quoc origin=6070 delta_ap=0.027137 ap=0.237853 base_ap=0.210716 donor_failures=28 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_italia origin=6070 delta_ap=0.034003 ap=0.244719 base_ap=0.210716 donor_failures=139 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_nga origin=6070 delta_ap=-0.003238 ap=0.207479 base_ap=0.210716 donor_failures=184 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_phap origin=6070 delta_ap=0.027734 ap=0.238450 base_ap=0.210716 donor_failures=59 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=0.020693 ap=0.231410 base_ap=0.210716 donor_failures=113 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.028961 ap=0.181755 base_ap=0.210716 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_nhat source=khoa_viet_nam_hoc origin=6070 delta_ap=0.013691 ap=0.224408 base_ap=0.210716 donor_failures=258 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_nhat origin=6069 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_cong_nghe_thong_tin,khoa_tieng_han_quoc policy_donor_count=3 base_ap=0.213171 policy_ap=0.259800 delta_ap=0.046629 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_nhat origin=6070 policy_sources=khoa_cong_nghe_thong_tin,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_tay_ban_nha policy_donor_count=3 base_ap=0.210716 policy_ap=0.234587 delta_ap=0.023871 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_nhat selected=['khoa_cong_nghe_thong_tin', 'khoa_quan_tri_kinh_doanh_va_du_lich', 'khoa_tieng_tay_ban_nha'] reason=admitted_by_older-origin_prequential_replay replay_mean=0.035250 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=hgb_local_common rows=2593 failures=69 prevalence=0.026610 ap=0.229278 auc=0.839876 f2_at20=0.327044 brier=0.024394 ece10=0.020693 rho=0.000000 donor_rows=0 effective_n=29332.000000 seconds=0.694366 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=hgb_local_common in_sample_bce=0.010235 local_bce=0.010235 donor_bce=nan in_sample_local_ap=0.914750 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=hgb_local_common mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 282} positive_ess=282.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=hgb_local_native rows=2593 failures=69 prevalence=0.026610 ap=0.231716 auc=0.846195 f2_at20=0.314465 brier=0.024393 ece10=0.018293 rho=0.000000 donor_rows=0 effective_n=29332.000000 seconds=1.316025 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=hgb_local_native in_sample_bce=0.009811 local_bce=0.009811 donor_bce=nan in_sample_local_ap=0.925248 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=hgb_local_native mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 282} positive_ess=282.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=hgb_pool_all rows=2593 failures=69 prevalence=0.026610 ap=0.286247 auc=0.853083 f2_at20=0.345912 brier=0.022356 ece10=0.017208 rho=0.250000 donor_rows=4095 effective_n=23634.714524 seconds=0.727156 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=hgb_pool_all in_sample_bce=0.015143 local_bce=0.013596 donor_bce=0.034728 in_sample_local_ap=0.849400 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=hgb_pool_all mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.025799402770835738, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.038175887833698595, 'khoa_quoc_te_hoc': 0.012664846783805606, 'khoa_tieng_anh': 0.05024350545324054, 'khoa_tieng_bo_dao_nha': 0.004807310290027811, 'khoa_tieng_duc': 0.011747215707383341, 'khoa_tieng_han_quoc': 0.01917861765799018, 'khoa_tieng_italia': 0.012034295820677457, 'khoa_tieng_nga': 0.008598305714688505, 'khoa_tieng_phap': 0.016047650172376228, 'khoa_tieng_tay_ban_nha': 0.00858100401143194, 'khoa_tieng_trung_quoc': 0.03708139490176477, 'khoa_viet_nam_hoc': 0.0050405628820792804} source_failures={'khoa_tieng_nhat': 282, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 7, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 12, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 16} positive_ess=254.192667 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=hgb_pool_equal rows=2593 failures=69 prevalence=0.026610 ap=0.256215 auc=0.842833 f2_at20=0.327044 brier=0.022719 ece10=0.014878 rho=0.250000 donor_rows=4095 effective_n=29036.406545 seconds=0.690871 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=hgb_pool_equal in_sample_bce=0.017325 local_bce=0.014438 donor_bce=0.025984 in_sample_local_ap=0.829464 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=hgb_pool_equal mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_nhat': 282, 'khoa_cong_nghe_thong_tin': 11, 'khoa_quan_tri_kinh_doanh_va_du_lich': 12, 'khoa_quoc_te_hoc': 7, 'khoa_tieng_anh': 3, 'khoa_tieng_bo_dao_nha': 13, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 12, 'khoa_tieng_phap': 2, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 16} positive_ess=322.253678 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=hgb_language_prior rows=2593 failures=69 prevalence=0.026610 ap=0.212631 auc=0.856456 f2_at20=0.327044 brier=0.023516 ece10=0.014532 rho=0.250000 donor_rows=4095 effective_n=29036.406545 seconds=0.716961 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=hgb_language_prior in_sample_bce=0.016619 local_bce=0.014116 donor_bce=0.024128 in_sample_local_ap=0.848093 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=hgb_language_prior mass={'khoa_tieng_nhat': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_nhat': 282, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 21, 'khoa_tieng_duc': 8, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 11, 'khoa_tieng_nga': 20, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 7, 'khoa_tieng_trung_quoc': 3} positive_ess=309.025856 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=hgb_random_donor rows=2593 failures=69 prevalence=0.026610 ap=0.246718 auc=0.866180 f2_at20=0.352201 brier=0.023174 ece10=0.014384 rho=0.250000 donor_rows=4095 effective_n=29036.406545 seconds=0.708983 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=hgb_random_donor in_sample_bce=0.019688 local_bce=0.014780 donor_bce=0.034410 in_sample_local_ap=0.838198 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=hgb_random_donor mass={'khoa_tieng_nhat': 0.75, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 0.08333333333333333, np.str_('khoa_viet_nam_hoc'): 0.08333333333333333, np.str_('khoa_tieng_han_quoc'): 0.08333333333333333} source_failures={'khoa_tieng_nhat': 282, np.str_('khoa_quan_tri_kinh_doanh_va_du_lich'): 36, np.str_('khoa_viet_nam_hoc'): 86, np.str_('khoa_tieng_han_quoc'): 14} positive_ess=348.155244 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_common rows=2593 failures=69 prevalence=0.026610 ap=0.227662 auc=0.874067 f2_at20=0.339623 brier=0.023766 ece10=0.015528 rho=0.250000 donor_rows=4095 effective_n=27737.414994 seconds=0.701852 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_common in_sample_bce=0.019002 local_bce=0.014994 donor_bce=0.033477 in_sample_local_ap=0.830842 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_common mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.11550538045999624, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.08526115226127935, 'khoa_tieng_tay_ban_nha': 0.04923346727872443} source_failures={'khoa_tieng_nhat': 282, 'khoa_cong_nghe_thong_tin': 49, 'khoa_quan_tri_kinh_doanh_va_du_lich': 36, 'khoa_tieng_tay_ban_nha': 37} positive_ess=308.372028 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_native rows=2593 failures=69 prevalence=0.026610 ap=0.253216 auc=0.868859 f2_at20=0.358491 brier=0.023093 ece10=0.013072 rho=0.250000 donor_rows=4095 effective_n=27737.414994 seconds=1.401242 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_native in_sample_bce=0.018939 local_bce=0.014616 donor_bce=0.034718 in_sample_local_ap=0.836813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_native mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.11550538045999624, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.08526115226127935, 'khoa_tieng_tay_ban_nha': 0.04923346727872443} source_failures={'khoa_tieng_nhat': 282, 'khoa_cong_nghe_thong_tin': 49, 'khoa_quan_tri_kinh_doanh_va_du_lich': 36, 'khoa_tieng_tay_ban_nha': 37} positive_ess=308.372028 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=native_cache_nan_indicator rows=2593 failures=69 prevalence=0.026610 ap=0.231716 auc=0.846195 f2_at20=0.314465 brier=0.024393 ece10=0.018293 rho=0.000000 donor_rows=0 effective_n=29332.000000 seconds=2.502597 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=native_cache_nan_indicator in_sample_bce=0.009811 local_bce=0.009811 donor_bce=nan in_sample_local_ap=0.925248 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=native_cache_nan_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 282} positive_ess=282.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=native_naive_zero_indicator rows=2593 failures=69 prevalence=0.026610 ap=0.231716 auc=0.846195 f2_at20=0.314465 brier=0.024393 ece10=0.018293 rho=0.000000 donor_rows=0 effective_n=29332.000000 seconds=2.550842 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=native_naive_zero_indicator in_sample_bce=0.009811 local_bce=0.009811 donor_bce=nan in_sample_local_ap=0.925248 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=native_naive_zero_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 282} positive_ess=282.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=native_placebo_indicator rows=2593 failures=69 prevalence=0.026610 ap=0.231716 auc=0.846195 f2_at20=0.314465 brier=0.024393 ece10=0.018293 rho=0.000000 donor_rows=0 effective_n=29332.000000 seconds=2.383468 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=native_placebo_indicator in_sample_bce=0.009811 local_bce=0.009811 donor_bce=nan in_sample_local_ap=0.925248 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=native_placebo_indicator mass={'khoa_tieng_nhat': 1.0} source_failures={'khoa_tieng_nhat': 282} positive_ess=282.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan rows=2593 failures=69 prevalence=0.026610 ap=0.253216 auc=0.868859 f2_at20=0.358491 brier=0.023093 ece10=0.013072 rho=0.250000 donor_rows=4095 effective_n=27737.414994 seconds=2.831366 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan in_sample_bce=0.018939 local_bce=0.014616 donor_bce=0.034718 in_sample_local_ap=0.836813 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_nhat model=suite32_selective_native_cache_nan mass={'khoa_tieng_nhat': 0.75, 'khoa_cong_nghe_thong_tin': 0.11550538045999624, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.08526115226127935, 'khoa_tieng_tay_ban_nha': 0.04923346727872443} source_failures={'khoa_tieng_nhat': 282, 'khoa_cong_nghe_thong_tin': 49, 'khoa_quan_tri_kinh_doanh_va_du_lich': 36, 'khoa_tieng_tay_ban_nha': 37} positive_ess=308.372028 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_nhat model=committee_reference ap=0.226421 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_nhat models=13 positive_rows=69 num_edges=52 num_replays=2 chosen=['khoa_cong_nghe_thong_tin', 'khoa_quan_tri_kinh_doanh_va_du_lich', 'khoa_tieng_tay_ban_nha'] [SELECTION_ORIGINS] target=khoa_tieng_phap fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_phap fold=5 origin=6066 ap=0.247265 rows=2606 failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.012879 ap=0.260144 base_ap=0.247265 donor_failures=136 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=0.011050 ap=0.258315 base_ap=0.247265 donor_failures=109 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6066 delta_ap=0.052576 ap=0.299841 base_ap=0.247265 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_anh origin=6066 delta_ap=-0.012413 ap=0.234851 base_ap=0.247265 donor_failures=78 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=0.001476 ap=0.248741 base_ap=0.247265 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_duc origin=6066 delta_ap=-0.006278 ap=0.240987 base_ap=0.247265 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6066 delta_ap=0.043009 ap=0.290274 base_ap=0.247265 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_italia origin=6066 delta_ap=0.023410 ap=0.270675 base_ap=0.247265 donor_failures=117 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nga origin=6066 delta_ap=0.048422 ap=0.295687 base_ap=0.247265 donor_failures=175 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6066 delta_ap=-0.030251 ap=0.217014 base_ap=0.247265 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=0.004803 ap=0.252068 base_ap=0.247265 donor_failures=97 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6066 delta_ap=0.052322 ap=0.299587 base_ap=0.247265 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6066 delta_ap=0.043997 ap=0.291262 base_ap=0.247265 donor_failures=250 [SCREEN_BASE] target=khoa_tieng_phap fold=5 origin=6067 ap=0.486940 rows=2204 failures=40 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.000829 ap=0.486111 base_ap=0.486940 donor_failures=118 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.046794 ap=0.533734 base_ap=0.486940 donor_failures=125 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6067 delta_ap=0.123666 ap=0.610606 base_ap=0.486940 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_anh origin=6067 delta_ap=0.016590 ap=0.503529 base_ap=0.486940 donor_failures=68 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.039823 ap=0.447117 base_ap=0.486940 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_duc origin=6067 delta_ap=-0.001330 ap=0.485609 base_ap=0.486940 donor_failures=74 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.000166 ap=0.486774 base_ap=0.486940 donor_failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_italia origin=6067 delta_ap=0.068843 ap=0.555783 base_ap=0.486940 donor_failures=125 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nga origin=6067 delta_ap=-0.081771 ap=0.405168 base_ap=0.486940 donor_failures=194 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6067 delta_ap=0.040522 ap=0.527462 base_ap=0.486940 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.083171 ap=0.570111 base_ap=0.486940 donor_failures=97 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6067 delta_ap=0.049734 ap=0.536674 base_ap=0.486940 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.003629 ap=0.483311 base_ap=0.486940 donor_failures=238 [SCREEN_BASE] target=khoa_tieng_phap fold=5 origin=6069 ap=0.153702 rows=2693 failures=43 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.042949 ap=0.110754 base_ap=0.153702 donor_failures=158 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.009444 ap=0.163146 base_ap=0.153702 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6069 delta_ap=-0.057203 ap=0.096499 base_ap=0.153702 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_anh origin=6069 delta_ap=-0.065101 ap=0.088601 base_ap=0.153702 donor_failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.022090 ap=0.131612 base_ap=0.153702 donor_failures=121 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_duc origin=6069 delta_ap=0.029479 ap=0.183181 base_ap=0.153702 donor_failures=74 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.061673 ap=0.092029 base_ap=0.153702 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_italia origin=6069 delta_ap=-0.047148 ap=0.106554 base_ap=0.153702 donor_failures=151 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nga origin=6069 delta_ap=0.043554 ap=0.197256 base_ap=0.153702 donor_failures=188 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6069 delta_ap=-0.058594 ap=0.095109 base_ap=0.153702 donor_failures=33 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.015870 ap=0.137833 base_ap=0.153702 donor_failures=98 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6069 delta_ap=-0.022591 ap=0.131111 base_ap=0.153702 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6069 delta_ap=0.053660 ap=0.207363 base_ap=0.153702 donor_failures=242 [SCREEN_BASE] target=khoa_tieng_phap fold=5 origin=6070 ap=0.297011 rows=2275 failures=19 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.075403 ap=0.221608 base_ap=0.297011 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.077264 ap=0.219747 base_ap=0.297011 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.126734 ap=0.170277 base_ap=0.297011 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_anh origin=6070 delta_ap=-0.044358 ap=0.252653 base_ap=0.297011 donor_failures=44 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=-0.061908 ap=0.235103 base_ap=0.297011 donor_failures=147 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_duc origin=6070 delta_ap=-0.127841 ap=0.169171 base_ap=0.297011 donor_failures=70 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_han_quoc origin=6070 delta_ap=0.022890 ap=0.319901 base_ap=0.297011 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_italia origin=6070 delta_ap=-0.101897 ap=0.195114 base_ap=0.297011 donor_failures=141 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nga origin=6070 delta_ap=-0.135816 ap=0.161195 base_ap=0.297011 donor_failures=195 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_nhat origin=6070 delta_ap=-0.138271 ap=0.158741 base_ap=0.297011 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.090462 ap=0.206549 base_ap=0.297011 donor_failures=105 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.006096 ap=0.290915 base_ap=0.297011 donor_failures=24 [TRANSFER_EDGE] fold=5 target=khoa_tieng_phap source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.035119 ap=0.261892 base_ap=0.297011 donor_failures=229 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_phap origin=6069 policy_sources=khoa_quoc_te_hoc,khoa_tieng_trung_quoc,khoa_tieng_italia policy_donor_count=3 base_ap=0.153702 policy_ap=0.107387 delta_ap=-0.046315 policy_donor_rows=4095 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_phap origin=6070 policy_sources=khoa_quoc_te_hoc,khoa_viet_nam_hoc,khoa_tieng_trung_quoc policy_donor_count=3 base_ap=0.297011 policy_ap=0.221382 delta_ap=-0.075629 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_phap selected=[] reason=prequential_policy_delta_-0.060972_below_threshold replay_mean=-0.060972 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=hgb_local_common rows=2983 failures=57 prevalence=0.019108 ap=0.228799 auc=0.889454 f2_at20=0.260606 brier=0.016625 ece10=0.009786 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=0.639717 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=hgb_local_common in_sample_bce=0.015205 local_bce=0.015205 donor_bce=nan in_sample_local_ap=0.906442 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=hgb_local_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=hgb_local_native rows=2983 failures=57 prevalence=0.019108 ap=0.256756 auc=0.890390 f2_at20=0.254545 brier=0.016211 ece10=0.008501 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=1.687482 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=hgb_local_native in_sample_bce=0.015213 local_bce=0.015213 donor_bce=nan in_sample_local_ap=0.907489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=hgb_local_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=hgb_pool_all rows=2983 failures=57 prevalence=0.019108 ap=0.281865 auc=0.910791 f2_at20=0.272727 brier=0.016217 ece10=0.010133 rho=0.250000 donor_rows=4095 effective_n=22030.283097 seconds=0.674320 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=hgb_pool_all in_sample_bce=0.021519 local_bce=0.020447 donor_bce=0.042440 in_sample_local_ap=0.835635 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=hgb_pool_all mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.025518857878830903, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03776075999432083, 'khoa_quoc_te_hoc': 0.012527128166643003, 'khoa_tieng_anh': 0.04969715331724236, 'khoa_tieng_bo_dao_nha': 0.004755035190556356, 'khoa_tieng_duc': 0.011619475488307, 'khoa_tieng_han_quoc': 0.018970067744356327, 'khoa_tieng_italia': 0.011903433868121616, 'khoa_tieng_nga': 0.008504807009715433, 'khoa_tieng_nhat': 0.018591667849826583, 'khoa_tieng_tay_ban_nha': 0.008487693446646249, 'khoa_tieng_trung_quoc': 0.03667816867127761, 'khoa_viet_nam_hoc': 0.004985751374155731} source_failures={'khoa_tieng_phap': 322, 'khoa_cong_nghe_thong_tin': 17, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 1, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 18} positive_ess=304.190849 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=hgb_pool_equal rows=2983 failures=57 prevalence=0.019108 ap=0.248858 auc=0.902121 f2_at20=0.254545 brier=0.016629 ece10=0.009082 rho=0.250000 donor_rows=4095 effective_n=26508.406734 seconds=0.712578 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=hgb_pool_equal in_sample_bce=0.023858 local_bce=0.021436 donor_bce=0.031122 in_sample_local_ap=0.826310 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=hgb_pool_equal mass={'khoa_tieng_phap': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_phap': 322, 'khoa_cong_nghe_thong_tin': 17, 'khoa_quan_tri_kinh_doanh_va_du_lich': 13, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 6, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 4, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 13, 'khoa_tieng_nga': 18, 'khoa_tieng_nhat': 1, 'khoa_tieng_tay_ban_nha': 8, 'khoa_tieng_trung_quoc': 0, 'khoa_viet_nam_hoc': 18} positive_ess=388.726190 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=hgb_language_prior rows=2983 failures=57 prevalence=0.019108 ap=0.251860 auc=0.869926 f2_at20=0.272727 brier=0.016587 ece10=0.008936 rho=0.250000 donor_rows=4095 effective_n=26508.406734 seconds=0.682784 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=hgb_language_prior in_sample_bce=0.022256 local_bce=0.020516 donor_bce=0.027474 in_sample_local_ap=0.839827 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=hgb_language_prior mass={'khoa_tieng_phap': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_phap': 322, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 23, 'khoa_tieng_nhat': 3, 'khoa_tieng_tay_ban_nha': 12, 'khoa_tieng_trung_quoc': 7} positive_ess=368.637428 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=hgb_random_donor rows=2983 failures=57 prevalence=0.019108 ap=0.222356 auc=0.877283 f2_at20=0.260606 brier=0.016851 ece10=0.008239 rho=0.250000 donor_rows=4096 effective_n=26511.025377 seconds=0.686952 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=hgb_random_donor in_sample_bce=0.030447 local_bce=0.022743 donor_bce=0.053559 in_sample_local_ap=0.809994 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=hgb_random_donor mass={'khoa_tieng_phap': 0.75, np.str_('khoa_tieng_nga'): 0.25} source_failures={'khoa_tieng_phap': 322, np.str_('khoa_tieng_nga'): 186} positive_ess=449.043174 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=suite32_selective_common rows=2983 failures=57 prevalence=0.019108 ap=0.228799 auc=0.889454 f2_at20=0.260606 brier=0.016625 ece10=0.009786 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=0.653133 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=suite32_selective_common in_sample_bce=0.015205 local_bce=0.015205 donor_bce=nan in_sample_local_ap=0.906442 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=suite32_selective_common mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=suite32_selective_native rows=2983 failures=57 prevalence=0.019108 ap=0.256756 auc=0.890390 f2_at20=0.254545 brier=0.016211 ece10=0.008501 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=1.729109 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=suite32_selective_native in_sample_bce=0.015213 local_bce=0.015213 donor_bce=nan in_sample_local_ap=0.907489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=suite32_selective_native mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=native_cache_nan_indicator rows=2983 failures=57 prevalence=0.019108 ap=0.256756 auc=0.890390 f2_at20=0.254545 brier=0.016211 ece10=0.008501 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=4.028103 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=native_cache_nan_indicator in_sample_bce=0.015213 local_bce=0.015213 donor_bce=nan in_sample_local_ap=0.907489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=native_cache_nan_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=native_naive_zero_indicator rows=2983 failures=57 prevalence=0.019108 ap=0.256756 auc=0.890390 f2_at20=0.254545 brier=0.016211 ece10=0.008501 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=3.945985 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=native_naive_zero_indicator in_sample_bce=0.015213 local_bce=0.015213 donor_bce=nan in_sample_local_ap=0.907489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=native_naive_zero_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=native_placebo_indicator rows=2983 failures=57 prevalence=0.019108 ap=0.256756 auc=0.890390 f2_at20=0.254545 brier=0.016211 ece10=0.008501 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=3.867559 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=native_placebo_indicator in_sample_bce=0.015213 local_bce=0.015213 donor_bce=nan in_sample_local_ap=0.907489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=native_placebo_indicator mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan rows=2983 failures=57 prevalence=0.019108 ap=0.256756 auc=0.890390 f2_at20=0.254545 brier=0.016211 ece10=0.008501 rho=0.000000 donor_rows=0 effective_n=25043.000000 seconds=3.708620 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan in_sample_bce=0.015213 local_bce=0.015213 donor_bce=nan in_sample_local_ap=0.907489 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_phap model=suite32_selective_native_cache_nan mass={'khoa_tieng_phap': 1.0} source_failures={'khoa_tieng_phap': 322} positive_ess=322.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_phap model=committee_reference ap=0.254867 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_phap models=13 positive_rows=57 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_tay_ban_nha fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=5 origin=6066 ap=0.257596 rows=1401 failures=63 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=-0.016147 ap=0.241449 base_ap=0.257596 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.014870 ap=0.242727 base_ap=0.257596 donor_failures=125 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6066 delta_ap=0.019654 ap=0.277250 base_ap=0.257596 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6066 delta_ap=-0.022389 ap=0.235207 base_ap=0.257596 donor_failures=77 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.057171 ap=0.200425 base_ap=0.257596 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6066 delta_ap=-0.072697 ap=0.184899 base_ap=0.257596 donor_failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.024384 ap=0.233212 base_ap=0.257596 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6066 delta_ap=-0.006044 ap=0.251552 base_ap=0.257596 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6066 delta_ap=-0.072389 ap=0.185207 base_ap=0.257596 donor_failures=162 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6066 delta_ap=-0.015603 ap=0.241993 base_ap=0.257596 donor_failures=46 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6066 delta_ap=0.006579 ap=0.264175 base_ap=0.257596 donor_failures=63 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.024565 ap=0.233032 base_ap=0.257596 donor_failures=17 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6066 delta_ap=-0.006408 ap=0.251188 base_ap=0.257596 donor_failures=283 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=5 origin=6067 ap=0.405225 rows=1242 failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.049277 ap=0.355948 base_ap=0.405225 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=-0.028264 ap=0.376961 base_ap=0.405225 donor_failures=117 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.010454 ap=0.394771 base_ap=0.405225 donor_failures=48 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6067 delta_ap=-0.040255 ap=0.364970 base_ap=0.405225 donor_failures=67 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.010955 ap=0.416180 base_ap=0.405225 donor_failures=129 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6067 delta_ap=0.046323 ap=0.451548 base_ap=0.405225 donor_failures=82 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.013944 ap=0.391281 base_ap=0.405225 donor_failures=17 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6067 delta_ap=-0.049493 ap=0.355733 base_ap=0.405225 donor_failures=148 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6067 delta_ap=-0.056603 ap=0.348623 base_ap=0.405225 donor_failures=180 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6067 delta_ap=0.026181 ap=0.431406 base_ap=0.405225 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6067 delta_ap=-0.025384 ap=0.379841 base_ap=0.405225 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.009714 ap=0.395511 base_ap=0.405225 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6067 delta_ap=-0.011084 ap=0.394141 base_ap=0.405225 donor_failures=253 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=5 origin=6069 ap=0.303630 rows=1417 failures=54 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.026414 ap=0.330044 base_ap=0.303630 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.010832 ap=0.314462 base_ap=0.303630 donor_failures=120 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6069 delta_ap=0.067513 ap=0.371143 base_ap=0.303630 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6069 delta_ap=0.038103 ap=0.341733 base_ap=0.303630 donor_failures=70 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=0.107436 ap=0.411067 base_ap=0.303630 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6069 delta_ap=0.096328 ap=0.399958 base_ap=0.303630 donor_failures=71 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.008524 ap=0.295106 base_ap=0.303630 donor_failures=22 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6069 delta_ap=0.040700 ap=0.344330 base_ap=0.303630 donor_failures=158 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6069 delta_ap=0.158696 ap=0.462326 base_ap=0.303630 donor_failures=197 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6069 delta_ap=0.057141 ap=0.360771 base_ap=0.303630 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6069 delta_ap=0.038686 ap=0.342316 base_ap=0.303630 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.021781 ap=0.325411 base_ap=0.303630 donor_failures=23 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6069 delta_ap=0.203262 ap=0.506892 base_ap=0.303630 donor_failures=245 [SCREEN_BASE] target=khoa_tieng_tay_ban_nha fold=5 origin=6070 ap=0.576896 rows=1306 failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.044318 ap=0.532578 base_ap=0.576896 donor_failures=144 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.034712 ap=0.542184 base_ap=0.576896 donor_failures=114 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.000359 ap=0.576537 base_ap=0.576896 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_anh origin=6070 delta_ap=-0.011860 ap=0.565036 base_ap=0.576896 donor_failures=72 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.004050 ap=0.580946 base_ap=0.576896 donor_failures=139 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_duc origin=6070 delta_ap=-0.020909 ap=0.555987 base_ap=0.576896 donor_failures=73 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_han_quoc origin=6070 delta_ap=-0.023908 ap=0.552988 base_ap=0.576896 donor_failures=29 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_italia origin=6070 delta_ap=-0.094610 ap=0.482286 base_ap=0.576896 donor_failures=126 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nga origin=6070 delta_ap=-0.018935 ap=0.557961 base_ap=0.576896 donor_failures=184 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_nhat origin=6070 delta_ap=-0.032195 ap=0.544701 base_ap=0.576896 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_phap origin=6070 delta_ap=-0.041917 ap=0.534979 base_ap=0.576896 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.017191 ap=0.559705 base_ap=0.576896 donor_failures=30 [TRANSFER_EDGE] fold=5 target=khoa_tieng_tay_ban_nha source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.020276 ap=0.556620 base_ap=0.576896 donor_failures=245 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_tay_ban_nha origin=6069 policy_sources=khoa_tieng_nhat,khoa_quoc_te_hoc policy_donor_count=2 base_ap=0.303630 policy_ap=0.273273 delta_ap=-0.030357 policy_donor_rows=4096 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_tay_ban_nha origin=6070 policy_sources=khoa_quoc_te_hoc,khoa_tieng_duc,khoa_tieng_nhat policy_donor_count=3 base_ap=0.576896 policy_ap=0.582725 delta_ap=0.005829 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_tay_ban_nha selected=[] reason=prequential_policy_delta_-0.012264_below_threshold replay_mean=-0.012264 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common rows=1606 failures=84 prevalence=0.052304 ap=0.334844 auc=0.859529 f2_at20=0.463526 brier=0.044143 ece10=0.036032 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=0.447341 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common in_sample_bce=0.028323 local_bce=0.028323 donor_bce=nan in_sample_local_ap=0.927468 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_local_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native rows=1606 failures=84 prevalence=0.052304 ap=0.319500 auc=0.837369 f2_at20=0.417933 brier=0.043937 ece10=0.034026 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=0.945906 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native in_sample_bce=0.028231 local_bce=0.028231 donor_bce=nan in_sample_local_ap=0.928149 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_local_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all rows=1606 failures=84 prevalence=0.052304 ap=0.299669 auc=0.853334 f2_at20=0.433131 brier=0.045080 ece10=0.033241 rho=0.250000 donor_rows=4095 effective_n=15581.590137 seconds=0.495857 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all in_sample_bce=0.036281 local_bce=0.035097 donor_bce=0.064695 in_sample_local_ap=0.892806 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_all mass={'khoa_tieng_tay_ban_nha': 0.75, 'khoa_cong_nghe_thong_tin': 0.024786616298426895, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.036677247608821996, 'khoa_quoc_te_hoc': 0.012167673046424807, 'khoa_tieng_anh': 0.04827113643751416, 'khoa_tieng_bo_dao_nha': 0.0046185935637663884, 'khoa_tieng_duc': 0.011286064677548046, 'khoa_tieng_han_quoc': 0.018425738039184784, 'khoa_tieng_italia': 0.011561875116973177, 'khoa_tieng_nga': 0.008260768920103625, 'khoa_tieng_nhat': 0.018058196002718702, 'khoa_tieng_phap': 0.015417680434204434, 'khoa_tieng_trung_quoc': 0.03562572030851368, 'khoa_viet_nam_hoc': 0.004842689545799308} source_failures={'khoa_tieng_tay_ban_nha': 368, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 5, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 25} positive_ess=424.881080 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal rows=1606 failures=84 prevalence=0.052304 ap=0.293825 auc=0.860983 f2_at20=0.440729 brier=0.045140 ece10=0.034263 rho=0.250000 donor_rows=4095 effective_n=17461.655057 seconds=0.570653 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal in_sample_bce=0.040789 local_bce=0.036799 donor_bce=0.052757 in_sample_local_ap=0.881530 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_pool_equal mass={'khoa_tieng_tay_ban_nha': 0.7500000000000001, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_tay_ban_nha': 368, 'khoa_cong_nghe_thong_tin': 12, 'khoa_quan_tri_kinh_doanh_va_du_lich': 10, 'khoa_quoc_te_hoc': 6, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 11, 'khoa_tieng_duc': 10, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 9, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 5, 'khoa_tieng_trung_quoc': 2, 'khoa_viet_nam_hoc': 25} positive_ess=482.320288 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior rows=1606 failures=84 prevalence=0.052304 ap=0.297048 auc=0.840443 f2_at20=0.417933 brier=0.044533 ece10=0.032100 rho=0.250000 donor_rows=4095 effective_n=17461.655057 seconds=0.530550 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior in_sample_bce=0.034360 local_bce=0.034501 donor_bce=0.033935 in_sample_local_ap=0.892189 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_language_prior mass={'khoa_tieng_tay_ban_nha': 0.7500000000000001, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_trung_quoc': 0.027777777777777776} source_failures={'khoa_tieng_tay_ban_nha': 368, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 6, 'khoa_tieng_han_quoc': 0, 'khoa_tieng_italia': 16, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 5, 'khoa_tieng_phap': 3, 'khoa_tieng_trung_quoc': 2} positive_ess=442.510620 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor rows=1606 failures=84 prevalence=0.052304 ap=0.313949 auc=0.828054 f2_at20=0.440729 brier=0.043971 ece10=0.032541 rho=0.250000 donor_rows=4096 effective_n=17462.791284 seconds=0.483986 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor in_sample_bce=0.043240 local_bce=0.037005 donor_bce=0.061946 in_sample_local_ap=0.872837 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=hgb_random_donor mass={'khoa_tieng_tay_ban_nha': 0.75, np.str_('khoa_tieng_nga'): 0.25} source_failures={'khoa_tieng_tay_ban_nha': 368, np.str_('khoa_tieng_nga'): 182} positive_ess=549.076172 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common rows=1606 failures=84 prevalence=0.052304 ap=0.334844 auc=0.859529 f2_at20=0.463526 brier=0.044143 ece10=0.036032 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=0.440359 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common in_sample_bce=0.028323 local_bce=0.028323 donor_bce=nan in_sample_local_ap=0.927468 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_common mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native rows=1606 failures=84 prevalence=0.052304 ap=0.319500 auc=0.837369 f2_at20=0.417933 brier=0.043937 ece10=0.034026 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=0.940199 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native in_sample_bce=0.028231 local_bce=0.028231 donor_bce=nan in_sample_local_ap=0.928149 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator rows=1606 failures=84 prevalence=0.052304 ap=0.319500 auc=0.837369 f2_at20=0.417933 brier=0.043937 ece10=0.034026 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=1.824851 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator in_sample_bce=0.028231 local_bce=0.028231 donor_bce=nan in_sample_local_ap=0.928149 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_cache_nan_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator rows=1606 failures=84 prevalence=0.052304 ap=0.326598 auc=0.827381 f2_at20=0.395137 brier=0.043972 ece10=0.034336 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=1.768100 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator in_sample_bce=0.028335 local_bce=0.028335 donor_bce=nan in_sample_local_ap=0.927329 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_naive_zero_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator rows=1606 failures=84 prevalence=0.052304 ap=0.319500 auc=0.837369 f2_at20=0.417933 brier=0.043937 ece10=0.034026 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=1.875010 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator in_sample_bce=0.028231 local_bce=0.028231 donor_bce=nan in_sample_local_ap=0.928149 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=native_placebo_indicator mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan rows=1606 failures=84 prevalence=0.052304 ap=0.319500 auc=0.837369 f2_at20=0.417933 brier=0.043937 ece10=0.034026 rho=0.000000 donor_rows=0 effective_n=13391.000000 seconds=1.866415 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan in_sample_bce=0.028231 local_bce=0.028231 donor_bce=nan in_sample_local_ap=0.928149 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_tay_ban_nha model=suite32_selective_native_cache_nan mass={'khoa_tieng_tay_ban_nha': 1.0} source_failures={'khoa_tieng_tay_ban_nha': 368} positive_ess=368.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_tay_ban_nha model=committee_reference ap=0.265166 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_tay_ban_nha models=13 positive_rows=84 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_tieng_trung_quoc fold=5 origins=[6067, 6069, 6070, 6071] availability=semester_proxy [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=5 origin=6067 ap=0.103548 rows=5584 failures=76 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=0.029565 ap=0.133113 base_ap=0.103548 donor_failures=127 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.052242 ap=0.155790 base_ap=0.103548 donor_failures=112 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6067 delta_ap=0.043923 ap=0.147471 base_ap=0.103548 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6067 delta_ap=0.073304 ap=0.176852 base_ap=0.103548 donor_failures=83 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=0.079393 ap=0.182942 base_ap=0.103548 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6067 delta_ap=0.040012 ap=0.143560 base_ap=0.103548 donor_failures=64 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.016305 ap=0.087244 base_ap=0.103548 donor_failures=24 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6067 delta_ap=-0.010490 ap=0.093059 base_ap=0.103548 donor_failures=159 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6067 delta_ap=-0.002017 ap=0.101532 base_ap=0.103548 donor_failures=183 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6067 delta_ap=0.010059 ap=0.113607 base_ap=0.103548 donor_failures=37 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6067 delta_ap=0.033262 ap=0.136810 base_ap=0.103548 donor_failures=53 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=0.016956 ap=0.120504 base_ap=0.103548 donor_failures=90 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6067 delta_ap=0.016901 ap=0.120450 base_ap=0.103548 donor_failures=254 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=5 origin=6069 ap=0.130208 rows=6920 failures=103 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=-0.006778 ap=0.123429 base_ap=0.130208 donor_failures=128 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=0.004993 ap=0.135201 base_ap=0.130208 donor_failures=108 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6069 delta_ap=0.022364 ap=0.152571 base_ap=0.130208 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6069 delta_ap=0.015319 ap=0.145527 base_ap=0.130208 donor_failures=69 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.004884 ap=0.125323 base_ap=0.130208 donor_failures=139 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6069 delta_ap=-0.000095 ap=0.130113 base_ap=0.130208 donor_failures=83 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.037905 ap=0.092302 base_ap=0.130208 donor_failures=23 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6069 delta_ap=-0.020660 ap=0.109548 base_ap=0.130208 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6069 delta_ap=-0.009764 ap=0.120444 base_ap=0.130208 donor_failures=209 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6069 delta_ap=-0.015013 ap=0.115195 base_ap=0.130208 donor_failures=38 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6069 delta_ap=0.059782 ap=0.189990 base_ap=0.130208 donor_failures=55 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=0.052826 ap=0.183033 base_ap=0.130208 donor_failures=124 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6069 delta_ap=0.016001 ap=0.146208 base_ap=0.130208 donor_failures=258 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=5 origin=6070 ap=0.449768 rows=5107 failures=104 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.103568 ap=0.346200 base_ap=0.449768 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.119639 ap=0.330128 base_ap=0.449768 donor_failures=108 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.102891 ap=0.346877 base_ap=0.449768 donor_failures=27 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6070 delta_ap=-0.118333 ap=0.331435 base_ap=0.449768 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=-0.158322 ap=0.291446 base_ap=0.449768 donor_failures=133 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6070 delta_ap=-0.012440 ap=0.437328 base_ap=0.449768 donor_failures=65 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6070 delta_ap=-0.113330 ap=0.336438 base_ap=0.449768 donor_failures=26 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6070 delta_ap=-0.068488 ap=0.381279 base_ap=0.449768 donor_failures=137 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6070 delta_ap=-0.059047 ap=0.390721 base_ap=0.449768 donor_failures=188 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6070 delta_ap=-0.068652 ap=0.381116 base_ap=0.449768 donor_failures=41 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6070 delta_ap=-0.082069 ap=0.367699 base_ap=0.449768 donor_failures=48 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=0.008803 ap=0.458570 base_ap=0.449768 donor_failures=108 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6070 delta_ap=-0.083666 ap=0.366102 base_ap=0.449768 donor_failures=239 [SCREEN_BASE] target=khoa_tieng_trung_quoc fold=5 origin=6071 ap=0.422212 rows=82 failures=8 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_cong_nghe_thong_tin origin=6071 delta_ap=-0.060007 ap=0.362204 base_ap=0.422212 donor_failures=128 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6071 delta_ap=0.023839 ap=0.446050 base_ap=0.422212 donor_failures=136 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_quoc_te_hoc origin=6071 delta_ap=0.036083 ap=0.458294 base_ap=0.422212 donor_failures=46 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_anh origin=6071 delta_ap=0.195664 ap=0.617876 base_ap=0.422212 donor_failures=58 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_bo_dao_nha origin=6071 delta_ap=-0.017720 ap=0.404491 base_ap=0.422212 donor_failures=144 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_duc origin=6071 delta_ap=0.073211 ap=0.495422 base_ap=0.422212 donor_failures=86 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_han_quoc origin=6071 delta_ap=-0.081234 ap=0.340977 base_ap=0.422212 donor_failures=25 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_italia origin=6071 delta_ap=-0.196904 ap=0.225307 base_ap=0.422212 donor_failures=135 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nga origin=6071 delta_ap=0.060411 ap=0.482622 base_ap=0.422212 donor_failures=170 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_nhat origin=6071 delta_ap=0.022880 ap=0.445091 base_ap=0.422212 donor_failures=34 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_phap origin=6071 delta_ap=0.028356 ap=0.450567 base_ap=0.422212 donor_failures=45 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_tieng_tay_ban_nha origin=6071 delta_ap=0.102234 ap=0.524445 base_ap=0.422212 donor_failures=109 [TRANSFER_EDGE] fold=5 target=khoa_tieng_trung_quoc source=khoa_viet_nam_hoc origin=6071 delta_ap=-0.128691 ap=0.293521 base_ap=0.422212 donor_failures=220 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_trung_quoc origin=6070 policy_sources=khoa_tieng_phap,khoa_tieng_anh,khoa_tieng_bo_dao_nha policy_donor_count=3 base_ap=0.449768 policy_ap=0.350585 delta_ap=-0.099182 policy_donor_rows=4095 selected_using_origins=6067,6069 [POLICY_REPLAY] fold=5 faculty=khoa_tieng_trung_quoc origin=6071 policy_sources=khoa_tieng_tay_ban_nha,khoa_tieng_phap policy_donor_count=2 base_ap=0.422212 policy_ap=0.428169 delta_ap=0.005957 policy_donor_rows=4096 selected_using_origins=6067,6069,6070 [SELECTION_FINAL] fold=5 faculty=khoa_tieng_trung_quoc selected=[] reason=prequential_policy_delta_-0.046613_below_threshold replay_mean=-0.046613 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_local_common rows=6995 failures=155 prevalence=0.022159 ap=0.237046 auc=0.917137 f2_at20=0.336800 brier=0.019796 ece10=0.009211 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=1.005399 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_local_common in_sample_bce=0.015044 local_bce=0.015044 donor_bce=nan in_sample_local_ap=0.815347 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_local_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_local_native rows=6995 failures=155 prevalence=0.022159 ap=0.229005 auc=0.914562 f2_at20=0.334324 brier=0.019824 ece10=0.008297 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=2.508412 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_local_native in_sample_bce=0.014969 local_bce=0.014969 donor_bce=nan in_sample_local_ap=0.814914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_local_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_pool_all rows=6995 failures=155 prevalence=0.022159 ap=0.348284 auc=0.931603 f2_at20=0.346706 brier=0.017291 ece10=0.005076 rho=0.250000 donor_rows=4095 effective_n=30419.528103 seconds=1.026271 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_pool_all in_sample_bce=0.020762 local_bce=0.020134 donor_bce=0.039006 in_sample_local_ap=0.662849 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_pool_all mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.027835315265486726, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.04118846792035398, 'khoa_quoc_te_hoc': 0.013664269911504425, 'khoa_tieng_anh': 0.05420837942477876, 'khoa_tieng_bo_dao_nha': 0.005186670353982301, 'khoa_tieng_duc': 0.012674225663716815, 'khoa_tieng_han_quoc': 0.020692063053097346, 'khoa_tieng_italia': 0.01298396017699115, 'khoa_tieng_nga': 0.009276825221238938, 'khoa_tieng_nhat': 0.020279314159292034, 'khoa_tieng_phap': 0.017314021017699115, 'khoa_tieng_tay_ban_nha': 0.009258158185840708, 'khoa_viet_nam_hoc': 0.005438329646017699} source_failures={'khoa_tieng_trung_quoc': 511, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 19, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 9, 'khoa_viet_nam_hoc': 17} positive_ess=276.027259 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal rows=6995 failures=155 prevalence=0.022159 ap=0.344325 auc=0.926941 f2_at20=0.344230 brier=0.017659 ece10=0.006503 rho=0.250000 donor_rows=4095 effective_n=40027.088110 seconds=1.078233 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal in_sample_bce=0.023573 local_bce=0.021033 donor_bce=0.031195 in_sample_local_ap=0.653135 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_pool_equal mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_viet_nam_hoc': 0.019230769230769232} source_failures={'khoa_tieng_trung_quoc': 511, 'khoa_cong_nghe_thong_tin': 9, 'khoa_quan_tri_kinh_doanh_va_du_lich': 8, 'khoa_quoc_te_hoc': 4, 'khoa_tieng_anh': 5, 'khoa_tieng_bo_dao_nha': 19, 'khoa_tieng_duc': 5, 'khoa_tieng_han_quoc': 5, 'khoa_tieng_italia': 12, 'khoa_tieng_nga': 21, 'khoa_tieng_nhat': 3, 'khoa_tieng_phap': 4, 'khoa_tieng_tay_ban_nha': 9, 'khoa_viet_nam_hoc': 17} positive_ess=365.635911 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_language_prior rows=6995 failures=155 prevalence=0.022159 ap=0.325739 auc=0.925457 f2_at20=0.341753 brier=0.017835 ece10=0.003790 rho=0.250000 donor_rows=4095 effective_n=40027.088110 seconds=0.999217 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_language_prior in_sample_bce=0.021348 local_bce=0.020173 donor_bce=0.024875 in_sample_local_ap=0.669245 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_language_prior mass={'khoa_tieng_trung_quoc': 0.75, 'khoa_tieng_anh': 0.027777777777777776, 'khoa_tieng_bo_dao_nha': 0.027777777777777776, 'khoa_tieng_duc': 0.027777777777777776, 'khoa_tieng_han_quoc': 0.027777777777777776, 'khoa_tieng_italia': 0.027777777777777776, 'khoa_tieng_nga': 0.027777777777777776, 'khoa_tieng_nhat': 0.027777777777777776, 'khoa_tieng_phap': 0.027777777777777776, 'khoa_tieng_tay_ban_nha': 0.027777777777777776} source_failures={'khoa_tieng_trung_quoc': 511, 'khoa_tieng_anh': 7, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 9, 'khoa_tieng_han_quoc': 3, 'khoa_tieng_italia': 14, 'khoa_tieng_nga': 29, 'khoa_tieng_nhat': 4, 'khoa_tieng_phap': 7, 'khoa_tieng_tay_ban_nha': 12} positive_ess=354.340263 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_random_donor rows=6995 failures=155 prevalence=0.022159 ap=0.328809 auc=0.935056 f2_at20=0.351659 brier=0.017363 ece10=0.003950 rho=0.250000 donor_rows=4096 effective_n=40033.058999 seconds=0.994199 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_random_donor in_sample_bce=0.016929 local_bce=0.018519 donor_bce=0.012159 in_sample_local_ap=0.704969 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=hgb_random_donor mass={'khoa_tieng_trung_quoc': 0.75, np.str_('khoa_tieng_anh'): 0.25} source_failures={'khoa_tieng_trung_quoc': 511, np.str_('khoa_tieng_anh'): 65} positive_ess=341.948357 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_common rows=6995 failures=155 prevalence=0.022159 ap=0.237046 auc=0.917137 f2_at20=0.336800 brier=0.019796 ece10=0.009211 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=0.979303 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_common in_sample_bce=0.015044 local_bce=0.015044 donor_bce=nan in_sample_local_ap=0.815347 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_common mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_native rows=6995 failures=155 prevalence=0.022159 ap=0.229005 auc=0.914562 f2_at20=0.334324 brier=0.019824 ece10=0.008297 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=2.388596 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_native in_sample_bce=0.014969 local_bce=0.014969 donor_bce=nan in_sample_local_ap=0.814914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_native mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator rows=6995 failures=155 prevalence=0.022159 ap=0.229005 auc=0.914562 f2_at20=0.334324 brier=0.019824 ece10=0.008297 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=12.324762 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator in_sample_bce=0.014969 local_bce=0.014969 donor_bce=nan in_sample_local_ap=0.814914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=native_cache_nan_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator rows=6995 failures=155 prevalence=0.022159 ap=0.229005 auc=0.914562 f2_at20=0.334324 brier=0.019824 ece10=0.008297 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=4.811372 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator in_sample_bce=0.014969 local_bce=0.014969 donor_bce=nan in_sample_local_ap=0.814914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=native_naive_zero_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator rows=6995 failures=155 prevalence=0.022159 ap=0.229005 auc=0.914562 f2_at20=0.334324 brier=0.019824 ece10=0.008297 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=4.690890 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator in_sample_bce=0.014969 local_bce=0.014969 donor_bce=nan in_sample_local_ap=0.814914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=native_placebo_indicator mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan rows=6995 failures=155 prevalence=0.022159 ap=0.229005 auc=0.914562 f2_at20=0.334324 brier=0.019824 ece10=0.008297 rho=0.000000 donor_rows=0 effective_n=57867.000000 seconds=4.639236 [TRAINING_INSIGHT] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan in_sample_bce=0.014969 local_bce=0.014969 donor_bce=nan in_sample_local_ap=0.814914 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_tieng_trung_quoc model=suite32_selective_native_cache_nan mass={'khoa_tieng_trung_quoc': 1.0} source_failures={'khoa_tieng_trung_quoc': 511} positive_ess=511.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_tieng_trung_quoc model=committee_reference ap=0.331898 [FACULTY_COMPLETE] fold=5 faculty=khoa_tieng_trung_quoc models=13 positive_rows=155 num_edges=52 num_replays=2 chosen=[] [SELECTION_ORIGINS] target=khoa_viet_nam_hoc fold=5 origins=[6066, 6067, 6069, 6070] availability=semester_proxy [SCREEN_BASE] target=khoa_viet_nam_hoc fold=5 origin=6066 ap=0.199706 rows=813 failures=30 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6066 delta_ap=0.022713 ap=0.222420 base_ap=0.199706 donor_failures=131 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6066 delta_ap=-0.007094 ap=0.192613 base_ap=0.199706 donor_failures=104 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6066 delta_ap=-0.061084 ap=0.138623 base_ap=0.199706 donor_failures=37 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6066 delta_ap=-0.037691 ap=0.162016 base_ap=0.199706 donor_failures=61 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6066 delta_ap=-0.051418 ap=0.148288 base_ap=0.199706 donor_failures=123 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6066 delta_ap=0.010331 ap=0.210037 base_ap=0.199706 donor_failures=74 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6066 delta_ap=-0.035584 ap=0.164122 base_ap=0.199706 donor_failures=23 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6066 delta_ap=-0.072130 ap=0.127576 base_ap=0.199706 donor_failures=113 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6066 delta_ap=0.017255 ap=0.216962 base_ap=0.199706 donor_failures=167 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6066 delta_ap=0.018217 ap=0.217924 base_ap=0.199706 donor_failures=39 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6066 delta_ap=-0.073071 ap=0.126636 base_ap=0.199706 donor_failures=58 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6066 delta_ap=-0.011735 ap=0.187972 base_ap=0.199706 donor_failures=91 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6066 delta_ap=-0.019755 ap=0.179951 base_ap=0.199706 donor_failures=16 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=5 origin=6067 ap=0.429234 rows=556 failures=39 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6067 delta_ap=-0.099568 ap=0.329666 base_ap=0.429234 donor_failures=132 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6067 delta_ap=0.017392 ap=0.446626 base_ap=0.429234 donor_failures=113 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6067 delta_ap=-0.051115 ap=0.378119 base_ap=0.429234 donor_failures=47 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6067 delta_ap=-0.039276 ap=0.389957 base_ap=0.429234 donor_failures=53 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6067 delta_ap=-0.084154 ap=0.345080 base_ap=0.429234 donor_failures=130 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6067 delta_ap=-0.050729 ap=0.378505 base_ap=0.429234 donor_failures=82 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6067 delta_ap=-0.005072 ap=0.424162 base_ap=0.429234 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6067 delta_ap=-0.088531 ap=0.340703 base_ap=0.429234 donor_failures=152 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6067 delta_ap=-0.008716 ap=0.420518 base_ap=0.429234 donor_failures=200 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6067 delta_ap=-0.033077 ap=0.396157 base_ap=0.429234 donor_failures=42 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6067 delta_ap=-0.050443 ap=0.378791 base_ap=0.429234 donor_failures=51 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6067 delta_ap=-0.044284 ap=0.384950 base_ap=0.429234 donor_failures=108 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6067 delta_ap=-0.039223 ap=0.390011 base_ap=0.429234 donor_failures=23 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=5 origin=6069 ap=0.370378 rows=773 failures=33 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6069 delta_ap=0.112537 ap=0.482915 base_ap=0.370378 donor_failures=138 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6069 delta_ap=-0.048963 ap=0.321415 base_ap=0.370378 donor_failures=115 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6069 delta_ap=0.006283 ap=0.376661 base_ap=0.370378 donor_failures=35 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6069 delta_ap=0.067684 ap=0.438062 base_ap=0.370378 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6069 delta_ap=-0.009337 ap=0.361041 base_ap=0.370378 donor_failures=134 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6069 delta_ap=0.140597 ap=0.510975 base_ap=0.370378 donor_failures=87 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6069 delta_ap=-0.026822 ap=0.343556 base_ap=0.370378 donor_failures=21 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6069 delta_ap=0.091954 ap=0.462332 base_ap=0.370378 donor_failures=163 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6069 delta_ap=0.018101 ap=0.388479 base_ap=0.370378 donor_failures=188 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6069 delta_ap=-0.027225 ap=0.343153 base_ap=0.370378 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6069 delta_ap=0.054211 ap=0.424589 base_ap=0.370378 donor_failures=50 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6069 delta_ap=-0.031141 ap=0.339237 base_ap=0.370378 donor_failures=98 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6069 delta_ap=0.048038 ap=0.418416 base_ap=0.370378 donor_failures=38 [SCREEN_BASE] target=khoa_viet_nam_hoc fold=5 origin=6070 ap=0.572566 rows=505 failures=18 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_cong_nghe_thong_tin origin=6070 delta_ap=-0.069593 ap=0.502973 base_ap=0.572566 donor_failures=139 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quan_tri_kinh_doanh_va_du_lich origin=6070 delta_ap=-0.016341 ap=0.556226 base_ap=0.572566 donor_failures=121 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_quoc_te_hoc origin=6070 delta_ap=-0.045354 ap=0.527212 base_ap=0.572566 donor_failures=40 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_anh origin=6070 delta_ap=-0.022492 ap=0.550074 base_ap=0.572566 donor_failures=71 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_bo_dao_nha origin=6070 delta_ap=0.041476 ap=0.614042 base_ap=0.572566 donor_failures=141 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_duc origin=6070 delta_ap=-0.044540 ap=0.528026 base_ap=0.572566 donor_failures=80 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_han_quoc origin=6070 delta_ap=-0.011312 ap=0.561255 base_ap=0.572566 donor_failures=29 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_italia origin=6070 delta_ap=-0.011279 ap=0.561287 base_ap=0.572566 donor_failures=145 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nga origin=6070 delta_ap=-0.034521 ap=0.538045 base_ap=0.572566 donor_failures=204 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_nhat origin=6070 delta_ap=-0.038821 ap=0.533745 base_ap=0.572566 donor_failures=30 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_phap origin=6070 delta_ap=-0.001746 ap=0.570820 base_ap=0.572566 donor_failures=62 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_tay_ban_nha origin=6070 delta_ap=-0.068683 ap=0.503883 base_ap=0.572566 donor_failures=98 [TRANSFER_EDGE] fold=5 target=khoa_viet_nam_hoc source=khoa_tieng_trung_quoc origin=6070 delta_ap=-0.044999 ap=0.527567 base_ap=0.572566 donor_failures=32 [POLICY_REPLAY] fold=5 faculty=khoa_viet_nam_hoc origin=6069 policy_sources=khoa_quan_tri_kinh_doanh_va_du_lich,khoa_tieng_nga policy_donor_count=2 base_ap=0.370378 policy_ap=0.364190 delta_ap=-0.006188 policy_donor_rows=4096 selected_using_origins=6066,6067 [POLICY_REPLAY] fold=5 faculty=khoa_viet_nam_hoc origin=6070 policy_sources=khoa_tieng_duc,khoa_cong_nghe_thong_tin,khoa_tieng_nga policy_donor_count=3 base_ap=0.572566 policy_ap=0.536333 delta_ap=-0.036233 policy_donor_rows=4095 selected_using_origins=6066,6067,6069 [SELECTION_FINAL] fold=5 faculty=khoa_viet_nam_hoc selected=[] reason=prequential_policy_delta_-0.021211_below_threshold replay_mean=-0.021211 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_local_common rows=638 failures=61 prevalence=0.095611 ap=0.530598 auc=0.897349 f2_at20=0.564516 brier=0.059272 ece10=0.037295 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.418490 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=hgb_local_common in_sample_bce=0.060668 local_bce=0.060668 donor_bce=nan in_sample_local_ap=0.909161 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_local_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_local_native rows=638 failures=61 prevalence=0.095611 ap=0.572626 auc=0.911399 f2_at20=0.604839 brier=0.056588 ece10=0.019299 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.489313 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=hgb_local_native in_sample_bce=0.059940 local_bce=0.059940 donor_bce=nan in_sample_local_ap=0.912266 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_local_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_pool_all rows=638 failures=61 prevalence=0.095611 ap=0.523639 auc=0.925988 f2_at20=0.658602 brier=0.057761 ece10=0.022578 rho=0.250000 donor_rows=4095 effective_n=10726.268574 seconds=0.539250 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=hgb_pool_all in_sample_bce=0.061261 local_bce=0.068726 donor_bce=0.052547 in_sample_local_ap=0.875763 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_pool_all mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.02445390074368138, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.03618492180534061, 'khoa_quoc_te_hoc': 0.012004344012769648, 'khoa_tieng_anh': 0.04762318361714379, 'khoa_tieng_bo_dao_nha': 0.004556597287178603, 'khoa_tieng_duc': 0.011134569643902711, 'khoa_tieng_han_quoc': 0.018178405786186135, 'khoa_tieng_italia': 0.01140667782634153, 'khoa_tieng_nga': 0.008149883017776924, 'khoa_tieng_nhat': 0.01781579733771298, 'khoa_tieng_phap': 0.015210725921462775, 'khoa_tieng_tay_ban_nha': 0.0081334836407103, 'khoa_tieng_trung_quoc': 0.03514750935979261} source_failures={'khoa_viet_nam_hoc': 457, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 0, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 3} positive_ess=515.765577 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_pool_equal rows=638 failures=61 prevalence=0.095611 ap=0.521001 auc=0.915348 f2_at20=0.672043 brier=0.058887 ece10=0.013827 rho=0.250000 donor_rows=4095 effective_n=11524.346951 seconds=0.505638 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=hgb_pool_equal in_sample_bce=0.063418 local_bce=0.067423 donor_bce=0.051404 in_sample_local_ap=0.882956 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_pool_equal mass={'khoa_viet_nam_hoc': 0.75, 'khoa_cong_nghe_thong_tin': 0.019230769230769232, 'khoa_quan_tri_kinh_doanh_va_du_lich': 0.019230769230769232, 'khoa_quoc_te_hoc': 0.019230769230769232, 'khoa_tieng_anh': 0.019230769230769232, 'khoa_tieng_bo_dao_nha': 0.019230769230769232, 'khoa_tieng_duc': 0.019230769230769232, 'khoa_tieng_han_quoc': 0.019230769230769232, 'khoa_tieng_italia': 0.019230769230769232, 'khoa_tieng_nga': 0.019230769230769232, 'khoa_tieng_nhat': 0.019230769230769232, 'khoa_tieng_phap': 0.019230769230769232, 'khoa_tieng_tay_ban_nha': 0.019230769230769232, 'khoa_tieng_trung_quoc': 0.019230769230769232} source_failures={'khoa_viet_nam_hoc': 457, 'khoa_cong_nghe_thong_tin': 13, 'khoa_quan_tri_kinh_doanh_va_du_lich': 11, 'khoa_quoc_te_hoc': 2, 'khoa_tieng_anh': 9, 'khoa_tieng_bo_dao_nha': 17, 'khoa_tieng_duc': 7, 'khoa_tieng_han_quoc': 2, 'khoa_tieng_italia': 7, 'khoa_tieng_nga': 14, 'khoa_tieng_nhat': 2, 'khoa_tieng_phap': 0, 'khoa_tieng_tay_ban_nha': 5, 'khoa_tieng_trung_quoc': 3} positive_ess=538.003759 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_language_prior rows=638 failures=61 prevalence=0.095611 ap=0.530598 auc=0.897349 f2_at20=0.564516 brier=0.059272 ece10=0.037295 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.415791 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=hgb_language_prior in_sample_bce=0.060668 local_bce=0.060668 donor_bce=nan in_sample_local_ap=0.909161 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_language_prior mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_random_donor rows=638 failures=61 prevalence=0.095611 ap=0.497260 auc=0.901057 f2_at20=0.618280 brier=0.060438 ece10=0.024409 rho=0.250000 donor_rows=4096 effective_n=11524.841851 seconds=0.482375 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=hgb_random_donor in_sample_bce=0.063204 local_bce=0.068037 donor_bce=0.048705 in_sample_local_ap=0.880464 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=hgb_random_donor mass={'khoa_viet_nam_hoc': 0.75, np.str_('khoa_tieng_duc'): 0.25} source_failures={'khoa_viet_nam_hoc': 457, np.str_('khoa_tieng_duc'): 88} positive_ess=534.437323 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_common rows=638 failures=61 prevalence=0.095611 ap=0.530598 auc=0.897349 f2_at20=0.564516 brier=0.059272 ece10=0.037295 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.459490 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_common in_sample_bce=0.060668 local_bce=0.060668 donor_bce=nan in_sample_local_ap=0.909161 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_common mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_native rows=638 failures=61 prevalence=0.095611 ap=0.572626 auc=0.911399 f2_at20=0.604839 brier=0.056588 ece10=0.019299 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.497383 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_native in_sample_bce=0.059940 local_bce=0.059940 donor_bce=nan in_sample_local_ap=0.912266 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_native mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator rows=638 failures=61 prevalence=0.095611 ap=0.572626 auc=0.911399 f2_at20=0.604839 brier=0.056588 ece10=0.019299 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.743301 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator in_sample_bce=0.059940 local_bce=0.059940 donor_bce=nan in_sample_local_ap=0.912266 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=native_cache_nan_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator rows=638 failures=61 prevalence=0.095611 ap=0.602575 auc=0.916342 f2_at20=0.618280 brier=0.055367 ece10=0.017656 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.688885 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator in_sample_bce=0.060615 local_bce=0.060615 donor_bce=nan in_sample_local_ap=0.909628 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=native_naive_zero_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=native_placebo_indicator rows=638 failures=61 prevalence=0.095611 ap=0.572626 auc=0.911399 f2_at20=0.604839 brier=0.056588 ece10=0.019299 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.763360 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=native_placebo_indicator in_sample_bce=0.059940 local_bce=0.059940 donor_bce=nan in_sample_local_ap=0.912266 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=native_placebo_indicator mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan rows=638 failures=61 prevalence=0.095611 ap=0.572626 auc=0.911399 f2_at20=0.604839 brier=0.056588 ece10=0.019299 rho=0.000000 donor_rows=0 effective_n=7866.000000 seconds=0.826249 [TRAINING_INSIGHT] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan in_sample_bce=0.059940 local_bce=0.059940 donor_bce=nan in_sample_local_ap=0.912266 fitted_iterations=100 note=not_validation_metrics [SOURCE_MASS] fold=5 faculty=khoa_viet_nam_hoc model=suite32_selective_native_cache_nan mass={'khoa_viet_nam_hoc': 1.0} source_failures={'khoa_viet_nam_hoc': 457} positive_ess=457.000000 [FACULTY_RESULTS] fold=5 faculty=khoa_viet_nam_hoc model=committee_reference ap=0.524450 [FACULTY_COMPLETE] fold=5 faculty=khoa_viet_nam_hoc models=13 positive_rows=61 num_edges=52 num_replays=2 chosen=[] [REPORT] source=saved_row_keyed_predictions output=/root/disk_3tb/earlywarning/30072026/results/suite32_selective_borrowing ==================================================================================================================== SUITE32 DEVELOPMENT F1-F5 — 14-FACULTY MACRO ==================================================================================================================== model cells faculty_count ap auc f2_at_20 brier ece10 committee_reference 70 14 0.355336 0.881401 0.339324 0.023894 0.017470 hgb_pool_equal 70 14 0.340947 0.885416 0.338058 0.024444 0.014280 hgb_language_prior 70 14 0.338552 0.882797 0.339894 0.024465 0.015188 hgb_pool_all 70 14 0.334922 0.884030 0.340875 0.024465 0.014643 hgb_random_donor 70 14 0.331377 0.881853 0.337686 0.024542 0.014903 suite32_selective_common 70 14 0.330632 0.878134 0.334753 0.024669 0.015975 hgb_local_common 70 14 0.329947 0.874562 0.332879 0.024769 0.016412 suite32_selective_native_cache_nan 70 14 0.328611 0.879312 0.336048 0.024555 0.015518 suite32_selective_native 70 14 0.328611 0.879312 0.336048 0.024555 0.015518 native_naive_zero_indicator 70 14 0.325182 0.873528 0.331882 0.024711 0.015812 hgb_local_native 70 14 0.324014 0.873054 0.332219 0.024742 0.015897 native_cache_nan_indicator 70 14 0.324014 0.873054 0.332219 0.024742 0.015897 native_placebo_indicator 70 14 0.324014 0.873054 0.332219 0.024742 0.015897 ==================================================================================================================== SUITE32 RETROSPECTIVE F6-F7 — 14-FACULTY MACRO ==================================================================================================================== No eligible cells [FOLD_MACRO] fold=1 rows=45800 summary={'committee_reference': 0.268776, 'hgb_language_prior': 0.261129, 'hgb_local_common': 0.244764, 'hgb_local_native': 0.227653, 'hgb_pool_all': 0.251966, 'hgb_pool_equal': 0.268105, 'hgb_random_donor': 0.252398, 'native_cache_nan_indicator': 0.227653, 'native_naive_zero_indicator': 0.230832, 'native_placebo_indicator': 0.227653, 'suite32_selective_common': 0.243237, 'suite32_selective_native': 0.227652, 'suite32_selective_native_cache_nan': 0.227652} [FOLD_MACRO] fold=2 rows=42122 summary={'committee_reference': 0.504181, 'hgb_language_prior': 0.444018, 'hgb_local_common': 0.444352, 'hgb_local_native': 0.43701, 'hgb_pool_all': 0.429494, 'hgb_pool_equal': 0.431132, 'hgb_random_donor': 0.430187, 'native_cache_nan_indicator': 0.43701, 'native_naive_zero_indicator': 0.438058, 'native_placebo_indicator': 0.43701, 'suite32_selective_common': 0.45689, 'suite32_selective_native': 0.450293, 'suite32_selective_native_cache_nan': 0.450293} [FOLD_MACRO] fold=3 rows=47420 summary={'committee_reference': 0.298085, 'hgb_language_prior': 0.327281, 'hgb_local_common': 0.300051, 'hgb_local_native': 0.284619, 'hgb_pool_all': 0.315198, 'hgb_pool_equal': 0.321348, 'hgb_random_donor': 0.312911, 'native_cache_nan_indicator': 0.284619, 'native_naive_zero_indicator': 0.287504, 'native_placebo_indicator': 0.284619, 'suite32_selective_common': 0.28802, 'suite32_selective_native': 0.290343, 'suite32_selective_native_cache_nan': 0.290343} [FOLD_MACRO] fold=4 rows=41376 summary={'committee_reference': 0.426142, 'hgb_language_prior': 0.384948, 'hgb_local_common': 0.395395, 'hgb_local_native': 0.393362, 'hgb_pool_all': 0.378491, 'hgb_pool_equal': 0.38927, 'hgb_random_donor': 0.377492, 'native_cache_nan_indicator': 0.393362, 'native_naive_zero_indicator': 0.39318, 'native_placebo_indicator': 0.393362, 'suite32_selective_common': 0.399192, 'suite32_selective_native': 0.393927, 'suite32_selective_native_cache_nan': 0.393927} [FOLD_MACRO] fold=5 rows=50698 summary={'committee_reference': 0.279498, 'hgb_language_prior': 0.275385, 'hgb_local_common': 0.265172, 'hgb_local_native': 0.277424, 'hgb_pool_all': 0.29946, 'hgb_pool_equal': 0.294881, 'hgb_random_donor': 0.283898, 'native_cache_nan_indicator': 0.277424, 'native_naive_zero_indicator': 0.276337, 'native_placebo_indicator': 0.277424, 'suite32_selective_common': 0.26582, 'suite32_selective_native': 0.280841, 'suite32_selective_native_cache_nan': 0.280841} [HISTORICAL MODEL REFERENCE] Archive AP cell comparisons, NOT student-paired [INSIGHT TRANSFER] Strongest and most harmful observed historical directed relationships fold target source evaluated_origins mean_delta_ap min_delta_ap max_delta_ap win_rate mean_donor_failures 1 khoa_tieng_phap khoa_tieng_tay_ban_nha 4 +0.131451 -0.055666 +0.394069 +0.500000 +79.500000 5 khoa_tieng_nhat khoa_cong_nghe_thong_tin 4 +0.118292 +0.035507 +0.191963 +1.000000 +122.000000 3 khoa_tieng_duc khoa_cong_nghe_thong_tin 4 +0.096412 -0.033083 +0.241748 +0.750000 +138.250000 2 khoa_tieng_phap khoa_tieng_bo_dao_nha 4 +0.090039 -0.006043 +0.324985 +0.750000 +90.000000 1 khoa_tieng_phap khoa_tieng_bo_dao_nha 4 +0.089011 -0.011823 +0.221186 +0.750000 +71.500000 5 khoa_tieng_nhat khoa_quan_tri_kinh_doanh_va_du_lich 4 +0.087318 -0.002549 +0.132843 +0.750000 +118.500000 3 khoa_tieng_duc khoa_tieng_nga 4 +0.086184 +0.015662 +0.176134 +1.000000 +170.750000 2 khoa_tieng_anh khoa_tieng_italia 4 +0.085369 -0.021194 +0.171177 +0.750000 +138.250000 2 khoa_quoc_te_hoc khoa_tieng_trung_quoc 4 +0.085366 +0.037378 +0.143722 +1.000000 +26.000000 2 khoa_tieng_anh khoa_quoc_te_hoc 4 +0.084504 -0.023309 +0.179947 +0.750000 +34.250000 3 khoa_quoc_te_hoc khoa_tieng_trung_quoc 4 +0.084165 +0.058257 +0.142660 +1.000000 +23.250000 2 khoa_quoc_te_hoc khoa_tieng_anh 4 +0.083751 +0.010751 +0.134245 +1.000000 +67.250000 fold target source evaluated_origins mean_delta_ap min_delta_ap max_delta_ap win_rate mean_donor_failures 4 khoa_tieng_nhat khoa_tieng_anh 4 -0.127467 -0.162830 -0.068759 +0.000000 +58.500000 3 khoa_tieng_nhat khoa_tieng_anh 4 -0.102287 -0.181842 +0.048809 +0.250000 +63.500000 5 khoa_tieng_nhat khoa_tieng_anh 4 -0.101628 -0.281828 -0.029797 +0.000000 +65.750000 2 khoa_tieng_nhat khoa_quoc_te_hoc 4 -0.094606 -0.267574 +0.045370 +0.250000 +35.250000 5 khoa_tieng_nhat khoa_tieng_duc 4 -0.092153 -0.226564 -0.004685 +0.000000 +75.250000 3 khoa_tieng_nhat khoa_tieng_duc 4 -0.081511 -0.219549 +0.032377 +0.500000 +68.750000 2 khoa_tieng_nhat khoa_tieng_nga 4 -0.075524 -0.268197 +0.056284 +0.250000 +154.250000 5 khoa_tieng_trung_quoc khoa_tieng_italia 4 -0.074135 -0.196904 -0.010490 +0.000000 +141.000000 2 khoa_tieng_tay_ban_nha khoa_tieng_duc 4 -0.071824 -0.214616 +0.008372 +0.250000 +65.750000 2 khoa_tieng_trung_quoc khoa_tieng_nga 4 -0.071224 -0.216075 +0.050253 +0.250000 +158.750000 4 khoa_tieng_nhat khoa_quoc_te_hoc 4 -0.070667 -0.221204 +0.072904 +0.250000 +34.750000 2 khoa_tieng_nhat khoa_tieng_italia 4 -0.069240 -0.227427 +0.092369 +0.250000 +139.250000 [INSIGHT_TRANSFER] edges_tested=910 positive_mean_edges=474 stable_positive_edges=272 note=historical_screen_not_prospective_evidence [INSIGHT ADMISSION] Chosen donor-policy admission / local fallback per faculty and fold fold faculty borrow_admitted donors reason historical_replay_delta 1 khoa_cong_nghe_thong_tin False [] prequential policy exceeds harm tolerance 0.005574 1 khoa_quan_tri_kinh_doanh_va_du_lich False [] no stable positive directed edge -0.129557 1 khoa_quoc_te_hoc True [khoa_tieng_anh, khoa_tieng_duc, khoa_viet_nam_hoc] admitted by older-origin prequential replay 0.082175 1 khoa_tieng_anh False [] prequential policy exceeds harm tolerance 0.014840 1 khoa_tieng_bo_dao_nha True [khoa_tieng_duc, khoa_tieng_trung_quoc, khoa_tieng_nga] admitted by older-origin prequential replay 0.012516 1 khoa_tieng_duc False [] prequential policy exceeds harm tolerance 0.065728 1 khoa_tieng_han_quoc False [] prequential policy delta -0.002342 below threshold -0.002342 1 khoa_tieng_italia False [] prequential policy delta -0.008454 below threshold -0.008454 1 khoa_tieng_nga False [] prequential policy exceeds harm tolerance 0.117336 1 khoa_tieng_nhat False [] prequential policy delta -0.079773 below threshold -0.079773 1 khoa_tieng_phap True [khoa_tieng_tay_ban_nha, khoa_tieng_bo_dao_nha, khoa_tieng_nhat] admitted by older-origin prequential replay 0.132372 1 khoa_tieng_tay_ban_nha True [khoa_quan_tri_kinh_doanh_va_du_lich, khoa_cong_nghe_thong_tin, khoa_tieng_trung_quoc] admitted by older-origin prequential replay 0.021651 1 khoa_tieng_trung_quoc True [khoa_tieng_anh, khoa_tieng_tay_ban_nha, khoa_tieng_han_quoc] admitted by older-origin prequential replay 0.067666 1 khoa_viet_nam_hoc False [] no stable positive directed edge -0.040775 2 khoa_cong_nghe_thong_tin True [khoa_tieng_han_quoc, khoa_tieng_nga, khoa_quoc_te_hoc] admitted by older-origin prequential replay 0.019430 2 khoa_quan_tri_kinh_doanh_va_du_lich False [] no stable positive directed edge -0.079877 2 khoa_quoc_te_hoc True [khoa_tieng_trung_quoc, khoa_tieng_anh, khoa_tieng_duc] admitted by older-origin prequential replay 0.032777 2 khoa_tieng_anh False [] prequential policy exceeds harm tolerance 0.022803 2 khoa_tieng_bo_dao_nha True [khoa_tieng_nga, khoa_cong_nghe_thong_tin, khoa_tieng_tay_ban_nha] admitted by older-origin prequential replay 0.031414 2 khoa_tieng_duc True [khoa_tieng_nga, khoa_tieng_phap, khoa_quoc_te_hoc] admitted by older-origin prequential replay 0.058855 2 khoa_tieng_han_quoc False [] prequential policy exceeds harm tolerance 0.008205 2 khoa_tieng_italia True [khoa_tieng_tay_ban_nha, khoa_tieng_trung_quoc, khoa_tieng_phap] admitted by older-origin prequential replay 0.029380 2 khoa_tieng_nga False [] prequential policy delta -0.003584 below threshold -0.003584 2 khoa_tieng_nhat False [] prequential policy exceeds harm tolerance 0.050063 2 khoa_tieng_phap False [] prequential policy exceeds harm tolerance 0.020366 2 khoa_tieng_tay_ban_nha True [khoa_tieng_trung_quoc, khoa_cong_nghe_thong_tin, khoa_quan_tri_kinh_doanh_va_du_lich] admitted by older-origin prequential replay 0.021345 2 khoa_tieng_trung_quoc False [] no stable positive directed edge -0.037729 2 khoa_viet_nam_hoc False [] no stable positive directed edge 0.000000 3 khoa_cong_nghe_thong_tin False [] prequential policy delta -0.007433 below threshold -0.007433 3 khoa_quan_tri_kinh_doanh_va_du_lich False [] no stable positive directed edge 0.000000 3 khoa_quoc_te_hoc True [khoa_tieng_trung_quoc, khoa_tieng_anh, khoa_tieng_duc] admitted by older-origin prequential replay 0.027297 3 khoa_tieng_anh False [] prequential policy delta -0.015320 below threshold -0.015320 3 khoa_tieng_bo_dao_nha True [khoa_tieng_nga, khoa_tieng_tay_ban_nha, khoa_cong_nghe_thong_tin] admitted by older-origin prequential replay 0.020527 3 khoa_tieng_duc True [khoa_cong_nghe_thong_tin, khoa_tieng_nga, khoa_quoc_te_hoc] admitted by older-origin prequential replay 0.036320 3 khoa_tieng_han_quoc False [] prequential policy delta -0.004552 below threshold -0.004552 3 khoa_tieng_italia False [] prequential policy delta -0.000090 below threshold -0.000090 3 khoa_tieng_nga False [] prequential policy delta -0.033121 below threshold -0.033121 3 khoa_tieng_nhat False [] prequential policy exceeds harm tolerance 0.055305 3 khoa_tieng_phap True [khoa_tieng_tay_ban_nha, khoa_tieng_bo_dao_nha, khoa_quan_tri_kinh_doanh_va_du_lich] admitted by older-origin prequential replay 0.030857 3 khoa_tieng_tay_ban_nha False [] prequential policy exceeds harm tolerance 0.006526 3 khoa_tieng_trung_quoc False [] prequential policy delta -0.095374 below threshold -0.095374 3 khoa_viet_nam_hoc False [] prequential policy delta -0.000370 below threshold -0.000370 4 khoa_cong_nghe_thong_tin False [] prequential policy exceeds harm tolerance 0.004009 4 khoa_quan_tri_kinh_doanh_va_du_lich False [] prequential policy delta -0.027816 below threshold -0.027816 4 khoa_quoc_te_hoc False [] prequential policy exceeds harm tolerance 0.022315 4 khoa_tieng_anh False [] prequential policy delta -0.044819 below threshold -0.044819 4 khoa_tieng_bo_dao_nha False [] prequential policy delta -0.022808 below threshold -0.022808 4 khoa_tieng_duc False [] prequential policy delta -0.019827 below threshold -0.019827 4 khoa_tieng_han_quoc False [] prequential policy delta -0.011651 below threshold -0.011651 4 khoa_tieng_italia True [khoa_viet_nam_hoc, khoa_quan_tri_kinh_doanh_va_du_lich, khoa_cong_nghe_thong_tin] admitted by older-origin prequential replay 0.007580 4 khoa_tieng_nga False [] prequential policy delta -0.001784 below threshold -0.001784 4 khoa_tieng_nhat False [] prequential policy exceeds harm tolerance 0.028930 4 khoa_tieng_phap False [] prequential policy delta -0.005443 below threshold -0.005443 4 khoa_tieng_tay_ban_nha False [] prequential policy exceeds harm tolerance 0.001440 4 khoa_tieng_trung_quoc False [] prequential policy delta -0.015840 below threshold -0.015840 4 khoa_viet_nam_hoc False [] no stable positive directed edge -0.047700 5 khoa_cong_nghe_thong_tin False [] prequential policy exceeds harm tolerance 0.018228 5 khoa_quan_tri_kinh_doanh_va_du_lich False [] prequential policy delta -0.037874 below threshold -0.037874 5 khoa_quoc_te_hoc False [] prequential policy delta -0.052183 below threshold -0.052183 5 khoa_tieng_anh False [] no stable positive directed edge -0.084747 5 khoa_tieng_bo_dao_nha False [] prequential policy delta -0.064685 below threshold -0.064685 5 khoa_tieng_duc False [] prequential policy delta -0.028981 below threshold -0.028981 5 khoa_tieng_han_quoc False [] prequential policy exceeds harm tolerance 0.019190 5 khoa_tieng_italia False [] prequential policy delta -0.026056 below threshold -0.026056 5 khoa_tieng_nga True [khoa_tieng_trung_quoc, khoa_tieng_anh, khoa_quan_tri_kinh_doanh_va_du_lich] admitted by older-origin prequential replay 0.006030 5 khoa_tieng_nhat True [khoa_cong_nghe_thong_tin, khoa_quan_tri_kinh_doanh_va_du_lich, khoa_tieng_tay_ban_nha] admitted by older-origin prequential replay 0.035250 5 khoa_tieng_phap False [] prequential policy delta -0.060972 below threshold -0.060972 5 khoa_tieng_tay_ban_nha False [] prequential policy delta -0.012264 below threshold -0.012264 5 khoa_tieng_trung_quoc False [] prequential policy delta -0.046613 below threshold -0.046613 5 khoa_viet_nam_hoc False [] prequential policy delta -0.021211 below threshold -0.021211 [INSIGHT_ADMISSION] target_fold_cells=70 admitted=18 rejected=52 [INSIGHT TRAINING] Training cost, mass concentration and effective support model fits mean_seconds mean_ess mean_positive_ess mean_donor_rows mean_in_sample_loss mean_local_in_sample_ap hgb_language_prior 70 0.669 23072.570 476.493 2925.000 0.033 0.863 hgb_local_common 70 0.642 23616.986 468.600 0.000 0.028 0.908 hgb_local_native 70 1.284 23616.986 468.600 0.000 0.028 0.910 hgb_pool_all 70 0.686 18997.096 407.506 4095.000 0.033 0.850 hgb_pool_equal 70 0.895 22803.467 473.927 4095.000 0.036 0.841 hgb_random_donor 70 0.710 22805.439 489.849 4095.743 0.034 0.852 native_cache_nan_indicator 70 2.653 23616.986 468.600 0.000 0.028 0.910 native_naive_zero_indicator 70 2.514 23616.986 468.600 0.000 0.028 0.910 native_placebo_indicator 70 2.435 23616.986 468.600 0.000 0.028 0.910 suite32_selective_common 70 0.703 24320.465 487.628 1053.000 0.030 0.896 suite32_selective_native 70 1.320 24320.465 487.628 1053.000 0.030 0.899 suite32_selective_native_cache_nan 70 2.615 24320.465 487.628 1053.000 0.030 0.899 [INSIGHT NOTES] Missingness masks from cached NaNs are NOT database-provenance masks; [INSIGHT NOTES] F6/F7 were repeatedly inspected historically; retrospective not untouched prospective tests. [INSIGHT NOTES] Historic Council/Suite25 aggregate references use non-identical protocols until row-keys verified. [COMPLETE] stage=development folder=/root/disk_3tb/earlywarning/30072026/results/suite32_selective_borrowing scope=development_and_or_retrospective [RUN STATUS] SUCCESS stage=development exit=0 finished=2026-10-16T19:53:31+00:00