# Suite26 production evidence restored for Suite27

The latest supplied production reports support incremental improvements over the joint tree control, but do not establish MINT as the strongest earlier model. Historical F6/F7 macro AP is **0.3382577658506684** for MINT, **0.3695255926437980** for Suite25 blend, and **0.3691592501395352** for Suite22. MINT trails them by 0.031267827 and 0.030901484 AP respectively. Its improvement over the separately fitted joint tree is 0.025486356; its improvement over the separately fitted memory model is 0.003804850. This is evidence about the archived Suite26 production run only. No Suite27 model was trained or scored for this restoration.

## Provenance and scope

`production_reports.zip` is a byte-for-byte copy of the latest user-supplied `suite26_review_reports.zip`, SHA-256 `5b2df8e420562eabc86ebea932de09c72a882fa70935551bec31cf10286cf5cb`. The archive contains 81 entries: 80 collected reports plus the collection manifest. Every listed report matches its byte count and SHA-256; ZIP CRC checks pass. All 10 production run-manifest Python hashes match the prior local `suite26_joint_tree_memory` source. `production_verification.json` records each comparison. These checks establish file integrity and source identity, not independent correctness of the external input caches.

The collection operation was `archive_existing_reports_without_recomputation`; no requested report was missing. It explicitly excludes prediction tables, model weights and input caches. The run manifest records 196 external input fingerprints, but those bytes are absent here and their hashes cannot be reverified. The archive has all seven fold data/training/structure/memory/regime/intervention reports and aggregate comparisons, so it supersedes the earlier console-log-only review's statements that these numerical reports were unavailable. That earlier review remains useful for source interpretation; synthetic smoke and validation directories are not production evidence.

There are 324,420 scored target course rows over F1–F7, 14 faculties, and 26 named models/references. F6/F7 contain 97,004 rows and 28 faculty/fold cells; 12,287 unique shared students are reported in full-support bootstrap comparisons. All 28 full-support cells have both classes. Metrics are an unweighted mean of eligible faculty/fold cells; AP/AUC are undefined for one-class cells. Counts are summed, while Brier, ECE and budget metrics are cell means, not pooled metrics. A faculty's holdout AP is the mean of its two semester APs, not AP after pooling semester scores.

The production run is one seed recipe (`20260930 + 1009 * fold`), with 64 depth-6 seed trees, history cap 32, 180,000 sampled examples per epoch, rank weight 0.05, and separate variant early stopping/full-history refits. Different fold seeds are not repeated seeds on one task. The full manifest is retained as `production_run_manifest.json`.

## Complete model comparison and coverage

This table includes every named model in the latest report, including unavailable references. Full faculty details appear in `full_faculty_holdout_comparison.csv` (364 rows: 14 faculties × 26 models) and `full_faculty_fold_comparison.csv` (2,548 rows: 14 × 7 × 26). Both retain all supplied support, AP, AUC, calibration and 10/15/20/25% budget metrics. Added `mint_shared_*`, `model_shared_*`, and `mint_minus_model_*` columns use exactly the archived pair's shared scored rows. Positive Brier/ECE deltas favor the comparator. Missing results remain blank; they are not replaced with zeros. The MINT self-deltas are identities.

| model | rows | valid_ap_cells | coverage | ap | auc | f2_at_10 | f2_at_15 | f2_at_20 | f2_at_25 | brier | ece10 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| v2 | 12577 | 2 | 0.129654 | 0.422654 | 0.826598 | 0.407362 | 0.482359 | 0.527904 | 0.552518 | 0.142849 | 0.162288 |
| suite25_blend | 97004 | 28 | 1.000000 | 0.369526 | 0.854274 | 0.424040 | 0.447655 | 0.454289 | 0.450406 | 0.052186 | 0.022578 |
| suite22 | 97004 | 28 | 1.000000 | 0.369159 | 0.855570 | 0.428971 | 0.453714 | 0.457353 | 0.453166 | 0.052508 | 0.023325 |
| suite24_posthoc | 97004 | 28 | 1.000000 | 0.362838 | 0.853278 | 0.424382 | 0.450066 | 0.454824 | 0.453194 | 0.052062 | 0.019741 |
| suite23 | 97004 | 28 | 1.000000 | 0.358265 | 0.850081 | 0.416086 | 0.444706 | 0.452802 | 0.446568 | 0.052685 | 0.022269 |
| suite17 | 97004 | 28 | 1.000000 | 0.357815 | 0.852825 | 0.417437 | 0.448243 | 0.453504 | 0.449479 | 0.053247 | 0.025095 |
| suite18 | 97004 | 28 | 1.000000 | 0.357805 | 0.852900 | 0.416996 | 0.449033 | 0.455319 | 0.449493 | 0.053261 | 0.025283 |
| suite21 | 97004 | 28 | 1.000000 | 0.357066 | 0.846907 | 0.412924 | 0.443145 | 0.446429 | 0.443557 | 0.053102 | 0.023850 |
| suite16 | 97004 | 28 | 1.000000 | 0.356077 | 0.851952 | 0.415130 | 0.452146 | 0.453872 | 0.448490 | 0.072796 | 0.088931 |
| suite24 | 97004 | 28 | 1.000000 | 0.354254 | 0.846098 | 0.410939 | 0.438871 | 0.448556 | 0.444515 | 0.052775 | 0.024209 |
| committee | 97004 | 28 | 1.000000 | 0.350152 | 0.843399 | 0.407801 | 0.439858 | 0.446492 | 0.442580 | 0.053294 | 0.024045 |
| suite25 | 97004 | 28 | 1.000000 | 0.341269 | 0.842423 | 0.397773 | 0.428029 | 0.439753 | 0.437815 | 0.053728 | 0.023776 |
| suite22_relational | 97004 | 28 | 1.000000 | 0.340764 | 0.846168 | 0.402966 | 0.437762 | 0.449392 | 0.444732 | 0.053463 | 0.027534 |
| suite26_original_joint_head | 97004 | 28 | 1.000000 | 0.339302 | 0.834148 | 0.404801 | 0.431545 | 0.442753 | 0.433863 | 0.054513 | 0.028979 |
| suite26_mint | 97004 | 28 | 1.000000 | 0.338258 | 0.845466 | 0.401843 | 0.431082 | 0.444953 | 0.438476 | 0.054723 | 0.030597 |
| suite22_primary | 97004 | 28 | 1.000000 | 0.336369 | 0.840535 | 0.394990 | 0.424367 | 0.438565 | 0.435104 | 0.054646 | 0.030688 |
| suite26_memory | 97004 | 28 | 1.000000 | 0.334453 | 0.844939 | 0.406290 | 0.437914 | 0.445575 | 0.439817 | 0.054670 | 0.032183 |
| suite20 | 97004 | 28 | 1.000000 | 0.324935 | 0.839578 | 0.395732 | 0.435020 | 0.436498 | 0.432744 | 0.099934 | 0.118914 |
| suite23_graph | 97004 | 28 | 1.000000 | 0.321909 | 0.831552 | 0.395014 | 0.425625 | 0.430633 | 0.428841 | 0.055275 | 0.026463 |
| suite26_tree_only | 97004 | 28 | 1.000000 | 0.312771 | 0.838584 | 0.384984 | 0.422430 | 0.436158 | 0.437983 | 0.055711 | 0.030676 |
| suite26_original_tree | 97004 | 28 | 1.000000 | 0.312259 | 0.843597 | 0.393166 | 0.430578 | 0.444876 | 0.443065 | 0.054311 | 0.020942 |
| suite26_original | 97004 | 28 | 1.000000 | 0.302149 | 0.830961 | 0.382773 | 0.412522 | 0.418191 | 0.417629 | 0.056116 | 0.026845 |
| suite26_original_memory | 97004 | 28 | 1.000000 | 0.287641 | 0.827734 | 0.370516 | 0.403520 | 0.411756 | 0.412610 | 0.057084 | 0.027961 |
| suite26_seed_forest | 97004 | 28 | 1.000000 | 0.280593 | 0.821782 | 0.347447 | 0.385531 | 0.404698 | 0.411446 | 0.059209 | 0.034971 |
| suite14 | 0 | 0 | 0.000000 | unavailable | unavailable | unavailable | unavailable | unavailable | unavailable | unavailable | unavailable |
| council | 0 | 0 | 0.000000 | unavailable | unavailable | unavailable | unavailable | unavailable | unavailable | unavailable | unavailable |

Budget F2 uses stable descending score order within a faculty/fold cell and `round(rate * scored_rows)`, with at least one warning for nonempty support. This is a course-row warning capacity, not a distinct-student capacity. AP, AUC and F2 are higher-is-better; Brier and 10-bin ECE are lower-is-better. ECE is a bin-dependent descriptive diagnostic, not a test of calibration correctness. Probabilities from different faculties/origins are not pooled to compute these macro values.

Suite25 blend is first among the 23 references/models with complete scored-row coverage; Suite22 is very close in AP and has the better macro AUC and 20% budget F2. V2's AP 0.422654 is only IT across two cells (12,577 rows, 12.9654% of holdout rows). Its paired MINT difference therefore answers the IT-only question, not the all-faculty question. Suite14 was rejected in all 98 faculty/fold alignments because usable complete unique identifiers were absent. Council was missing in 91 alignments and rejected for insufficient identifiers in seven. Their numerical scores are unavailable under this keyed comparison; the report cannot establish that they underperform. Six/28 MINT faculty/fold AP wins against Suite25 blend and nine/28 against Suite22 are descriptive counts.

All prior suites exposed by this production report are included. Earlier suites absent from its schema have no new reconstructable all-faculty numerical result here. The archive's reference alignment checks cannot be repeated without the excluded reference predictions; their recorded statuses are preserved in the source archive.

## Faculty and semester losses

| Faculty | Suite22 | Suite25 blend | Memory | MINT | Joint tree | MINT minus blend |
| --- | --- | --- | --- | --- | --- | --- |
| IT | 0.424266 | 0.420793 | 0.373012 | 0.377446 | 0.332595 | -0.043347 |
| Business / Tourism | 0.429798 | 0.419906 | 0.386594 | 0.385484 | 0.343530 | -0.034422 |
| International Studies | 0.273008 | 0.272923 | 0.216086 | 0.233305 | 0.202243 | -0.039618 |
| English | 0.260863 | 0.257801 | 0.249348 | 0.255518 | 0.200430 | -0.002283 |
| Portuguese | 0.462717 | 0.458830 | 0.396052 | 0.415356 | 0.374746 | -0.043474 |
| German | 0.402625 | 0.400027 | 0.328650 | 0.338867 | 0.290794 | -0.061161 |
| Korean | 0.358607 | 0.360616 | 0.375205 | 0.395348 | 0.348481 | 0.034733 |
| Italian | 0.314573 | 0.316856 | 0.300377 | 0.303219 | 0.264993 | -0.013637 |
| Russian | 0.393063 | 0.390821 | 0.367995 | 0.361376 | 0.337073 | -0.029445 |
| Japanese | 0.219105 | 0.230154 | 0.242362 | 0.253195 | 0.213133 | 0.023041 |
| French | 0.307482 | 0.317518 | 0.233100 | 0.234043 | 0.263789 | -0.083476 |
| Spanish | 0.422826 | 0.428023 | 0.429287 | 0.423908 | 0.401878 | -0.004115 |
| Chinese | 0.276961 | 0.275364 | 0.220380 | 0.213566 | 0.205016 | -0.061799 |
| Vietnamese Studies | 0.622336 | 0.623726 | 0.563893 | 0.544979 | 0.600100 | -0.078747 |

MINT wins the two-fold faculty mean against Suite25 blend in only Korean and Japanese (2/14 faculties), and against Suite22 in Korean, Japanese and Spanish (3/14). The largest faculty mean AP losses to Suite25 blend are French (-0.083476), Vietnamese Studies (-0.078747), Chinese (-0.061799), and German (-0.061161). These faculty summaries average two folds equally and do not quantify per-faculty uncertainty. A smaller faculty can materially affect the macro average even though it supplies few rows. F2 changes need not share the AP sign: for example, French loses AP but gains 20% budget F2 slightly against the blend. AP loss should not be converted directly into missed-failure counts.

| fold | model | ap | auc | f2_at_20 | brier | ece10 |
| --- | --- | --- | --- | --- | --- | --- |
| 6 | suite25_blend | 0.384644 | 0.865993 | 0.449756 | 0.043394 | 0.018463 |
| 6 | suite22 | 0.378657 | 0.864063 | 0.452094 | 0.043554 | 0.018846 |
| 6 | suite26_mint | 0.368707 | 0.866486 | 0.454774 | 0.044190 | 0.022240 |
| 6 | suite26_original_joint_head | 0.366995 | 0.864574 | 0.448534 | 0.045224 | 0.028284 |
| 6 | committee | 0.362279 | 0.854561 | 0.443233 | 0.045126 | 0.022310 |
| 6 | suite26_memory | 0.359737 | 0.862273 | 0.451046 | 0.043914 | 0.022226 |
| 6 | suite26_tree_only | 0.280156 | 0.839036 | 0.415739 | 0.048353 | 0.023767 |
| 6 | suite26_seed_forest | 0.257493 | 0.822802 | 0.385932 | 0.050738 | 0.028431 |
| 7 | suite22 | 0.359661 | 0.847076 | 0.462612 | 0.061463 | 0.027803 |
| 7 | suite25_blend | 0.354407 | 0.842555 | 0.458822 | 0.060979 | 0.026693 |
| 7 | suite26_tree_only | 0.345387 | 0.838131 | 0.456576 | 0.063068 | 0.037585 |
| 7 | committee | 0.338025 | 0.832237 | 0.449752 | 0.061462 | 0.025779 |
| 7 | suite26_original_joint_head | 0.311610 | 0.803722 | 0.436973 | 0.063802 | 0.029674 |
| 7 | suite26_memory | 0.309169 | 0.827605 | 0.440103 | 0.065425 | 0.042141 |
| 7 | suite26_mint | 0.307808 | 0.824446 | 0.435132 | 0.065256 | 0.038954 |
| 7 | suite26_seed_forest | 0.303692 | 0.820762 | 0.423465 | 0.067681 | 0.041510 |

| fold | component | mean_delta_ap | wins | mean_delta_auc | mean_delta_f2_20 |
| --- | --- | --- | --- | --- | --- |
| 6 | episodic_memory | 0.079581 | 13 | 0.023237 | 0.035307 |
| 6 | joint_tree_learning | 0.022662 | 11 | 0.016234 | 0.029807 |
| 6 | readiness_conditioning | 0.008970 | 8 | 0.004213 | 0.003728 |
| 7 | episodic_memory | -0.036218 | 2 | -0.010527 | -0.016473 |
| 7 | joint_tree_learning | 0.041695 | 11 | 0.017369 | 0.033111 |
| 7 | readiness_conditioning | -0.001361 | 7 | -0.003159 | -0.004971 |

Memory versus jointly fitted tree changes AP by +0.079581 in F6 (13/14 cells improve) and -0.036218 in F7 (2/14 improve). MINT versus tree is +0.088552 in F6 (14/14 improve) and -0.037579 in F7 (2/14 improve). MINT versus memory adds +0.008970 in F6 and -0.001361 in F7. The F6/F7 mean obscures this reversal. MINT has worse F7 AUC, 20% budget F2 and Brier than tree-only. Suite25 blend exceeds MINT AP in both folds; MINT's F6 AUC and 20% budget F2 are slightly better than the blend, but its F7 deficits are larger.

These are separately trained variants with different chosen training lengths. F6 tree/memory/MINT refit for 1/3/3 epochs and F7 for 5/7/7 epochs; each epoch uses 180,000 examples. MINT-minus-memory on those two folds has matched step counts, but the aggregate memory-minus-tree comparison is not an equal-step causal estimate. The aggregate memory gain accounts for roughly 85% of the MINT-minus-tree point gain; that arithmetic is a descriptive decomposition, not proof that memory caused that proportion.

## Validation support and structure learning

| fold | validation_rows | represented_faculties | largest_faculty_rows | semester | largest_faculty_fraction |
| --- | --- | --- | --- | --- | --- |
| 1 | 740 | 5 | 687 | 6065 | 0.928378 |
| 2 | 45800 | 14 | 8164 | 6066 | 0.178253 |
| 3 | 985 | 2 | 984 | 6068 | 0.998985 |
| 4 | 47420 | 14 | 7279 | 6069 | 0.153501 |
| 5 | 1092 | 5 | 1005 | 6071 | 0.920330 |
| 6 | 50698 | 14 | 8345 | 6072 | 0.164602 |
| 7 | 1334 | 5 | 809 | 6074 | 0.606447 |

The default one-latest-semester selection origin alternates between broad major cohorts and tiny sparse cohorts. F3 has 985 validation rows: 984 Business / Tourism and one English row. The single English row cannot have both classes, so the saved finite validation macro AP necessarily comes solely from Business. This is a demonstrated selection-population mismatch for an all-14-faculty objective, not merely a hypothesis from small total support. F1's 740 rows are 687 Business (92.84%); F5's 1,092 are 1,005 Business (92.03%). F7's 1,334 are Business 809, IT 374, Chinese 109, French 38 and English 4, with nine faculties absent. Positive counts/per-faculty selection AP are not saved, so exact eligible validation faculty counts beyond the F3 logical conclusion are unavailable.

Selection MINT AP ranges 0.55335–0.82331 in F1 and 0.40916–0.60323 in F7. These training-epoch values do not estimate the historical target macro AP. Sparse support and cohort shift are compatible with unstable stopping; this does not prove they caused F7 losses. The fixed selection-origin history bank is limited to the training partition, while the refit rebuilds preprocessing, course vocabulary, seed forest and history on all historical rows. Selected epoch transport to that different recipe remains untested by these reports.

Selector argmax stayed at its seed choice for every MINT and memory fold and for tree-only F2–F7. The blanket statement “selector argmax never moved” requires one correction: F1 tree-only changed one of 4,032 node choices (`feature_changed_fraction` 0.0002480158873; active-seed fraction 0.0003036744602). Positive logged selector gradients do not imply discrete structure search succeeded. The source initializes the chosen selector logit at 4 and alternatives at 0 and uses straight-through surrogate gradients; a persistent initial margin is a plausible mechanism, not a experimentally verified cause of unchanged choices.

Thresholds, temperatures, leaf probabilities, static context and memory shifts can still learn with fixed selectors. MINT mean base-threshold changes are 0.08221 in F6 and 0.14324 in F7; mean temperatures are 0.22718 and 0.20401. Thus the production run demonstrates mostly conditional split-threshold/leaf adaptation of seed-selected features. It does not demonstrate broad learned discrete feature replacement. Structural diagnostics are post-refit summaries, not a full trajectory of selector probabilities.

## F7 cold history and postfit sensitivities

F6 has 342/43,359 rows with zero eligible episodic history (0.7888%); F7 has 14,859/53,645 (27.6988%). These are no eligible events in the supplied earlier-semester bank, not proven first-time institutional students. Unknown target courses change much less (14.7490% to 15.5802%). The cold rows' observed failure prevalence is 12.8655% in F6 and 11.4207% in F7, compared with 5.4653% and 6.3992% for rows with history. This shift makes coverage/cohort effects a credible concern, but it does not isolate a causal explanation for the memory reversal.

| fold | stratum | model | rows | valid_ap_faculties | ap | f2_at_20 | brier |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 6 | 0 | suite26_memory | 342 | 9 | 0.615737 | 0.420052 | 0.142272 |
| 6 | 0 | suite26_mint | 342 | 9 | 0.585606 | 0.364496 | 0.146703 |
| 6 | 0 | suite26_tree_only | 342 | 9 | 0.568872 | 0.341769 | 0.142086 |
| 6 | 1-4 | suite26_memory | 5924 | 14 | 0.420402 | 0.444506 | 0.070939 |
| 6 | 1-4 | suite26_mint | 5924 | 14 | 0.459162 | 0.445167 | 0.071983 |
| 6 | 1-4 | suite26_tree_only | 5924 | 14 | 0.299902 | 0.382793 | 0.079686 |
| 6 | 17+ | suite26_memory | 16711 | 14 | 0.351334 | 0.335196 | 0.025065 |
| 6 | 17+ | suite26_mint | 16711 | 14 | 0.364643 | 0.336901 | 0.025292 |
| 6 | 17+ | suite26_tree_only | 16711 | 14 | 0.253180 | 0.316541 | 0.026900 |
| 6 | 5-16 | suite26_memory | 20382 | 14 | 0.396248 | 0.452792 | 0.048933 |
| 6 | 5-16 | suite26_mint | 20382 | 14 | 0.403139 | 0.452931 | 0.048645 |
| 6 | 5-16 | suite26_tree_only | 20382 | 14 | 0.308968 | 0.414822 | 0.055527 |
| 7 | 0 | suite26_memory | 14859 | 14 | 0.363159 | 0.437395 | 0.084140 |
| 7 | 0 | suite26_mint | 14859 | 14 | 0.367240 | 0.443370 | 0.082421 |
| 7 | 0 | suite26_tree_only | 14859 | 14 | 0.379559 | 0.464702 | 0.080696 |
| 7 | 1-4 | suite26_memory | 745 | 12 | 0.457566 | 0.278364 | 0.226347 |
| 7 | 1-4 | suite26_mint | 745 | 12 | 0.492558 | 0.272375 | 0.231044 |
| 7 | 1-4 | suite26_tree_only | 745 | 12 | 0.490434 | 0.278681 | 0.156284 |
| 7 | 17+ | suite26_memory | 16909 | 13 | 0.274200 | 0.352749 | 0.041464 |
| 7 | 17+ | suite26_mint | 16909 | 13 | 0.275887 | 0.350701 | 0.043061 |
| 7 | 17+ | suite26_tree_only | 16909 | 13 | 0.277299 | 0.370005 | 0.040575 |
| 7 | 5-16 | suite26_memory | 21132 | 14 | 0.332528 | 0.422032 | 0.063555 |
| 7 | 5-16 | suite26_mint | 21132 | 14 | 0.328523 | 0.422096 | 0.063036 |
| 7 | 5-16 | suite26_tree_only | 21132 | 14 | 0.353585 | 0.427120 | 0.063141 |

These regime APs average eligible faculty strata, which can change across strata/folds, and cannot be combined as row-weighted pieces of whole-fold AP. F6 zero-history AP uses nine eligible faculties and only 342 rows. F7 1–4-history AP uses 12 eligible faculties and 745 rows; 17+ uses 13. In F7, MINT/tree AP is 0.367240/0.379559 for cold rows, 0.328523/0.353585 for 5–16 histories, and 0.275887/0.277299 for 17+ histories. Losses are therefore not confined to cold rows. MINT also trails tree in both known- and unknown-course F7 strata. The supplied regime reports include Suite26 controls and committee, not Suite22 or Suite25 blend; no strongest-prior regime comparison can be invented.

| intervention | ap F6 | ap F7 | delta_ap_vs_fitted_primary F6 | delta_ap_vs_fitted_primary F7 |
| --- | --- | --- | --- | --- |
| hard_routing | 0.364873 | 0.305123 | -0.003835 | -0.002685 |
| history_k_16 | 0.368671 | 0.299429 | -0.000036 | -0.008379 |
| history_k_32 | 0.368707 | 0.307808 | 0.000000 | 0.000000 |
| history_k_4 | 0.348824 | 0.274809 | -0.019883 | -0.033000 |
| history_k_8 | 0.356076 | 0.286880 | -0.012631 | -0.020928 |
| no_ability | 0.361667 | 0.314996 | -0.007041 | 0.007187 |
| no_context | 0.257864 | 0.308568 | -0.110843 | 0.000760 |
| no_memory | 0.273320 | 0.324423 | -0.095387 | 0.016614 |
| no_outcomes | 0.254365 | 0.279864 | -0.114342 | -0.027944 |
| primary | 0.368707 | 0.307808 | 0.000000 | 0.000000 |
| shuffled_students | 0.158499 | 0.195797 | -0.210208 | -0.112012 |
| uniform_attention | 0.361642 | 0.320295 | -0.007065 | 0.012487 |

All rows above are **postfit inference sensitivities of the same fitted MINT**, not retrained ablations. `no_memory` suppresses both the episodic memory shift and readiness gap shift, while keeping static context, learned thresholds/leaves and other fitted parameters. Its F7 AP rises by 0.016614173 to 0.324422517, whereas F6 falls by 0.095387195. It is not equivalent to the separately trained tree model and does not show that retraining without memory would yield those scores. `no_ability` suppresses the gap shift only; F7 rises by 0.007187378 and F6 falls by 0.007040547. This is mixed sensitivity evidence for readiness conditioning, not validated latent ability or requirement semantics.

Uniform attention improves F7 by 0.012486910 and reduces F6 by 0.007065360. Hard routing alters AP only -0.003834640 (F6) and -0.002685288 (F7), far smaller than F7's MINT-versus-blend gap. This indicates inference softness is not a large direct repair in these checkpoints; it does not exclude a different outcome from hard-routing training. No-outcomes removes explicit historical failure/grade channels and the readiness gap but leaves raw historical features and course context intact; its AP drop cannot be interpreted as removing all historical outcome information. Inference history caps 4/8/16/32 show no universal best history length; 32 helps F7 relative to 16, while F4 favors four. These postfit observations cannot select a new hyperparameter on F6/F7 and then claim untouched evaluation.

Shuffled-student histories reduce AP substantially in both folds. The shuffle matches recipient/donor query faculty and semester, but not course, history length, grade release, or course novelty. Its audit records all donors as different students, no reuse, and changed histories for 43,352/43,359 F6 and 49,527/53,645 F7 rows. Identical empty histories explain why different donors need not change the bank indices; support/cohort mismatches remain possible. This tests sensitivity to another student's bank, not the causal value of personalized memory under a matched counterfactual or training-time removal.

## Calibration and frozen warning policy

At equal 20% course-row budgets, holdout F2 is 0.444953 MINT, 0.445575 memory, 0.436158 tree, 0.454289 Suite25 blend, and 0.457353 Suite22. MINT's marginal AP gain over memory accompanies a small F2 reduction; its larger AP gain over tree accompanies a modest F2 gain. Its Brier 0.054723 and ECE 0.030597 are worse than Suite25 blend (0.052186/0.022578) and Suite22 (0.052508/0.023325). These probability-quality losses are compatible with inadequate calibration/transport, but cannot establish which upstream modeling choice produced them. Brier includes both discrimination and calibration and is not a pure calibration error.

The separately reported frozen policy chooses a per-faculty/model F2 threshold using development folds 1–5, then applies it to F6/F7. The `POOLED` row in operational metrics recomputes confusion-derived quantities after pooling each faculty's two holdout terms, so it differs from the cell-mean 20% budget table. Under this fixed policy IT is:

| model | tp | fp | fn | recall | precision | f2 | flag_rate |
| --- | --- | --- | --- | --- | --- | --- | --- |
| committee | 1152.000000 | 2455.000000 | 459.000000 | 0.715084 | 0.319379 | 0.573077 | 0.286793 |
| suite26_seed_forest | 899.000000 | 2420.000000 | 712.000000 | 0.558038 | 0.270865 | 0.460412 | 0.263894 |
| suite26_tree_only | 1003.000000 | 2281.000000 | 608.000000 | 0.622595 | 0.305420 | 0.515522 | 0.261112 |
| suite26_memory | 906.000000 | 1723.000000 | 705.000000 | 0.562384 | 0.344618 | 0.499284 | 0.209032 |
| suite26_mint | 845.000000 | 1538.000000 | 766.000000 | 0.524519 | 0.354595 | 0.478645 | 0.189473 |
| suite22 | 1149.000000 | 2483.000000 | 462.000000 | 0.713222 | 0.316355 | 0.570167 | 0.288781 |
| suite25_blend | 1094.000000 | 2170.000000 | 517.000000 | 0.679081 | 0.335172 | 0.563453 | 0.259521 |
| suite26_original_joint_head | 730.000000 | 1255.000000 | 881.000000 | 0.453135 | 0.367758 | 0.433029 | 0.157828 |

MINT misses 766 IT failures versus committee 459, Suite25 blend 517 and Suite22 462. Compared with committee, it misses 307 more while generating 917 fewer false alarms; the realized warning rates differ. This supports an unfavorable recall/F2 tradeoff under the selected policy, not a causal conclusion about rank quality at matched capacity. Course-row capacity metrics and threshold transport must be considered separately. Unique-student warning burden cannot be reconstructed from these aggregate reports because the keyed predictions were excluded.

## Conditional uncertainty and claims boundary

| model_b | delta_ap | ci_low | ci_high | cells | rows | students | valid_replicates |
| --- | --- | --- | --- | --- | --- | --- | --- |
| committee | -0.011894 | -0.024565 | 0.005465 | 28 | 97004 | 12287 | 200 |
| suite26_tree_only | 0.025486 | 0.007962 | 0.037411 | 28 | 97004 | 12287 | 200 |
| suite26_memory | 0.003805 | -0.003556 | 0.009587 | 28 | 97004 | 12287 | 200 |
| suite22 | -0.030901 | -0.044832 | -0.015940 | 28 | 97004 | 12287 | 200 |
| suite25_blend | -0.031268 | -0.042118 | -0.015554 | 28 | 97004 | 12287 | 200 |
| v2 | -0.045208 | -0.070325 | -0.023013 | 2 | 12577 | 1491 | 200 |
| suite26_original_joint_head | -0.001045 | -0.011830 | 0.011867 | 28 | 97004 | 12287 | 200 |

Each scored comparison has 200 valid paired student-cluster draws and zero invalid draws; Suite14/council have no eligible cells and no draws. The documented bootstrap reuses each student's sampled multiplicity across all faculties/folds and fixes the original eligible cell set. Archived replicate quantiles reconcile with the reported intervals. That checks saved interval arithmetic; AP, score alignment and student resampling cannot be independently recomputed without the excluded predictions.

The MINT-minus-memory interval crosses zero, so this conditional student-sampling calculation does not distinguish the small gain from zero. The tree comparison is positive and strongest-prior differences are negative under the same calculation. These are descriptive intervals conditional on the trained checkpoints, the observed two calendar origins, their support and the report's identity assumptions. They do not include training-seed variation, unseen-semester/cohort shifts, earlier adaptive model selection, or subsequent Suite27 choices informed by F6/F7. There are many comparator/metric views and no multiplicity correction, practical improvement threshold or confirmatory design here. No future performance guarantee or seed-significance claim is supported.

F6/F7 are explicitly labeled historical comparative benchmarks with acknowledged project-level reuse in `report.json`. They can guide failure diagnosis and regression checks for Suite27, but cannot become untouched confirmatory tests again. The useful design evidence is the validation population mismatch, the mostly fixed seed feature choices, the F6/F7 memory reversal, the F7 postfit removal sensitivity, and weaker calibration/operational results relative to the strongest full-support priors. These motivate prospective development controls and availability-aware rolling origins; they do not prove any proposed Suite27 architecture will improve performance.

## Strict grade and raw-cache limitations

The source enforces nonmissing grade values in 0–10 and requires failure labels to match grade <5, rejecting nonnumeric or inconsistent known grades. Missing historical grades are allowed with a presence mask and binary-outcome fallback. All seven production audits report **zero missing historical grades** and target grades discarded. This verifies the executed loader's reported consistency/availability flags; it is not independent institutional ground truth and does not establish when the grades were released. No release timestamps are supplied, so “previous-semester grades available by the next semester” remains an explicit assumption. A delayed-grade/embargo experiment cannot be recomputed from this archive.

Strict earlier-semester episodic lookup and global history/target cutoff checks protect calendar order. Training-only imputation/scaling/vocabulary fitting protects the fitted preprocessing. Nevertheless canonical raw features and covered structured history/course fields retain their external cache values; the loader only reconstructs the uncovered history course fields chronologically. Each fold reports 10,802 reconstructed history rows, while the larger covered remainder is trusted. Raw/structured cache upstream as-of feature provenance, grade-dependent summaries, prior model score generation and ID namespace correctness cannot be certified by report hashes. History grades being internally consistent with labels does not establish future-free raw covariates.

Every production faculty/fold audit reports explicit `target_row_index` alignment (98/98), so the missing-target-index fallback is not an observed deficiency of this run. A permutation index still does not independently prove semantic event identity without unavailable underlying tables. Globally canonical student/course identifiers are an input assumption; global lookup can merge entities if institutional IDs are locally scoped. No such collision is demonstrated here. Genuine origin-time reconstruction and per-feature availability metadata require the original caches/raw event records; this report-only restoration neither performs nor claims that audit.

## Files and reproducibility

The two full-faculty CSVs are the complete comparison views. `production_macro_comparison.csv`, `production_paired_holdout.csv`, `production_bootstrap_ap.csv`, `production_component_uplifts.csv`, and `production_operational_metrics.csv` retain the aggregate production comparisons. `production_validation_support.csv`, `production_structure_diagnostics.csv`, `production_memory_summary.csv`, `production_regime_summary.csv`, and `production_intervention_summary.csv` expose the diagnosis arithmetic. The authoritative archived reports remain in `production_reports.zip`.

Re-run `python evidence/production_restore_analysis.py --suite26-source /path/to/suite26_joint_tree_memory` from a saved Suite27 package, or pass `--reports /path/to/suite26_review_reports.zip --out /path/to/evidence`. The script verifies every collected report, reconciles all faculty/fold/macro tables within 2e-12 supplied-CSV precision, verifies archived bootstrap quantiles, and writes only evidence summaries. Missing source files are recorded as unavailable; a present source directory with mismatching hashes is rejected. It never loads joblib inputs, retrains a model, resamples predictions, or estimates Suite27 performance.
