# Suite32 SELECT — Target-conditioned historical borrowing with faculty-native information

**Status:** Complete code package and synthetic integration tested. **No actual AQ F1–F7 experiment has been run by this package's author.** Real-score claims are prohibited until executed on the canonical server caches.

## Primary hypothesis and implementation boundary

A *single target-conditioned training procedure* can improve a course-registration failure ranking by selecting eligible history from useful other faculties, without blindly sharing every observation or discarding the target's native predictors.

- Task: binary `grade10 < 5` at semester origin, one row per student × course × semester registration.
- Data: canonical cross-fitted history cache `results/cross_faculty_residual_moe/cache/<faculty>_fold_<fold>.joblib`; structured features `results/suite12_structured_cache/<faculty>_fold_<fold>.joblib`. The last 17 structured course fields preserve Suite25–31's contract.
- Stage: F1–F5 development, F6/F7 *retrospective*; no new independent holdout is implied.
- Primary ranking: equal-weight faculty × fold AP. Fixed 10/15/20/25% alert budgets, AUC, Brier, ECE, and development-frozen F2 thresholds are supplementary.
- The predictor is **one trained HGB per target faculty and fold** under the same learning algorithm, with learned historical donor policy. There is no Committee, Council, SACR or any second model at inference inside the Suite32 predictions; those are comparisons only.
- Source schema features are used by name. The common block is **intersection across all selected source faculties** (normally 136 raw fields) + the 17 structured course variables. Target-native mode additionally retains all additional raw fields actually present in the target faculty cache. It **does not create 275 fields from nothing**.

## Why donor screening is not a learned oracle on F6/F7

For each target fold and faculty:

1. Construct historical selection origins from the target's *historical*, chronologically ordered semesters only. Require minimum history, positives and validation support.
2. For each historical origin, fit the **same HGB** from earlier histories for (a) target-only and (b) target + each other faculty, using a fixed 25%-mass donor policy and a uniform donor cap. Save each directed historical AP difference.
3. Replay source selection: at historical validation origin `u`, only *earlier* historical validation origins choose donors; fit a single combined model and measure performance at `u`. A source must have support across earlier origins and positive mean/positive fraction.
4. Source admission for the outer target requires sufficient replay origins, positive mean replay uplift, and no replay harm worse than the configured tolerance. Otherwise take a **zero-borrowing local-only** path. No label from the outer target influences this decision.
5. Refit selected sparse weighted donors and target history on all eligible earlier labeled rows; make target predictions from one HGB fitted under the policy. Save row-keyed outputs.

The directional edge matrix is a *historical discovery instrument*, not a proven causal similarity map. The replay tests the **policy selection procedure** rather than retrospectively choosing whichever donor looked best on the outer target. Donor weights are derived from historical edge mean AP, normalized within the overall donor mass `rho`. This is a conservative practical policy, not a mathematically guaranteed no-regression learner.

## Experimental arms

| Model name | Input | Historical training contribution | Intended question |
| --- | --- | --- | --- |
| `hgb_local_common` | common+course | target only | common local control |
| `hgb_local_native` | common+course+target-native | target only | benefit of retained target-native fields |
| `hgb_pool_all` | common+course | target+all, volume-proportional donor masses | ordinary pooling |
| `hgb_pool_equal` | common+course | target+all, equal donor masses | equal-faculty contribution |
| `hgb_language_prior` | common+course | target+other `khoa_tieng_*` if applicable | prespecified grouping prior |
| `hgb_random_donor` | common+course | target+random eligible donor(s) | donor identity control |
| `suite32_selective_common` | common+course | target+replay-admitted weighted donor subset | borrowing effect alone |
| `suite32_selective_native` | common+course+target-native | same replay-admitted subset | primary integrated native+borrow model |
| `committee_reference` | n/a | NEVER consumed by learned models | original target-cache scores only |

All weighted policies allocate `1-rho` training loss mass to local history and `rho` to historical donors. Each donor's individual samples have normalized weights. The total additional donor-row budget is fixed by `--donor-budget` to make exposure costs comparable; actual rows may differ if a source has insufficient eligible history. Counts and ESS are always logged. Both pool-all variants use identical sampled donor rows and fixed model initialization; only donor mass allocation changes.

If `--mask-audit` is enabled, extra diagnostics fit `native_cache_nan_indicator`, `native_naive_zero_indicator`, `native_placebo_indicator` and `suite32_selective_native_cache_nan`. **These are not true source-provenance masks.** The base cache may already have lost original missingness through zero-filling. The observation ledger marks source observation status as **UNKNOWN**, not `MAR`/`MNAR`, and does not label a zero as missing. True observation-status recovery from the raw database is a separate prerequisite not supplied in the attached source archives.

## Install and run (on the AQ server)

```bash
cd /root/disk_3tb/earlywarning/30072026
unzip suite32_selective_borrowing_complete.zip
cd suite32_selective_borrowing
/root/miniconda3/envs/Huy/bin/python3 -m pip install -r requirements.txt
/root/miniconda3/envs/Huy/bin/python3 -m unittest discover -s tests -v
./run_suite32.sh preflight
./run_suite32.sh development --mask-audit
./run_suite32.sh retrospective --mask-audit
./run_suite32.sh report
```

Run separate stages with the **same hyperparameters** and output directory. The manifest guards against accidentally merging incompatible configurations. `--force` is for deliberate reruns; prefer a new output directory for different hyperparameters.

For a quick early run of *one target* (never cite as the full all-faculty benchmark):

```bash
./run_suite32.sh development --folds 1 --max-targets 1 --max-iter 30 \
  --selection-origins 4 --donor-budget 1024 --output-dir /tmp/aq_suite32_pilot
```

For the full directed 14-faculty screen this is intentionally computationally expensive (up to 182 off-diagonal source edges per selected historical origin); start with preflight and pilot to establish cache contracts and runtime. `--rho`, history thresholds, `--selection-origins`, donor budget and learner capacity should be frozen on development before running retrospective. Do not tune them to F6/F7.

To use nondefault paths:

```bash
python train_suite32.py --stage all \
  --cache-dir /path/to/results/cross_faculty_residual_moe/cache \
  --structured-dir /path/to/results/suite12_structured_cache \
  --output-dir /path/to/results/suite32_selective_borrowing
```

## Logging and result-directory contract (Suite32 runner)

**Use `./run_suite32.sh` for production and all evaluation stages.** The runner
streams Python **stdout and stderr** live to the terminal with `tee`, preserving
`[FACULTY_RESULTS]`, donor-screening/replay decisions, insights, final reports,
Python warnings, fatal exceptions and tracebacks. It fails with a non-zero
status when training fails, even though output is piped through `tee`.

With the default `AQ_ROOT=/root/disk_3tb/earlywarning/30072026`, files are:

```text
/root/disk_3tb/earlywarning/30072026/
├── log/
│   ├── suite32_selective_borrowing.log                 # all stages, append-only
│   ├── suite32_selective_borrowing_preflight.log        # most recent preflight
│   ├── suite32_selective_borrowing_development.log      # most recent dev run
│   ├── suite32_selective_borrowing_retrospective.log    # most recent F6/F7 run
│   ├── suite32_selective_borrowing_report.log           # most recent report run
│   └── suite32_selective_borrowing_runs/
│       └── <stage>_<timestamp>_<pid>.log                # distinct invocation
└── results/
    └── suite32_selective_borrowing/
        ├── audits/
        ├── selection/
        ├── insights/
        ├── predictions/
        ├── reports/
        └── run_manifest.json
```

The canonical log **appends**, so running development followed by retrospective
preserves both stages. Stage-specific logs describe the **latest invocation**
of that stage; timestamped run logs preserve each previous invocation. All
lines have a run header and an explicit success/failure footer. Results files
remain exactly where the original Suite32 code wrote them.

`AQ_ROOT`, or `--root` supplied after the runner stage, changes both Python's
root and the log/result base together. Override the log folder via
`SUITE32_LOG_DIR`, the canonical log path via `SUITE32_LOG_FILE`, and Python
interpreter via `AQ_PYTHON`. `--output-dir` overrides results but not logs.

**Important:** Calling `python train_suite32.py` directly bypasses the runner
logging; use the shell runner to capture all terminal output. For example:

```bash
./run_suite32.sh development --mask-audit
./run_suite32.sh retrospective --mask-audit
./run_suite32.sh report
tail -f /root/disk_3tb/earlywarning/30072026/log/suite32_selective_borrowing.log
```

## Diagnostic outputs

- Console: `[DATA]`, `[OBSERVATION_AUDIT]`, `[SELECTION_ORIGINS]`, `[SCREEN_BASE]`, `[TRANSFER_EDGE]`, `[POLICY_REPLAY]`, `[SELECTION_FINAL]`, `[FACULTY_RESULTS]`, `[SOURCE_MASS]`, `[FOLD_MACRO]`, `[COUNCIL_HEAD_TO_HEAD]`, `[INSIGHT TRANSFER]`, `[INSIGHT ADMISSION]`, `[INSIGHT TRAINING]`.
- `audits/`: feature-count, missingness/zero-rate, time and provenance ledger per faculty/fold.
- `selection/`: per-edge per-origin AP deltas; replay decisions; sparse donor weights and local fallbacks.
- `insights/`: per-training run source/label rows, source mass, weighted positive rate, weighted ESS/positive ESS and fit seconds.
- `predictions/`: row-keyed (`fold`,`faculty`,`row_index`,`student`,`course`,`origin`) probabilities and labels for every method.
- `reports/faculty_fold_metrics.csv`: all new methods on identical target cells.
- `reports/macro_dev.csv`, `macro_holdout.csv`, `faculty_holdout.csv`: performance summaries.
- `reports/paired_cell_comparisons.csv`: mean changes, win/loss counts, worst cell and F2@20 changes versus local, pooled and Committee.
- `reports/directed_transfer_matrix_edges.csv`, `policy_admission.csv`, `training_resource_summary.csv`, `native_vs_borrowing_decomposition.csv`: mechanism diagnostics.
- `reports/operational_frozen_dev_threshold.csv`: development-frozen per-model/faculty F2 threshold deployed retrospectively in F6/F7.
- `reference/council_13_faculties.csv`: original rounded 13-faculty Council metric reference including dev/holdout; English excluded.
- `reference/historical_model_ap_cells.csv`: Suite27 review's archived per-cell AP for models such as V2, Committee, Suite22, Suite25, Suite25 blend and specialist systems. **These are metric-level comparisons**; identical row counts do not demonstrate identity of student keys or prediction arrays. `reports/historical_model_metric_comparison.csv` explicitly flags that distinction.

All model-vs-model paired comparisons from this new run use the *same target registration rows*; archived Council/Suite25 and others do **not** qualify for row-paired bootstrap unless prediction arrays are separately recovered and keys verified.

## Scientific and implementation limitations

1. Original source database, event-publication/revision timestamps and clean observation provenance are not included; semester-based chronology is a proxy, not an availability certification.
2. Same-named features across faculties are presumed comparable when put in the common block; source semantics need a separate audit. Native-only fields of the target are explicitly unavailable (`NaN`) in donor rows even if similarly named donor fields exist.
3. Donor sampling conserves normalized *loss mass* and caps exposure; training donor policies still change effective sample size, positive support and fit time. Diagnostics make these differences measurable, not magically identical.
4. This is a reproducible framework implementation, not a claim of improved real AQ AP. The synthetic tests are contract checks only.
5. No held-out prospective period beyond previously consulted F6/F7 exists here. The framework allows zero borrowing but **cannot guarantee** future no-negative-transfer.
6. The CPU HGB learner is deliberately fixed for a data-centric audit. A later shared-neural or hierarchical learner is conditional on first observing stable directed transfer and feature-recovery value.
7. Synthetic fixtures intentionally use 8 shared raw fields, not 136; the real input dimensions are dynamically audited in `preflight` and `schema_fold_*.json`.

See `DESIGN.md` for scientific questions, stop criteria and invariants; `VALIDATION.md` for checks executed locally.
