# Suite27 CLI and reporting restoration

Restored `evaluation.py`, `runtime.py`, `train_suite27.py`, `predict_suite27.py`, `run_suite27.sh`, `collect_suite27_reports.py`, `smoke_test.sh`, `requirements.txt`, and evaluation/end-to-end tests. Evaluation retains the vetted Suite26 keyed reference joining, exact development-only F2 thresholds, explicit one-class/missing-score support, paired shared-row comparisons, and paired student-cluster bootstrap. The prespecified primary is `suite27_mcgb`; the component ladder is native versus stock, deterministic versus native, global versus deterministic, path versus global, and mcgb versus path. Previous MINT score aliases include `p_suite26_mint`, `p_suite26_memory`, and `p_suite26_tree` plus the original Suite26 column names.

The production root defaults to `/root/disk_3tb/earlywarning/30072026`; results default to `results/suite27_memory_conditioned_boosting`. The CLI exposes every experiment configuration field, folds/faculties, seeds, reference directories, thread/device control, bootstrap, and every supported-origin threshold. Origin minimum rows apply to total semester support; minimum faculty rows apply to each validation faculty and its earlier training bank. Common numeric features are intersected across every requested fold and faculty. Unsupported origins print the complete faculty/semester audit, and production runs also save `origin_support_failure.csv`. Preflight validates inputs, schema, preprocessing, target histories, and origin support without creating result files. Both preflight and production inspect canonical target times before any model fitting or result writing: the latest configured development semester must strictly precede the earliest configured holdout semester. Evaluation enforces this again before applying thresholds; legacy metric-only frames lacking semester explicitly fall back to numeric fold order.

Startup and `target_support.csv` include every requested faculty with rows, positives, negatives and student counts. Faculties with zero target rows remain visible in the data audit and printed per-seed/reload metrics with empty support. No synthetic prediction rows are inserted; report cells continue to describe observed target rows.

Each manifest records configuration, source `.py`/`.sh` hashes, package/runtime versions (including SciPy and joblib), feature schema, every canonical/structured input file hash, and reference input CSV hashes. Configuration, source, runtime, schema, and input collections also have SHA256 digests. Resume refuses any manifest mismatch. A fold is reusable only after a complete marker verifies the exact artifact inventory and every SHA256; an incomplete fold is recomputed in full. Checkpoints do not duplicate the bank. Prediction reload checks original runtime and source fingerprints, canonical file hashes, and schema before loading the reader/forest pair. Exact replay requires the original Python/package/runtime environment; a device override within that environment is permitted.

Every variant/seed has `model_<variant>_seed_<seed>.pt` and keyed prediction CSVs. Published main scores average independent seed scores; per-row seed standard deviation and complete per-seed faculty/fold metrics remain available. Selection reader/adaptation/forest logs are preserved alongside refit records. Fold diagnostics retain tree structures, memory/intervention/regime metrics, and refit/selection time, checkpoint bytes, peak process RSS and CUDA memory. Host RSS is the process lifetime peak, explicitly identified as `lifetime_process_peak`; it is not an isolated per-variant allocation. All metric, operational confusion/budget, seed, and bootstrap tables print their complete rows and columns. Empty student workload outputs retain a stable header; populated workload tables aggregate course warnings into student counts for fixed development thresholds and all reported warning budgets.

The shell wrapper runs unbuffered Python, uses `pipefail`, appends both output streams through `tee`, records start/end timestamps and exit status, and honors `PYTHON`, `SUITE27_LOG_DIR`, and `SUITE27_LOG_FILE`. The collector includes CSV/JSON/log/text provenance and reports, excludes checkpoint weights, and excludes keyed prediction files unless requested. `COLLECTION_MANIFEST.json` records the archive method, explicit prediction inclusion and weight exclusion, and the relative path/byte count/SHA256 of every included artifact, calculated from the exact archived payload. The packaged smoke script creates 16-student canonical fixtures for folds 1/2, runs all six comparisons with 2/4 trees and one-epoch readers, preflights, reloads all six checkpoints, resumes, collects reports, and verifies keyed score identity. Fixture metrics verify software only and are not evidence of academic improvement.

Verification completed during restoration: evaluation tests passed 23/23 after chronology additions; CLI utility tests passed 3/3, including prior MINT aliases, cross-fold feature intersection and zero-row faculty support. The preceding all-six, two-seed end-to-end run passed 11/11 with 24 exact keyed checkpoint reloads, resume/configuration/input/source/artifact mismatch checks, collector, logger and full console/resource tables. The preceding packaged smoke passed all six reloads, safe resume and collection against the corrected total-origin support protocol. Subsequent regressions extend those checks to runtime mismatch refusal, collector payload hashes, checkpoint sizes and CLI chronology rejection; the parent restoration runs the final complete suite and extracted-package smoke against frozen sources.

Tested runtime: Python 3.12.14, NumPy 2.3.5, pandas 2.2.3, SciPy 1.17.0, scikit-learn 1.8.0, joblib 1.5.3 and PyTorch 2.14.1+cpu. Requirement bounds also admit the user's PyTorch 2.7.1/scikit-learn 1.7 environment. Exact runtime-specific results are recorded in each run manifest rather than assumed across versions.
