# Final Suite27 implementation review

Reviewed 2026-10-02 against `DESIGN.md` and `INTERFACES.md`. This was an independent, read-only review of the actual standalone files; there is no Git history to compare. Source was inspected in passes covering data/protocol, reader/grower/adaptation, selection/refit, and CLI/reporting/provenance. The only reviewer-authored package file is this report.

## Strengths

- The primary native score is exactly a faculty-weighted prevalence intercept plus hard Newton leaf contributions (`native.py:86-94`, `native.py:139-147`). Auxiliary heads, Committee and prior-suite predictions do not enter that score.
- Evidence is computed before each node's candidate scan. Completed raw ancestor predicates are carried to children; memory predicates are not fabricated as historical predicates. Faculty indicators are excluded from comparable context (`native.py:97-119`, `memory.py:173-184`).
- Each supported earlier origin owns its training-only preprocessing, vocabulary, bank and reader. Candidate accounting requires identical faculty/origin/seed cells, and deterministic tie-breaking favors the simpler recipe (`experiment.py:186-196`, `experiment.py:254-279`, `experiment.py:353-363`).
- Adaptation fixes the full hard reference and selected hard contributions, optimizes a bounded soft subforest proposal, and then reconstructs providers and regrows the complete forest. Intermediate proposals use the configured maximum tree budget; final scoring uses the selected budget (`adapt.py:44-79`, `experiment.py:333-348`, `experiment.py:423-435`).
- Reader fingerprints exclude auxiliary heads but protect retrieval weights. Reload now also verifies saved recipe/forest topology metadata and memory schema. Checkpoints contain weights and preprocessing without duplicating the history bank (`memory.py:23-29`, `experiment.py:438-448`, `experiment.py:473-494`).
- Support, paired comparisons, undefined one-class ranking metrics, bootstrap invalid draws, operational workloads, resource semantics and archive hashes are explicit. Resume verifies the complete fold inventory and every artifact hash. The shell wrapper preserves failure status and appends both output streams.

## Issues

### Critical

None found in the reviewed final source.

### Important

None unresolved after the targeted corrections below.

### Minor

None requiring a release change. The cost and temporal limitations below remain relevant when interpreting results.

## Findings corrected during review

| Finding | Final location and evidence |
|---|---|
| Baseline providers accepted supplied histories belonging to another student, or constructed under a weaker embargo. | `memory.py:110-119` now validates identity, the configured embargo and same-course metadata in bounded chunks. Foreign identities are permitted only for the explicit shuffle intervention. Both originally accepted reproductions now raise `ValueError`. |
| Disjoint fold lists alone allowed future development labels to determine an earlier holdout's operational threshold. | `evaluation.py:546-567` checks actual canonical semesters; `train_suite27.py:197-220` and `train_suite27.py:412` check before fitting or result writing. Development semester 63 for holdout semester 60 now raises `ValueError`. Missing-semester legacy evaluation has an explicitly documented numeric-fold fallback. |
| Reload recorded runtime provenance without enforcing it. | `predict_suite27.py:38-42` now rejects a mismatched runtime before output. The reviewer changed the saved sklearn version to `0.fake` and confirmed rejection with no prediction file created. Runtime fingerprints now include scipy and joblib (`runtime.py:83-88`). |
| Baseline reload accepted mismatched recipe path strength and forest metadata. | `experiment.py:475-494` now checks tree count, depth, path strength and memory schema. The real experiment regressions cover tampered strength/budget/depth/schema and retain intentional `no_path` intervention support after a proper load. |
| Zero-dispersion differentiation depended on the runtime's clamp boundary behavior. | `memory.py:43-45` now uses an explicit safe variance branch. Empty, single-event and constant-performance gradients are finite in the reviewed Torch 2.14.1 CPU runtime. A NaN failure on an older Torch version was not reproduced here; this is portability hardening. |
| Origin cohort support, zero-row faculty visibility and cost/archive records needed clarification. | `protocol.py:106-145` implements 2,000 total semester rows and 30 validation/earlier-training rows per faculty. Zero-row target support is retained without dummy predictions (`runtime.py:120-129`). Host RSS is labeled a lifetime process peak and checkpoint bytes are saved (`train_suite27.py:272-276`, `train_suite27.py:357-358`). Collector hashes match the exact archived payloads (`collect_suite27_reports.py:33-44`). |

## Independent verification

The following targeted runs passed after their relevant corrections:

| Command | Result |
|---|---|
| `python -m pytest tests/test_memory.py tests/test_adapt.py tests/test_protocol.py -q` | 22 tests and 9 subtests passed. |
| `python -m pytest tests/test_evaluation.py -q` | 23 tests and 10 subtests passed. |
| `python -m pytest tests/test_experiment.py tests/test_end_to_end.py::CLIUtilityTests -q` | 40 tests passed on frozen source. |

Earlier review runs also passed the native grower and cache-contract suites. Additional small probes verified rejection of foreign-student histories, embargo violations and future development semesters; runtime mismatch rejection before output; and SHA256/byte-count equality for every member in a collected report archive while excluding weights and default predictions. No source was modified by the reviewer and no heavy scale probe was run.

The coordinating agent's fresh full-suite and extracted-package smoke checks are the final release gate; this report does not claim their results in advance.

## Design limitations, not implementation bugs

- Canonical raw features and preserved covered course metadata remain external availability/provenance assumptions. Mapping and file hashes cannot certify their original generation process. Grade-release timestamps are absent. Memory embargo excludes retrieved events and priors, and does not regenerate those external current features.
- Earlier-only histories/priors do not turn the origin-wide supervised reader and forest into an online or causally identified learner. Independent earlier-origin fitting protects validation supervision; historical F6/F7 research reuse remains acknowledged.
- Histogram thresholds, greedy growth, root-evidence memory grids and the partial soft proposal are approximations. Hard-routing boundaries can differ across CPU/GPU. The proposal can be rejected and simpler variants can win.
- Storage avoids a persistent `N × history × raw_feature` tensor and a duplicated checkpoint bank, but still retains history indices/metadata and compact root caches scaling with rows × retained history. Current-only variants also construct history for support diagnostics. Host RSS records are process-lifetime maxima, not isolated per-model CPU peaks. Production-scale runtime, memory and predictive performance require actual runs.

## Declined to judge

- Academic AP improvement, convergence or causal benefit: this source review and synthetic tests cannot establish those claims.
- An exhaustive reconstruction of every external cache feature: original cache-generation sources and release timestamps are outside the supplied package.
- Production GPU reproducibility and maximum-scale cost: no such run was performed in this review.
- The historical Suite26 numerical comparison itself: its supplied evidence is context for the design, not a new experiment reproduced by this software review.

## Assessment

**Ready for release after the final full-suite and extracted-package smoke gates pass.** The final implementation follows the approved native memory-conditioned boosting design, and the reproduced identity, embargo, operational chronology and reload-consistency findings have been corrected. No unresolved critical or important implementation finding remains in this review. This is a software-contract assessment, not an academic performance claim.
