# Suite34 — Information-Preserving Multi-View Conditional Learning

Complete standalone PyTorch experiment for AQ course-failure prediction.
Read DESIGN_FINAL.md for the finalized contract, DESIGN_ORIGINAL.md for the
supplied specification, PROTOCOL.md for the frozen scientific rules, and
VALIDATION.md for the exact software-validation evidence.

## Version 1.0.1: fix repeated CUDA warnings

The launcher and supported Python entry points now configure
`CUBLAS_WORKSPACE_CONFIG=:4096:8` before importing PyTorch or starting CUDA.
An explicitly configured `:16:8` is preserved. Invalid/empty values fail with
an actionable message. The environment check exercises deterministic CUDA
matrix-multiplication forward **and backward**, and prints a single
`REPRODUCIBILITY` event. CPU fallback is explicitly marked `NOT_RUN_CPU` for
that hardware-only check. Other warnings and all research tables remain visible.

The new execution setting is recorded in run metadata and the signature.
For an upgrade from 1.0.0, keep the old results and start a separate run:

    bash run_suite34.sh \
        --output-dir /root/disk_3tb/earlywarning/30072026/results/suite34_ipmvcl_v1_0_1 \
        --log-file /root/disk_3tb/earlywarning/30072026/log/suite34_ipmvcl_v1_0_1.log

Retain your original optional flags, such as `--amp` or `--include-a3`, if used.
After that new run starts, resume **that same command** with `--resume`.
Do not use this version to resume 1.0.0 cells; the strict signature guard
deliberately rejects mixed execution configurations. Updating files or exporting
the variable in a later shell does not alter an already running Python process.
Apply the upgrade between processes. See PATCH_NOTES.md for scope and limitations.

Validation NaNs in refit epoch records mean no separate validation set exists
during the complete-history refit. They are not nonfinite training losses.

## What is implemented

- Full-input residual MLP A0 and parameter-matched A0M.
- Additive semantic views A1; identical-parameter multiplicative views A2.
- Label-free, dimension-matched random-feature-bundle placebo P1.
- Optional A3 auxiliary view heads; only the joint logit is used at inference.
- Chronological validation, training-only preprocessing, fresh full-history refit.
- Three matched seeds; all fourteen faculties and seven folds by default.
- New matched HGB and canonical cache-reference controls, plus historical
  support-qualified benchmarks. HGB is a comparison control within this DL study.
- Complete epoch, outcome, mechanism, warning-policy, subgroup, uncertainty and
  comparison tables in one log. Local prediction identifiers are not printed.
- Portable checkpoints with full-row CPU replay and completed-cell resume.

The full nonlinear MLP and A1 can learn interactions implicitly. The primary
question is the incremental value of the explicit semantic product operation.
All S/C/H views are derived from one registration's existing feature channels;
training and inference both receive the same available channels.

## Required caches — direct answer

Use these existing directories under AQ_ROOT:

    results/cross_faculty_residual_moe/cache/<faculty>_fold_<1..7>.joblib
    results/suite12_structured_cache/<faculty>_fold_<1..7>.joblib

No newer experiment cache, external pretrained model or new database export is
required. Source caches are read-only. No prior Python experiment package is
imported at runtime.

The adapter follows the inspected Suite32 source: history uses cf_covered_mask
AND finite cf_probabilities across members; sidecar train_row_index must equal
the exact retained canonical positions. Sidecars provide train.course_target
and target.course_target, twelve static plus five historical course channels.
The main input is the global common raw schema plus those 17 channels, ordinarily
136+17. Faculty-native extras are outside this matched common-input study.

The exact live schema is discovered across the entire source population, even
for a one-faculty/one-fold pilot. Do not remove other canonical files to make a
pilot. Preflight checks every faculty's seven folds and row/feature contracts.

Actual joblibs were not available in the authoring environment. Compatibility is
grounded in inspected loader source and synthetic replicas; the real-cache
preflight and smoke run below remain required on your server. Header-derived
feature catalogs are not falsely presented as the exact cached feature order.
When target sidecar indices are absent, target order inherits the inspected
source contract and is shape-checked; independent raw-DB identity reconciliation
is unavailable. Cached source imputation and semester-origin availability remain
upstream provenance limits.

## Installation

Use Python 3.10+ and your existing CUDA-compatible PyTorch 2.3+ environment.
Keep the GPU build appropriate for your server. The environment check exercises
the NumPy bridge and a CUDA kernel when CUDA is available.

    cd /root/disk_3tb/earlywarning/30072026
    unzip -o suite34_ipmvcl_complete.zip
    cd suite34_ipmvcl
    python -m pip install -r requirements.txt
    python check_environment.py

The package also runs on CPU. CUDA/FP16 training has to be validated on your
server; CPU integration timing is not a full-study runtime estimate.

## Recommended first run

Read-only source preflight, without model fitting:

    bash run_suite34.sh --preflight-only

Real-cache compatibility smoke run:

    bash run_suite34.sh --smoke-test --folds 1 --faculties all --seeds 29 \
        --epochs 2 --min-epochs 1 --patience 1 --bootstrap 0

Smoke uses separate results and log names ending in _smoke. It retains the
selected target lists and all principal arms; it cannot produce primary
research promotion. Epoch count two is a compatibility check, not a performance
assessment.

Complete study:

    bash run_suite34.sh

Optional GPU mixed precision, applied equally across arms:

    bash run_suite34.sh --amp

Choose AMP before starting the study and retain the same setting on resume.
Default A3 is off. Prespecify it before execution with --include-a3.

Alternate paths/interpreter:

    AQ_ROOT=/custom/project PYTHON=/path/to/python bash run_suite34.sh
    bash run_suite34.sh --root /custom/project \
        --cache-dir /path/to/canonical --structured-dir /path/to/sidecars

Flags accept comma-separated or space-separated lists, for example --seeds
29,129,229 or --folds 1 2 3. --faculties takes exact cache filename prefixes.
Inspect all options with bash run_suite34.sh --help.

## Resume and repeat reports

Rerun the ORIGINAL command with --resume, retaining every material setting:

    bash run_suite34.sh --resume
    bash run_suite34.sh --amp --resume

Use only the command matching your initial AMP choice. The source/configuration/
software/manifest signature and completed artifacts must still match. Changes
require a fresh --output-dir. Interrupted cells restart from their beginning;
completed cells are reused. There is no mid-epoch optimizer-state continuation.

Regenerate a complete report from an already completed unchanged run:

    bash run_suite34.sh --resume --report-only

The launcher normally runs the complete unit/integration suite first. After
initial validation, --skip-tests or S34_SKIP_TESTS=1 omits that setup cost.
No training metrics or diagnostic tables are omitted merely because cells resume:
the final log reconstructs complete stored epoch/result tables and hashed
selection/refit preprocessing audits, chronological cutoffs and fallback decisions.

## Single log and results

Default primary log:

    /root/disk_3tb/earlywarning/30072026/log/suite34_ipmvcl.log

Default primary results:

    /root/disk_3tb/earlywarning/30072026/results/suite34_ipmvcl/

The launcher backs up an existing active log before replacing it and propagates
Python/test failures through pipefail. Send the final active log for review.
Malformed CLI/configuration errors detected before log-path setup appear in the
terminal; environment, test, data-preflight, training and reporting failures are
captured in the active log.
It includes full faculty×fold×arm×seed metrics, per-seed/faculty/fold summaries,
AP/AUC/Brier/ECE, separate frozen F1/F2 thresholds, confusion counts and alert
loads, fixed-budget performance, every epoch, parameter counts, runtime, paired
effects, bootstrap intervals, views/interactions, rescues/harms, history strata,
the development decision and an explicit continue/refine/close interpretation.
No table relies on a truncated CSV preview.

Key machine-readable outputs:

- config.json, environment.json, source_manifest.json, feature_manifest.json.
- all_metrics.csv and all_predictions.csv.
- development_selection.json, threshold_policies.csv, operational_metrics.csv.
- training_curves.csv, paired_contrasts.csv, faculty_summary.csv, fold_summary.csv.
- interaction_ablations.csv, view_diagnostics.csv, history_strata.csv.
- rescues_and_harms.csv, historical_comparisons.csv, uncertainty.csv, runtimes.csv.
- report_manifest.json and all additional full tables as CSV.
- cells/<faculty>__f<fold>__<arm>__s<seed>/ with atomic per-cell evidence.
- checkpoints/<same-key>.pt hard links to completed cell checkpoints.

Source joblibs and generated checkpoints must come from a trusted local project;
they are Python serialization files. Predictions retain canonical local IDs for
alignment, while aggregate console tables suppress those identifiers.

## Checkpoint replay

Every trained cell is replayed on CPU over its entire target matrix before its
completion marker is committed (atol/rtol 2e-5). Checkpoints contain feature order,
preprocessing, view mapping, weights, selected epoch, seed, source identities,
architecture and software versions.

For an independent replay from the same canonical files:

    python s34_checkpoint.py --checkpoint /path/to/checkpoint.pt \
        --canonical /path/to/<faculty>_fold_1.joblib \
        --structured /path/to/sidecar/<faculty>_fold_1.joblib \
        --faculty <faculty> --fold 1 --output /tmp/replayed_scores.csv

Replay refuses source or schema mismatches. This command does not train.

## Scientific interpretation

Promotion is A2-specific and frozen using development evidence. F6 and F7 are
retrospective; their results never revise that decision or operating thresholds.
F7 can legitimately use earlier F6 labels inside its chronological history,
while its F1/F2 thresholds remain identical to F6's F1–F5-frozen policies.

The matched HGB is a new control with inspected Suite32 hyperparameters; it is
not an exact Suite33 reproduction. CacheReference uses qte or the mean of pte
members from the same canonical container. Its Committee identity is not
independently certified by the unavailable original builder.

Archived Suite22/Suite25/Committee/Council results are labeled metrics-only and
population-qualified. An IT-only or thirteen-faculty score is never treated as a
fourteen-faculty matched comparison.

Interaction/branch zeroing is sensitivity analysis. Full-input information remains
and joint LayerNorm changes other normalized channels. Nonzero gradients or large
latent norms do not establish utility. Bootstrap intervals concern observed
faculty clusters/fixed periods, not new semesters or independent seeds.

## Local software validation and efficiency

    python -m unittest discover -s tests -v
    bash run_suite34.sh --synthetic --root /tmp/suite34_fixture \
        --epochs 1 --min-epochs 1 --threads 1 --bootstrap 10 --include-a3

Synthetic data is explicitly marked and isolated; generation refuses an existing
non-synthetic cache. Test fixtures exercise two faculties, all seven folds, every
principal control and optional A3, full reports, replay, resume and failure gates.
Synthetic scores are not AQ results.

The primary study has 1,470 neural model cells with selection and refit phases
(2,940 training phases), plus 294 HGB fits and reference/diagnostic passes.
Sequential fitting bounds device memory. Source hashes and one-cell-at-a-time
loading add read overhead. Predictions are written incrementally; the consolidated
prediction file is streamed rather than assembled in RAM.

--diagnostics none can omit expensive sensitivity passes in a separately declared
run, --no-save-checkpoints omits replayable models, and --skip-hgb omits that control.
These choices are logged and change the material signature. Default full
diagnostics/checkpoints/HGB are recommended for the principal study.

No real AQ improvement or GPU-validation claim accompanies this software package.
