# Suite28 software validation

**Date:** 8 October 2026

## Passed in the packaging environment

- `python -m compileall`: PASS for package modules.
- `pytest -q tests/test_suite28_core.py`: **11/11 tests PASS**.
- Grade-bin coherence: six ordinal bins sum to 1, and first bin equals the failure marginal: PASS.
- Correct sign: increased readiness at fixed course parameters reduces predicted failure risk: PASS.
- Positive-definite 4×4 posterior covariance after Fisher update: PASS.
- Grade/failure scoring is a single coherent likelihood; missing numeric grade uses binary marginal: PASS.
- Permuting course order **within** one semester leaves the posterior update unchanged: PASS.
- All course aggregates use only events from strictly earlier semesters: PASS.
- Target outcome masking: changing **held-out target labels and grades** does not change model-side inference tensors: PASS.
- Grade/outcome identity support, missing-ID validation, stable ID normalization: PASS.
- Deterministic inference and saved checkpoint state_dict round-trip: PASS.
- Synthetic multi-fold all-stage run: PASS (F4/F5 development epoch selection; folds F4/F5/F6/F7 production; contextual B, compensatory MIRT A, primary DRRM B, full-feature HGB, interventions, uncertainty CSV, course requirement geometry, saved predictions, operational policies, 13-fac historical Council summary).
- Separately tested `--extended` branch: binary-only and K=1 control fit/selection/refit/evaluation PASS.
- Reload from **production** model and training-only course bank/preprocessor: predictions reproduce all 80 synthetic F6 target rows with maximum absolute error `3.7e-9`.

## Test commands

```bash
cd suite28_dynamic_readiness_requirement
python -m pytest -q tests/test_suite28_core.py
python tests/generate_synthetic_cache.py /tmp/suite28_fixture
OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 python train_suite28_drrm.py \
  --root /tmp/suite28_fixture \
  --out /tmp/suite28_fixture/results/suite28_test \
  --faculties khoa_cong_nghe_thong_tin,khoa_tieng_nhat \
  --folds 4,5,6,7 --dev-folds 4,5 \
  --max-epochs 1 --batch-students 20 --bootstrap 10 --device cpu
```

The generated fixtures are **fabricated** and unsuitable for model-performance claims. Synthetic Committee probabilities are artificial. Synthetic AP, accuracy, and speed are not estimates for the university dataset.

## What was not verified

1. Actual AQ canonical/structured caches are on the user's server and were not mounted into this environment. The exact selected context field count, course vocabulary, faculty support, grade completeness, and feature provenance are unverified on those production files. The preflight code fails if their contracts differ.
2. The Tesla V100-SXM2-32GB server runtime and memory peak could not be measured here. Use `run_suite28_pilot.sh` before the full run; the real model's full covariance and per-student chronological unroll have different cost from prior suites.
3. Long 7-fold/14-faculty GPU training was not run locally. Software tests cannot prove predictive AP gains, calibration, clinical/educational validity, or claims of factor interpretability.
4. Historical F6/F7 have been consulted during earlier research. They are reused comparisons, not a never-touched prospective test. A later unseen origin is needed for prospective confirmation.
5. Original 136 engineered feature-generators are external; this package reconstructs five course outcome aggregates as-of query time and checks student-semester invariance, but does not prove all upstream engineered attributes were generated using only permitted pre-enrollment information.
6. Historical Council 13-faculty summary is an archived faculty/fold table, not per-row paired predictions and cannot support student-cluster bootstrap by itself. Positional reference predictions are checked for row and label alignment, but remain subject to row-ID matching limitations in earlier artifacts.

## Validation decisions intended to prevent overclaiming

- Postfit interventions (`no_history`, `no_grade`, `no_course_id`, shuffled historical outcomes, `mean_only`) show fitted-model dependency, **not** independent retraining improvements.
- The retrained contextual-only and MIRT models, not interventions, test the two principal causal-style architectural increments.
- `suite28_fulltab_hgb` receives **all common raw feature columns** and recomputed as-of course statistics plus faculty indicators. This is a necessary information-loss control, not a hidden DRRM component.
- Primary DRRM never consumes Committee scores; there is **no blend** in the primary output.
- The ability factors remain statistical latent coordinates; course-effect identifiability and fairness/causal interpretation are **not established by passing tests**.
