# Suite33 v2 — Reconstructed Offering-Context Learning (AQ)

**Status: implemented and synthetic-test validated. NO real-AQ scores are claimed.**

The previous Suite33 release refused to run without independently attested institutional origin rosters. Following the October 2026 source audits, this revision adds a **raw-database reconstruction** mode using the available CSV tables. The mode is scientifically **EXPLORATORY**: its enrollment roster is the *current exported database state*, not proven historical origin membership. It is not a valid deployable start-of-semester early-warning system. An independently attested `--mode certified` is still supported and retains the original gate.

## 1. Installation / single-command real-data run

Install in `/root/disk_3tb/earlywarning/30072026/suite33_offering_context/`, preserving the directory layout. The existing results caches remain authoritative. Requires Python >=3.10, numpy, pandas >=2.0, scikit-learn, joblib (see `requirements.txt`). On the machine where prior suites ran:

```bash
cd /root/disk_3tb/earlywarning/30072026
unzip -o suite33_reconstructed_offering_context_v2.zip
cd suite33_offering_context
python -m unittest discover -s tests -v
./run_suite33.sh
```

**Default `--mode reconstructed`** loads raw tables from `/root/disk_3tb/earlywarning/database` (the parent of `30072026` + `/database`). There is **no `roster_manifest.json` requirement** for the default exploratory experiment. One launcher runs F1–F5 then F6–F7 and summarizes all arms. All output/errors append to ONE log: `/root/disk_3tb/earlywarning/30072026/log/suite33_offering_context.log`.

Results: `/root/disk_3tb/earlywarning/30072026/results/suite33_offering_context/`. If the Python environment differs from the original one, set `AQ_PYTHON=/path/to/python3`. If needed override root via `AQ_ROOT` and raw tables via `--database-dir /path/to/database`.

### Smoke test without real data

```bash
python -m tests.make_synthetic_raw /tmp/suite33_v2_synthetic
AQ_ROOT=/tmp/suite33_v2_synthetic ./run_suite33.sh \
  --database-dir /tmp/database --max-faculties 2 --smoke-test --max-iter 10 --bootstrap 0
```

**Do not mistake any synthetic result for university performance**. No real student rows are in the package.

### Original certified mode remains available

```bash
./run_suite33.sh --mode certified \
  --roster-manifest /path/to/independently-verified/asof_roster_manifest.json
```

Certified mode requires a legitimately attested immutable as-of enrollment roster and grade-release evidence with the original CSV schema. Do not set manifest booleans to true using raw current-state tables; this would be an invalid attestation.

## 2. What the raw-data adapter does

The adapter reads exactly four existing CSV tables, *not* all 28 at once:

| Table | Used columns | Purpose |
| --- | --- | --- |
| `MKQDK.csv` | `NHHK`, `IDKQDK`, `IDSV`, `IDToHoc`, `NgayThem`, `NgayDoiNhom`, optional `IDDVPC` | Current exported registrations and source-quality diagnostics |
| `KToHoc.csv` | `NHHK`, `IDToHoc`, `IDMH`, `NgayThem` | Section-course relationship |
| `HDMNHHK.csv` | `NHHK`, `NgayBDHK` | Explicit **proxy** origin cutoff: semester start (Asia/Ho_Chi_Minh) |
| `DDTBHK.csv` | `NHHK`, `IDSV`, `DTBHK`, `NgayThem` | Latest pre-target-semester GPA whose row insertion date is strictly before the target cutoff |

The exporter performs strict key equality, rejects ambiguously multiply assigned student-course-semester keys, excludes registrations with missing section/calendar mapping from constructed context, identifies late-created and late-changed source rows, and constructs student-excluded peer summaries. It includes registrations **regardless of whether they appear in the 14-faculty numeric-grade benchmark**. Earlier GPA values are filtered by both prior semester and entry date; this DOES NOT prove the grade was published at the recorded entry date. Source hashes and detailed counts are saved.

**Crucial:** the main exploratory context retains registrations stamped *after* the semester-start cutoff, because the supplied timing audit showed ~77% of registrations in audited semesters recorded after that cutoff. Filtering them would erase most offerings. This means the context may contain post-origin membership and must NOT be treated as a valid origin-available predictor. The main experiment evaluates whether this association has *research potential*, not a deployable expected improvement.

The output contains a narrower **pre-origin membership candidate** flag requiring registration-created, section-created, and any recorded section-change time not later than the cutoff. This is an **audit/subset sensitivity only**, not an independently verified historical snapshot. In particular, group changes can erase original section assignments; cancellations can be missing; and the feature block for that target subset is still built from the full reconstructed roster. The post-origin flags are never model inputs.

`MKQDK.IDDVPC` is an **administrative source unit**, not a validated faculty. In reconstructed mode, the existing `s1_sec_faculties` feature is really *distinct source-unit count*; cross-faculty labels elsewhere are an inherited field name and MUST NOT be interpreted as confirmed cross-faculty membership. Placebo shuffles stratify by this administrative unit, not a proven faculty mapping. This limitation is recorded in logs.

We do not use `MKQDK` current grades, `DDTBHP` target-semester results, current `SSVHK.HienDienSV`, attendance, or financial/sensitive fields. `MTGDK` is deliberately NOT used to silently replace cutoff dates because audit evidence found ambiguous/multiple registration windows and dates after term starts.

## 3. What the existing canonical model does

The same Suite32-aligned cache/structured sidecars are used for the S0 baseline and all six arms:

- Cache path: `results/cross_faculty_residual_moe/cache/<faculty>_fold_1..7.joblib`
- 17-feature sidecar path: `results/suite12_structured_cache/<faculty>_fold_1..7.joblib`
- Strict 136 common raw + 17 sidecar features, `train_row_index` must match `cf_covered_mask` and finite cross-fit probabilities.
- Training supervision remains **faculty-local**, and test canonical rows/labels remain unchanged.
- Chronological folds F1–F5 development; F6–F7 repeatedly consulted retrospective diagnostic.
- No new neural router or GPU learner; fixed HGB learner for all arms.

| Arm | Feature block |
| --- | --- |
| S0 | Own-history baseline (153 features) |
| S1 | S0 + roster size, previous GPA availability, source-unit count |
| S2 | S1 + course-wide prior-GPA mean and dispersion |
| S3 | S2 + reconstructed section mean/dispersion and relative-to-course contrasts |
| S4 | S3 + own relative preparation gap and section/course percentile |
| P1 | S4 with deterministic within-course/semester/source-unit shuffled section assignments; course-wide features unchanged |

All aggregates exclude the target student. S4/P1 use the same number of features and the same fixed learner. The matched-placebo contrast is descriptive, not causal. Stronger historical Council/Suite25 results appear as *metrics-only* references, not row-paired controls.

Development-only admission/decision logic is preserved. **In reconstructed mode, even an S4 success becomes a research selection, never `promoted_for_deployment`.** No model is selected using F6/F7 outcomes.

## 4. Complete diagnostics to inspect

The single log includes `[RAW_TABLE]`, `[RECONSTRUCTION_SOURCE_AUDIT]`, `[RECONSTRUCTION_SEMESTER]`, `[RESEARCH_MODE_WARNING]`, `[DATA_GATE]`, `[FOLD_START]`, `[ROW_COVERAGE]`, `[ASOF_ROSTER_AUDIT]`, `[ORIGIN_MEMBERSHIP_SENSITIVITY]`, `[TRAINING_DIAGNOSTICS]`, `[FACULTY_RESULTS]`, `[CONTEXT_INCREMENT]`, `[COURSE_VS_SECTION]`, `[PLACEBO_MATCH]`, `[DECISION]`, `[PROMOTION_GUARD]`, `[MACRO_RESULTS]`, `[FOLD_RESULTS]`, `[FACULTY_RESULTS_MATRIX]`, `[THRESHOLD_FREEZE]`, `[ORIGIN_CANDIDATE_SUBSET]`, `[RETROSPECTIVE_HISTORICAL_COMPARISON]`, `[UNCERTAINTY]`, and `[RUN_STATUS]`.

Key files include:

- `audits/raw_roster/reconstruction_quality_by_semester.csv`: counts of late registrations, sections, group changes, previous GPA availability.
- `audits/raw_roster/reconstruction_quality_per_membership.csv`: keyed diagnostic flags **never given to models**.
- `audits/raw_roster/raw_reconstruction_audit.json`: exclusions, ambiguities, hashes and scientific warnings.
- `context/section_metadata.csv`, `context/context_manifest.json`, `context/SOURCE_WARNING.json`.
- `development/frozen_decision.json`: development-only research selection and non-deployability flag.
- `reports/development_macro.csv`, `reports/retrospective_macro.csv`, individual matrices, placebo contrasts and `operational.csv`.
- `reports/retrospective_origin_candidate_subset.csv`: selective target-subset descriptive comparison, **not** verified-peer analysis.
- `reports/scientific_status.json`: explicit exploratory status.
- `reports/all_predictions.csv`: row-keyed predictions for reproducibility.

## 5. Limits and comparability

1. Raw exports represent an unversioned current database state; loss of canceled enrollments / historical section-change assignments may create **future-selection leakage**. Exploratory positive AP gains do not establish semester-origin performance.
2. `DDTBHK.NgayThem` is a record insertion timestamp, not certified grade-release time. Cached own-history features and training labels remain on previous-suite semester-proxy availability contracts.
3. `IDToHoc` and `IDMH` are fully matched in the audits, but historical membership at `NgayBDHK` is NOT established. The 20221–20252 source audit found ~77% of registration creation dates after recorded semester start.
4. Current source `IDDVPC` is not proven student faculty. Its group-stratified placebo is only an administrative-source-unit control.
5. The original row-coverage/missingness 2×2 study is **not** performed; the cross-fit coverage audit is not a substitute. The original 10,802 excluded histories require exact reconciliation separately.
6. Archived baselines are approximate cross-run comparators; F6/F7 are reused retrospective, not untouched. Summary AP cannot establish row-paired superiority.
7. The outcomes are existing numeric-grade binary labels. No grade-zero/borderline subtypes are manufactured.
8. Results for source-excluded rows retain the original benchmark target universe and have missing reconstructed-context blocks; inspect source coverage by faculty/fold.
9. Raw source dates are interpreted as local `Asia/Ho_Chi_Minh` and normalized to UTC. If the exports use another timezone, correct the source contract before using the results.

## 6. No further input required to *start* exploratory execution

With the four named raw CSV files and the already-tested AQ canonical caches in their stated locations, `./run_suite33.sh` now runs without an invented manifest. If a data-header or canonical key assumption fails, the runner stops with a detailed traceback in the single consolidated log. Do not fabricate columns or suppress the failure.
