# Track A Academic Early-Warning Experiment Suite Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build a separate runnable suite for A0–A8 that preserves the original E0 model and 276 baseline features while testing only upstream-fold-derived data and preprocessing improvements.

**Architecture:** The suite loads each expanding fold, applies an experiment-specific target filter and additive feature transformer, fits preprocessing statistics on training rows only, trains the unchanged effective-number MultiTaskNet, and evaluates development/holdout policies consistently. A8 alone uses semantic preprocessing; A0–A7 retain baseline mean imputation and z-score scaling.

**Tech Stack:** Python 3.10+, NumPy, pandas, PyTorch, pytest, tqdm.

## Global Constraints

- `IDSV` and `IDMH` remain excluded from neural-network inputs.
- A0 must reproduce the original requirement that both `target_grade10` and `target_grade4` are present.
- A1–A8 require only `target_grade10` and report rows recovered relative to A0.
- A2–A8 preserve all original E0 input columns and add derived columns; no original feature-family ablation is allowed.
- Every statistic, prior, imputation value, clipping bound, category vocabulary, and scaler parameter is fitted from the current fold's training data only.
- Development folds are 1–5 by default; holdout folds are 6–7 and use policies tuned from development predictions.

---

### Task 1: Data loading and target-filter audit

**Files:**
- Create: `track_a_suite/config.py`
- Create: `track_a_suite/data.py`
- Test: `tests/test_data.py`

**Interfaces:**
- Produces `load_fold(..., target_filter_mode)` and `audit_target_filter(...)`.

- [ ] Write tests showing A0 drops valid-grade10/missing-grade4 rows while A1 retains them.
- [ ] Implement train/test target coercion, metadata validation, and audit counts.
- [ ] Verify tests.

### Task 2: A0–A8 feature construction and preprocessing

**Files:**
- Create: `track_a_suite/experiments.py`
- Create: `track_a_suite/features.py`
- Create: `track_a_suite/preprocessing.py`
- Test: `tests/test_features.py`
- Test: `tests/test_preprocessing.py`

**Interfaces:**
- Produces `EXPERIMENTS`, `build_feature_frames(...)`, and `fit_transform_preprocessor(...)`.

- [ ] Test that every A2–A8 frame retains every A0 baseline column.
- [ ] Test depth/reliability, shrinkage, trend, ratio, compatibility, and missingness calculations.
- [ ] Test train-only priors, clipping, one-hot vocabulary, OOV handling, and finite matrices.
- [ ] Implement minimal functions and verify tests.

### Task 3: Baseline-compatible training, evaluation, CLI, and outputs

**Files:**
- Create: `track_a_suite/model.py`
- Create: `track_a_suite/metrics.py`
- Create: `track_a_suite/runner.py`
- Create: `track_a_suite/error_analysis.py`
- Create: `run_track_a.py`
- Create: `inspect_predictions.py`
- Create: `audit_target_filter.py`
- Test: `tests/test_model.py`
- Test: `tests/test_runner.py`

**Interfaces:**
- Produces development and holdout experiment comparison CSVs, per-fold predictions, manifests, diagnostics, target-filter audits, equal-budget metrics, and subgroup error reports.

- [ ] Test effective-number weighting and one-epoch prediction shapes.
- [ ] Test policy tuning uses development records for holdout evaluation.
- [ ] Implement runner and CLIs.
- [ ] Run complete unit tests, compile checks, and real-data one-epoch smoke tests.
- [ ] Package the directory as `track_a_experiment_suite.zip`.
