# Suite5 Failure-Aware Council Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build and verify a failure-aware Council that preserves Suite4 expert diversity, optionally sharpens competence weights, and supports V2-compatible production evaluation.

**Architecture:** Reuse Suite4's immutable fold cache and four experts. Add tested boundary features, failure-aware utility targets/sample weights, prequential gate selection across uniform/symmetric/failure-aware modes and raw/aligned score spaces, then freeze a development-tuned decision policy for holdout evaluation.

**Tech Stack:** Python, NumPy, pandas, scikit-learn HGB, PyTorch relational helper, existing train_v3/Suite4/Suite3 utilities.

## Global Constraints
- Target failure is grade < 5.
- Development folds are 1-5; holdout folds are 6-7.
- Holdout labels must not tune any model, gate, calibration, threshold, or warning budget.
- RF/ET/HGB committee contract remains unchanged.
- Exact committee fallback must always exist.

---

### Task 1: Failure-aware competence utilities
**Files:** Create `suite5_common.py`; Test `tests/test_suite5.py`.
- [ ] Write tests proving rescue > neutral, lost-TP < neutral, boundary rows get larger sample weight, and temperature controls weight entropy.
- [ ] Run tests and confirm RED due to missing Suite5 functions.
- [ ] Implement `boundary_context`, `failure_aware_targets`, `competence_sample_weights`, `weight_diagnostics`, and aligned score helpers.
- [ ] Run tests and confirm GREEN.

### Task 2: Four-expert chronological Council
**Files:** Create `train_suite5.py`; Test `tests/test_suite5.py`.
- [ ] Add tests for raw/aligned score-space invariants and exact alpha=0 committee fallback.
- [ ] Reuse Suite4 forward expert generation and relational OOV behavior.
- [ ] Fit symmetric and failure-aware HGB competence regressors.
- [ ] Implement prequential selection over gate mode, score space, alpha, and temperature.
- [ ] Run focused tests.

### Task 3: Full operational metrics
**Files:** Modify `train_suite5.py`; Test `tests/test_suite5.py`.
- [ ] Add tests for frozen-threshold metric rows and pooled confusion totals.
- [ ] Tune threshold only on pooled dev final predictions using `train_v3.tune_threshold`.
- [ ] Evaluate fixed and drift-aware decisions on dev/holdout for Suite5, committee, uniform council, and symmetric Council.
- [ ] Save threshold policy and warning metrics CSVs.

### Task 4: Diagnostics, launcher, and packaging
**Files:** Create `run_suite5.sh`, `README_suite5.txt`; copy immutable dependencies and cache builder.
- [ ] Save per-fold score, policy grid, competence, weight, history-regime, and confusion diagnostics.
- [ ] Add restart-safe cache preparation and run launcher.
- [ ] Compile all Python files and check shell syntax.
- [ ] Run chronological synthetic smoke through dev and holdout.
- [ ] Verify expected artifacts and ZIP integrity.
