# Suite8 True Specialist Discovery Design

## Goal
Test whether the residual-expert collapse observed in Suite5/Suite6/Suite7 can be reduced by giving every residual expert a distinct learned view of the shared-private representation while keeping the Committee, faculty adapter, router target, chronology, sampling, and operational evaluation unchanged.

## Empirical motivation
Suite6 target-only training did not create meaningful expert specialization: holdout router entropy stayed near 1.0 and mean pairwise residual correlations remained approximately 0.99 across IT, Business/Tourism, German, and International Studies. Suite7 faculty adapters improved AP breadth relative to Suite5 Shared, but residual correlations remained approximately 0.99-0.998. Therefore the next controlled intervention targets expert collapse rather than sharing/private adaptation.

## Baseline held fixed
Suite8 starts from Suite7:

    [x, faculty embedding] -> shared encoder h
    h_f = h + A_f(h)
    shared router(h_f)

The following remain unchanged from Suite7:
- faculty-specific frozen Committee and canonical cache;
- exact Suite5 136-common-raw-feature schema and 154 final inputs;
- faculty-private 128->16->128 adapter with normal down projection and zero up projection;
- four residual experts and explicit NULL route;
- bounded residual correction and Committee anchoring;
- BCE prediction objective, capability-target router loss, routing mutual-information term, residual-magnitude penalty;
- faculty-balanced/effective-number sampling;
- chronological macro-faculty validation, refit protocol, dev folds 1-5 and holdout folds 6-7;
- fixed and drift-aware operational evaluation.

## Learned expert views
After Suite7 constructs all shared and faculty-adapter modules, Suite8 adds a trainable ownership-logit matrix:

    Omega in R^(K x D)

where K is the number of residual experts and D is the adapted representation dimension.

For each latent dimension d, expert ownership is normalized across experts:

    alpha[k,d] = softmax(Omega[:,d] / T_view)[k]

and expert-specific multiplicative scales are:

    scale[k,d] = (1 - s) + s * K * alpha[k,d]

where s is `view_strength` in [0,1]. Uniform ownership gives alpha=1/K and therefore scale=1 exactly. Consequently the learned-view mechanism is an identity transformation when ownership is uniform.

Each residual expert receives its own view:

    h_k = h_f * scale[k]
    delta_k = Expert_k(h_k)

The router continues to consume the unmodified adapted representation h_f. This keeps routing context common while making expert information pathways different.

Ownership logits are initialized with a very small zero-mean Gaussian perturbation (`view_init_std=0.01`) after all Suite7-equivalent modules have been constructed. This preserves Suite7 initialization of the encoder, adapter, experts, and router. Residual output layers remain zero initialized, so the entire Suite8 model still predicts exactly the Committee at initialization.

## View-specialization objective
Two auxiliary losses are added.

### Ownership entropy
For every latent dimension, normalized entropy across experts is minimized:

    L_view_entropy = mean_d H(alpha[:,d]) / log(K)

Low entropy means a latent dimension is preferentially owned by a subset of experts instead of being used identically by all experts.

### Ownership load balance
To prevent one expert from owning every latent dimension:

    load_k = mean_d alpha[k,d]
    L_view_balance = mean_k (load_k - 1/K)^2

The complete Suite8 objective is the Suite7 objective plus:

    lambda_view_entropy * L_view_entropy
    + lambda_view_balance * L_view_balance

The default coefficients are deliberately moderate. No direct penalty is applied to residual-output correlation in this experiment: the causal question is whether distinct learned information views are sufficient to produce distinct useful residual functions. Residual decorrelation remains a future intervention if view specialization alone fails.

## Diagnostics
Suite8 must measure whether specialization actually emerges, not only final AP.

For every fold and faculty/group (`all`, `failures`, `normals`, `history0`, `history1_2`, `history3plus`), save and print:
- NULL weight, router normalized entropy, top routing weight;
- mean residual and correction magnitude;
- pairwise residual-expert correlation;
- counterfactual utility win rate for NULL and every expert;
- mean realized utility of each expert.

Per fold, save expert-view diagnostics:
- normalized ownership entropy;
- ownership load per expert;
- ownership-load imbalance;
- pairwise cosine similarity between ownership vectors;
- pairwise cosine similarity between actual expert views on validation/target rows;
- top owned latent dimensions for each expert and their ownership values;
- full K x D ownership and scale matrix.

## Official comparisons
Suite8 reads completed Suite5 and Suite7 predictions as read-only references and reports:

    Committee
    Suite5 Shared Residual-MoE
    Suite7 Shared+Faculty Adapter
    Suite8 True-Specialist Shared+Private Residual-MoE

For each faculty and pooled holdout:
- AP and AUC;
- F2 at exact 10%, 15%, 20%, 25% warning budgets;
- development-frozen fixed-threshold precision/recall/F1/F2/rate/confusion matrix;
- drift-aware precision/recall/F1/F2/rate/confusion matrix.

Cross-faculty summaries report mean, median, worst delta, and win rate for Suite8 minus Suite7 on AP, AUC, F2@20, fixed F2, and drift F2.

## Main success criterion
Performance improvement alone is insufficient. A successful Experiment C should show both:
1. materially lower residual-expert correlation and/or lower expert-view similarity than Suite7; and
2. no material degradation in macro-faculty AP/F2, ideally a positive median Suite8-minus-Suite7 AP and warning-budget F2.

If ownership becomes diverse but residual outputs remain ~0.99 correlated, the bottleneck is downstream expert objective/capability learning rather than representation collapse. If residual correlation drops but AP/F2 deteriorate, forced diversity is creating different but unhelpful experts. If both correlation drops and performance improves, the true-specialist hypothesis is supported.
