# CNN + GRU causal streaming solver for Track 03 (Sleep Onset)

A spec and a tested reference implementation of a **causal CNN + GRU regressor** that predicts the seconds remaining until the first N2 epoch, written as a Benchopt solver for `tracks/sleep_onset/` of the 2026-competition repository. Section 7 is the complete solver file: copy it to `tracks/sleep_onset/solvers/cnn_gru.py`.

## 1. In one paragraph

Each 5 s window (4 channels x 640 samples, raw microvolts, 128 Hz) goes through a **window encoder**: a 1-D CNN on the raw waveform plus a small MLP on the log power spectrum, fused into one 96-d embedding. A **unidirectional GRU** carries a recording-long state over the successive embeddings, and a small **head** maps the GRU output to seconds, `600 * sigmoid(.)`, so the prediction is always in [0, 600]. The model is causal (nothing but past windows can influence a prediction), has **318,945 parameters**, trains in minutes on one GPU, and is trained with a loss that equals the competition metric (per-recording W-bMAE).

```
window (4, 640) uV
   |  remove per-window mean, divide by 40 uV, clamp to +-10
   +--> CNN: 4 x [Conv1d, GroupNorm(8), GELU], stride 2 each --> mean+max over time --> 256
   +--> STFT (n_fft 256, hop 64) --> power, mean over frames --> bins 0-45 Hz --> log
   |        --> MLP 364 -> 128 -> 64                                              --> 64
   v
 concat (320) --> Linear 320 -> 96 + GELU                          window embedding (96)
   |
   |   ... one embedding per 5 s window, in time order ...
   v
 Dropout 0.1 --> GRU(96 -> 128, 1 layer, unidirectional, state kept between windows)
   v
 Linear 128 -> 64, GELU, Linear 64 -> 1 --> 600 * sigmoid --> seconds to first N2
```

## 2. The contract it satisfies

| Item | Value |
|---|---|
| Input to `predict(X)` | tensor `(B, 4, 640)`, float32, **microvolts**, 128 Hz, unfiltered, on `meta["device"]`; channel order TP9, AF7, AF8, TP10 |
| Output | tensor `(B,)`, float, seconds in [0, 600] |
| Evaluation | streamed: one window per call (`B = 1`), in time order, from recording start up to first N2 |
| Per recording | the platform runs `copy.deepcopy(model)` and calls `reset_state()` first; state may be kept inside a recording and nothing crosses to the next one |
| Files in the ZIP | `submission.py` (a copy of the solver file) and `weights.pt` (state dict + architecture config), both at the ZIP root |
| Imports | only `torch`, `numpy` and `benchmark_utils.base_solver`, all present in the worker image |
| Metadata used | `meta["n_chans"]`, `meta["sfreq"]`, `meta["device"]`, `meta["submission_dir"]` |

The model must not use recording length, any whole-recording statistic, or future windows. Every public Muse recording ends exactly 300 s after first N2, so recording length *is* the label.

## 3. Architecture in detail

Shapes are for one window; `N` is the number of windows (`B * L` inside the GRU path).

### 3.1 Window encoder (`WindowEncoder`)

| Step | Operation | Output shape | Params |
|---|---|---|---|
| 0 | `x - mean(x, time)` per channel, `x / 40`, `clamp(-10, 10)` | `(N, 4, 640)` | 0 |
| CNN 1 | Conv1d 4 -> 32, k 15, stride 2, pad 7, GroupNorm(8), GELU | `(N, 32, 320)` | 2,016 |
| CNN 2 | Conv1d 32 -> 64, k 9, stride 2, pad 4, GroupNorm(8), GELU | `(N, 64, 160)` | 18,624 |
| CNN 3 | Conv1d 64 -> 96, k 9, stride 2, pad 4, GroupNorm(8), GELU | `(N, 96, 80)` | 55,584 |
| CNN 4 | Conv1d 96 -> 128, k 5, stride 2, pad 2, GroupNorm(8), GELU | `(N, 128, 40)` | 61,824 |
| pool | concat(mean over time, max over time) | `(N, 256)` | 0 |
| STFT | per channel on the mean-removed signal: n_fft 256 (2 s), hop 64, Hann, `center=False` -> 7 frames; power; mean over frames | `(N*4, 129)` | 0 |
| band-limit + log | keep the first 91 bins (0-45 Hz, 0.5 Hz resolution), `log(power + 1e-3)`, reshape | `(N, 364)` | 0 |
| spectral MLP | Linear 364 -> 128, GELU, Linear 128 -> 64, GELU | `(N, 64)` | 54,976 |
| fuse | concat(256, 64) -> Linear 320 -> 96, GELU | `(N, 96)` | 30,816 |

The encoder is 223,840 parameters. All normalisation is per window (GroupNorm, per-window mean removal): **no BatchNorm, no running statistics, no dependence on batch size**, which is what makes `B = 1` streaming safe.

### 3.2 Temporal model and head (`CausalSleepNet`)

| Step | Operation | Output shape | Params |
|---|---|---|---|
| dropout | `Dropout(0.1)` on embeddings (training only) | `(B, L, 96)` | 0 |
| GRU | `nn.GRU(96, 128, num_layers=1, batch_first=True)`, unidirectional | `(B, L, 128)` | 86,784 |
| head | Linear 128 -> 64, GELU, Linear 64 -> 1 | `(B, L, 1)` | 8,321 |
| output | `600 * sigmoid(.)`, squeeze | `(B, L)` seconds | 0 |

Total: **318,945** parameters (CNN 138,048 + spectral MLP 54,976 + fuse 30,816 + GRU 86,784 + head 8,321).

### 3.3 Why these choices

- **Raw-waveform CNN and log-spectrum MLP side by side.** The spectrum carries the band-power changes that mark the approach to N2 (falling beta/gamma, falling alpha, rising slow activity); the CNN can additionally pick up waveform shape and artifacts. Both are cheap.
- **Fixed scale (40 uV) instead of fitted normalisation.** Nothing is fitted on the training set, so there is no train/test normalisation mismatch and nothing recording-level (hence nothing non-causal).
- **Per-window mean removal** drops any slow offset/drift inside a window. Consequence: information that lives *only* in the DC level is invisible to the model (this is why `Simulated`, whose signal is a DC drift, gives an almost constant prediction; see section 9).
- **GRU, unidirectional.** Sleep onset is a gradual process, so a state accumulated since the start of the recording is informative, and a unidirectional recurrence is exactly the "model may keep state within a recording" the contract allows. A bidirectional layer would read the future and is not allowed.
- **Sigmoid output scaled to 600.** The target is capped at 600 s; the head cannot leave the valid range.

## 4. Streaming semantics

- `predict(X)` reshapes `X` to `(B, 1, C, T)`, runs the encoder and one GRU step with the stored state `self._h` (shape `(1, B, 128)`), stores the new state, returns `(B,)`.
- `reset_state()` sets `self._h = None` (and re-flattens the GRU weights, which a `deepcopy` leaves non-contiguous). The platform calls it on every fresh copy; the state therefore never crosses a recording.
- The model keeps its state as an attribute, so `copy.deepcopy(model)` is enough to get an independent stream.
- Equivalence check performed: feeding 40 windows one by one through `predict` gives the same outputs as one forward pass over the 40-window sequence (max abs difference 6e-5 s on GPU, float32 noise). Training on whole sequences is therefore consistent with streaming inference.
- Cost: about 1.5 ms per window on a V100 and about 6 ms on CPU, so a 6,000-window test set takes well under a minute of model time against the 60-minute limit (data loading excluded).

## 5. Training recipe

| Item | Value |
|---|---|
| Training unit | one **whole recording** = the time-ordered sequence of its 5 s windows from recording start to first N2 |
| Batch | 16 recordings, right-padded with zeros to the longest; padded steps have weight 0 |
| Loss | per-recording W-bMAE, averaged over the recordings of the batch (details below), in seconds, divided by 100 before backward |
| Optimiser | AdamW, lr 2e-3, weight decay 1e-2 |
| Schedule | OneCycleLR, max lr 2e-3, 10% warm-up, total steps = epochs x batches per epoch |
| Epochs | 20 (last epoch kept; no early stopping in the reference) |
| Precision | fp16 autocast + GradScaler on CUDA; fp32 on CPU |
| Gradient clipping | global norm 1.0 |
| Regularisation | dropout 0.1 on the embeddings; weight decay |
| Seed | `torch.manual_seed(0)`, numpy `default_rng(0)` for the recording order |
| Option `random_start` | if True, every step drops a random number of leading windows from each recording (keeping at least 12 = 60 s) so the state cannot learn the recording clock |

**The loss is the metric.** For one recording with target vector `y` (seconds, capped at 600), each window gets the weight

```
w_t = bin_weight(y_t) / sum(bin_weight over the bins present in this recording) / n_windows_in_that_bin
```

with bins `[0,40) [40,90) [90,300) [300,600]` and weights `10 5 3 1`. These weights sum to 1 per recording, so `sum_t w_t * |pred_t - y_t|` is exactly that recording's W-bMAE (empty bins are dropped, as the scorer does). Example: a recording with 8 / 10 / 40 / 80 windows in the four bins gives weights `10/19/8`, `5/19/10`, `3/19/40`, `1/19/80`. Using this loss means the optimiser spends its effort where the metric does: one window of the 0-40 s bin weighs roughly 100 windows of the 300-600 s bin.

**Why `fit` does not iterate the train loader directly.** In this repo the train loader is shuffled and bin-balanced (`RegressionBinSampler`, sampling with replacement), so it neither gives sequences nor covers every window once. The solver instead reads `train_loader.dataset`, which returns `(X, y, info)` per window with `info["record_id"]` and `info["onset"]`, groups windows by recording and sorts them by onset. If a dataset has no `record_id`/`onset` (the `Simulated` train split) it falls back to a stateless window mode that only exercises the code path.

## 6. Reference results and what they do and do not show

All numbers are W-bMAE in seconds (lower is better), single seed, on the public Muse data. They come from three different runs, listed separately because they are not the same experiment.

| Run | Setting | Score |
|---|---|---|
| This solver, via `benchopt run -d Interaxon2026Muse`, 2026-10-13 | trained on the repo's train split, last epoch, scored by the official streamed objective on the 40 public test recordings | **50.4** (bMAE 117.7, MAE 218.0) |
| Same architecture, earlier independent implementation, 5-fold CV | recordings grouped by participant; mean of "seen" and "unseen" groups, +- SD over 5 folds | 45.3 +- 4.3 (seen 43.9, unseen 46.6) |
| Same earlier implementation, trained on the 500 public train recordings | 40 public test recordings, official benchmark replay | 55.8 (standard error across recordings about 7) |
| Reference points, earlier runs | constant 35 s: 62.1 CV / 60.7 test; boosted trees on causal spectral features: 45.6 CV / 52.5 test | |

How to read them:

- The three CNN+GRU numbers (50.4, 45.3, 55.8) are different splits and seeds. With 40 test recordings the standard error is about 6-7 s, so the 50.4 versus 55.8 difference is **noise**, and the CNN+GRU is not distinguishable from the boosted-tree baseline.
- The model is **clearly better than the constant (about 62) overall**, but the gain comes from windows more than 90 s before N2. **In the last 40 s it is not better than the constant**: on the 40 test recordings the earlier implementation's MAE there was about 27 s against 17 s for the constant, and that bin carries about half of the metric weight. Treat this solver as a baseline to improve on, not as a finished solution.
- Training-set W-bMAE at epoch 20 was 40.1 in the real run, which is a training number, not a generalisation estimate.
- A variant trained with `random_start` was about 4 s worse in CV (49.4 +- 5.0) and 58.2 on the test recordings: inside the noise, kept as an option because the public recordings are cut at 25 minutes and a recurrent state can learn that clock.

## 7. The solver file (`tracks/sleep_onset/solvers/cnn_gru.py`)

```python
"""CNN + GRU causal streaming regressor for Track 03 (Sleep Onset).

Per 5 s window: a 1-D CNN on the raw signal plus an MLP on the log power spectrum give one
embedding; a unidirectional GRU carries a recording-long state over successive windows; a
small head maps the state to seconds to the first N2, in [0, 600].

Streaming contract (benchmark_utils/streaming.py): every recording reaches ``predict`` one
window at a time (B = 1), in time order, on a ``copy.deepcopy`` of the loaded model whose
``reset_state()`` is called first. The GRU state therefore lives on the model object.
"""

import numpy as np
import torch
from torch import nn

from benchmark_utils.base_solver import CompetSolver

CAP_S = 600.0
BIN_EDGES = (0.0, 40.0, 90.0, 300.0, 600.0)
BIN_WEIGHTS = (10.0, 5.0, 3.0, 1.0)


class WindowEncoder(nn.Module):
    """One window (C, T) in microvolts -> embedding (dim,). Per-window preprocessing only,
    so it is causal by construction (no recording-level statistics)."""

    def __init__(self, n_chans=4, dim=96, n_fft=256, sfreq=128.0, fmax=45.0,
                 input_scale=40.0):
        super().__init__()
        self.n_fft = n_fft
        self.n_freq = int(fmax * n_fft / sfreq) + 1       # bins 0..fmax Hz
        self.input_scale = input_scale                      # 40 uV: raw EEG -> ~unit scale
        self.register_buffer("window", torch.hann_window(n_fft), persistent=False)

        def block(cin, cout, k, stride):
            return nn.Sequential(nn.Conv1d(cin, cout, k, stride, k // 2),
                                 nn.GroupNorm(8, cout), nn.GELU())

        self.cnn = nn.Sequential(block(n_chans, 32, 15, 2), block(32, 64, 9, 2),
                                 block(64, 96, 9, 2), block(96, 128, 5, 2))
        self.spec = nn.Sequential(nn.Linear(n_chans * self.n_freq, 128), nn.GELU(),
                                  nn.Linear(128, 64), nn.GELU())
        self.out = nn.Sequential(nn.Linear(2 * 128 + 64, dim), nn.GELU())

    def forward(self, x):                                   # (N, C, T), microvolts
        x = x.float()
        x = x - x.mean(dim=-1, keepdim=True)                # remove per-window DC offset
        h = self.cnn(torch.clamp(x / self.input_scale, -10.0, 10.0))
        h = torch.cat([h.mean(dim=-1), h.amax(dim=-1)], dim=1)       # (N, 256)
        n, c, t = x.shape
        n_fft = min(self.n_fft, t)
        win = self.window if n_fft == self.n_fft else torch.hann_window(n_fft, device=x.device)
        s = torch.stft(x.reshape(n * c, t), n_fft, hop_length=max(n_fft // 4, 1), window=win,
                       center=False, return_complex=True).abs().pow(2).mean(dim=-1)
        s = torch.log(s[:, : self.n_freq] + 1e-3)           # log power, 0..fmax Hz
        if s.shape[1] < self.n_freq:
            s = nn.functional.pad(s, (0, self.n_freq - s.shape[1]))
        return self.out(torch.cat([h, self.spec(s.reshape(n, -1))], dim=1))


class CausalSleepNet(nn.Module):
    """Window encoder -> unidirectional GRU -> seconds to N2 (capped at 600)."""

    def __init__(self, n_chans=4, dim=96, hidden=128, layers=1, dropout=0.1, sfreq=128.0,
                 input_scale=40.0):
        super().__init__()
        self.config = dict(n_chans=n_chans, dim=dim, hidden=hidden, layers=layers,
                           dropout=dropout, sfreq=sfreq, input_scale=input_scale)
        self.encoder = WindowEncoder(n_chans, dim, sfreq=sfreq, input_scale=input_scale)
        self.drop = nn.Dropout(dropout)
        self.gru = nn.GRU(dim, hidden, layers, batch_first=True)
        self.head = nn.Sequential(nn.Linear(hidden, 64), nn.GELU(), nn.Linear(64, 1))
        self._h = None                                      # streaming state

    def forward(self, x, h=None, return_state=False):       # x: (B, L, C, T)
        b, l = x.shape[:2]
        z = self.encoder(x.reshape(b * l, *x.shape[2:])).reshape(b, l, -1)
        out, h = self.gru(self.drop(z), h)
        y = CAP_S * torch.sigmoid(self.head(out).squeeze(-1))        # (B, L) seconds
        return (y, h) if return_state else y

    # ---- Codabench streaming interface -----------------------------------------------
    def reset_state(self):
        self._h = None
        self.gru.flatten_parameters()       # a deepcopy leaves RNN weights non-contiguous

    @torch.inference_mode()
    def predict(self, X):                                   # (B, C, T); B = 1 on Codabench
        self.eval()
        X = torch.as_tensor(X, dtype=torch.float32, device=next(self.parameters()).device)
        if self._h is not None and self._h.shape[1] != X.shape[0]:
            self._h = None
        y, self._h = self.forward(X[:, None], self._h, return_state=True)
        return y[:, 0].float()                              # (B,) seconds


def window_weights(y):
    """Weights of one recording's windows (sum = 1) such that sum(w * |err|) is exactly its
    W-bMAE: bin weight / sum of weights of the bins present / windows in the bin."""
    b = np.digitize(y, BIN_EDGES[1:-1])
    present = np.unique(b)
    norm = sum(BIN_WEIGHTS[k] for k in present)
    w = np.zeros(len(y), dtype=np.float32)
    for k in present:
        w[b == k] = BIN_WEIGHTS[k] / norm / (b == k).sum()
    return w


class Solver(CompetSolver):

    name = "CNN-GRU"

    # training hyper-parameters (the validated recipe)
    n_epochs = 20
    batch_recordings = 16          # recordings per optimisation step
    lr = 2e-3
    weight_decay = 1e-2
    dropout = 0.1
    random_start = False           # True: drop a random number of leading windows per step
    seed = 0

    def load_model(self, meta):
        weights = meta["submission_dir"] / "weights.pt"
        if weights.exists() and self.train_loader is None:    # inference-only run
            ckpt = torch.load(weights, map_location=meta["device"], weights_only=True)
            model = CausalSleepNet(**ckpt["config"])
            model.load_state_dict(ckpt["state_dict"])
        else:                                                  # fresh model for training
            model = CausalSleepNet(n_chans=meta["n_chans"], sfreq=float(meta["sfreq"]),
                                   dropout=self.dropout)
        return model.to(meta["device"]).eval()

    def save_model(self, model, path):
        torch.save(dict(state_dict=model.state_dict(), config=model.config),
                   path / "weights.pt")

    # ---- training ----------------------------------------------------------------------
    def _recordings(self, train_loader):
        """Rebuild time-ordered per-recording sequences from the window dataset (the train
        loader itself is shuffled and bin-balanced, so it cannot give sequences). Returns
        None when the dataset carries no record_id / onset (e.g. Simulated)."""
        ds = getattr(train_loader, "dataset", None)
        if ds is None or len(ds) == 0:
            return None
        info = ds[0][2]
        if not isinstance(info, dict) or "record_id" not in info or "onset" not in info:
            return None
        rows = {}
        for i in range(len(ds)):
            X, y, info = ds[i]
            rows.setdefault(int(info["record_id"]), []).append(
                (float(info["onset"]), X.float(), float(y)))
        recs = []
        for items in rows.values():
            items.sort(key=lambda t: t[0])
            recs.append((torch.stack([t[1] for t in items]),
                         np.array([t[2] for t in items], dtype=np.float32)))
        return recs

    def fit(self, model, train_loader):
        torch.manual_seed(self.seed)
        rng = np.random.default_rng(self.seed)
        device = next(model.parameters()).device
        use_amp = device.type == "cuda"
        recs = self._recordings(train_loader)
        if recs is None:                       # smoke-test fallback: windows as length-1 runs
            print("[CNN-GRU] no record_id/onset in the train loader: stateless window mode")
        n_steps = self.n_epochs * (int(np.ceil(len(recs) / self.batch_recordings))
                                   if recs else len(train_loader))
        opt = torch.optim.AdamW(model.parameters(), lr=self.lr,
                                weight_decay=self.weight_decay)
        sched = torch.optim.lr_scheduler.OneCycleLR(opt, max_lr=self.lr,
                                                    total_steps=max(n_steps, 1), pct_start=0.1)
        scaler = torch.amp.GradScaler(enabled=use_amp)
        pad = nn.utils.rnn.pad_sequence
        for epoch in range(self.n_epochs):
            model.train()
            if recs is not None:
                order = rng.permutation(len(recs))
                batches = [order[i:i + self.batch_recordings]
                           for i in range(0, len(order), self.batch_recordings)]
            else:
                batches = train_loader
            total = 0.0
            for item in batches:
                if recs is not None:
                    Xs, ys, ws = [], [], []
                    for j in item:
                        X, y = recs[j]
                        if self.random_start and len(y) > 12:
                            s = int(rng.integers(0, len(y) - 12))   # keep >= 60 s before N2
                            X, y = X[s:], y[s:]
                        Xs.append(X)
                        ys.append(torch.from_numpy(y))
                        ws.append(torch.from_numpy(window_weights(y)))
                    X = pad(Xs, batch_first=True).to(device)
                    y = pad(ys, batch_first=True).to(device)
                    w = pad(ws, batch_first=True).to(device)
                else:
                    X, y, _ = item
                    X, y = X[:, None].to(device), y.float().reshape(-1, 1).to(device)
                    w = torch.full_like(y, 1.0 / len(y))
                with torch.autocast(device.type, dtype=torch.float16, enabled=use_amp):
                    pred = model(X)
                loss = (w * (pred.float() - y).abs()).sum(dim=1).mean()   # mean W-bMAE (s)
                opt.zero_grad(set_to_none=True)
                scaler.scale(loss / 100.0).backward()
                scaler.unscale_(opt)
                nn.utils.clip_grad_norm_(model.parameters(), 1.0)
                scaler.step(opt)
                scaler.update()
                sched.step()
                total += loss.item()
            print(f"[CNN-GRU] epoch {epoch + 1:2d}/{self.n_epochs}  "
                  f"train W-bMAE {total / len(batches):6.1f}", flush=True)
        model.eval()
        model.reset_state()
```

## 8. Adding it to the repository and running it

```bash
# 1. Save the code block of section 7 as tracks/sleep_onset/solvers/cnn_gru.py

# 2. Contract check, no data needed (also trains once and exports a submission):
benchopt run tracks/sleep_onset -d Simulated -s CNN-GRU -o "Sleep-onset[training=True]"

# 3. Train on the real Muse data and export a ready submission folder
#    (training.yml pins Interaxon2026Muse and training=True):
benchopt prepare tracks/sleep_onset --config tracks/sleep_onset/training.yml
benchopt run     tracks/sleep_onset --config tracks/sleep_onset/training.yml -s CNN-GRU
#    -> tracks/sleep_onset/outputs/CNN-GRU/submission.py and weights.pt

# 4. Replay the exported folder inference-only, exactly as the platform does:
COMPET_SUBMISSION_DIR="$PWD/tracks/sleep_onset/outputs/CNN-GRU" \
  benchopt run tracks/sleep_onset -d Interaxon2026Muse \
  -s "$PWD/tracks/sleep_onset/outputs/CNN-GRU/submission.py"

# 5. Upload the CONTENTS of that folder (both files at the ZIP root):
cd tracks/sleep_onset/outputs/CNN-GRU && zip -j ../CNN-GRU.zip submission.py weights.pt
```

Notes:

- `benchopt` runs the solver under the name `CNN-GRU` (the `name` class attribute). `-s CNN-GRU` selects it; `-s path/to/submission.py` selects a file.
- Training hyper-parameters are class attributes (`n_epochs`, `lr`, `random_start`, ...). Change them in the file; changing them does not change the architecture config stored in `weights.pt`, which is what inference uses.
- Do not upload to the live Codabench warm-up while it still scores the Sleep-EDF proxy (2 channels at 100/120 Hz): this model expects 4 channels at 128 Hz. Check the Codabench Track description first.

## 9. What was verified, and what was not

Verified by running (2026-10-13, Python 3.13, torch 2.8 cu126, benchopt 1.10, one V100):

- Parameter count and tensor shapes as in section 3; streaming output equals the sequence forward pass; `reset_state()` restores a fresh state; `copy.deepcopy` produces independent streams; no RNN-contiguity warning after `deepcopy` + `reset_state()`.
- `benchopt run -d Simulated -o "Sleep-onset[training=True]"`: trains, exports `submission.py` + `weights.pt`, evaluates. An inference-only replay of the exported folder loads the weights and evaluates (`weights_only=True` load works).
- The sequence path on synthetic recordings that carry `record_id`/`onset`: 40 recordings rebuilt in time order from a shuffled loader, and the streamed held-out W-bMAE beats a constant (41.1 against 51.8, synthetic data, so only a plumbing check).
- The real Muse data through benchopt: training ran to completion on the real loader (sequences rebuilt from `_NBWindows`), weights exported, and the streamed evaluation on the 40 test recordings gave 50.4.

Not verified:

- Anything about the sealed phase (unseen participants, cropping, scorer). The seen/unseen macro-average is not reproduced by `benchopt run`, which scores the public warm-up set.
- Seed variance and hyper-parameter tuning: one seed, the recipe above untuned in this repo; no validation-based early stopping or checkpoint selection.
- Inference time on the Codabench A100 worker (local latency only); GPU memory on the worker.
- The first benchopt training run of this session was killed by the kernel OOM killer during evaluation on the shared machine (the weights had already been exported); the numbers above come from the inference-only replay of those exported weights.

## 10. Pitfalls

- **Do not feed it a shuffled stream.** The state is meaningful only if windows arrive in time order within a recording. The streamed objective guarantees this; a batch objective that mixes windows would make the GRU state meaningless (use a stateless baseline there).
- **Use `B = 1` semantics.** If `predict` is ever called with a different batch size than the stored state, the state is dropped rather than mixed; do not rely on batched inference for streams.
- **Collecting sequences is slow the first time.** `ds[i]` extracts each window lazily from the neuralset caches (tens of thousands of items, about 1 GB of float32 in RAM). Keep `num_workers=0` as in the dataset adapter, or cache the arrays once.
- **Shared machine memory.** Training holds all training windows on the CPU; on a crowded host leave headroom or store them as float16.
- **Sampling-rate dependence.** The spectral branch derives its bins from `meta["sfreq"]` and `n_fft = 256`; at 128 Hz a bin is exactly 0.5 Hz and the kept band is 0-45 Hz. Other rates run, but the learned weights are specific to 128 Hz windows of 640 samples.
- **The recurrent state can count.** A GRU can learn "how many windows since the start". On the public Muse set this correlates with the target (recordings are cut at 25 minutes); `random_start = True` is the guard if the sealed cohort is cut differently.
- **Amplitude is anchored at 40 uV.** `input_scale` is part of the saved config; do not change it between training and inference.

## 11. Directions that were not tested here

Each of these is a hypothesis from earlier analysis of the public Muse data, **not** validated in this solver:

1. A distributional head (discrete hazard or ordinal bins over [0, 600) plus an explicit "600 or more" class), decoded with the metric-weighted median, instead of a single sigmoid output; the model currently shrinks toward the middle and never predicts above about 300 s.
2. Recording-relative inputs, such as the 30 s exponential moving average of band power minus its running mean over the recording so far, next to the raw level.
3. Shrinking predictions to the constant when the model is uncertain, especially in the first minutes of a recording.
4. Training with `random_start = True` and masking the first 5-10 s of each recording, because the start of public recordings is noisier and has a settling transient.
