# Known issues of the Track 03 dataset (Interaxon2026Muse)

The task: predict, for every non-overlapping 5 s window of a 4-channel Muse EEG recording (TP9, AF7, AF8, TP10; 128 Hz; µV, unfiltered),
the seconds remaining until the first N2 sleep epoch, capped at 600 s. Windows arrive one at a time (causal streaming).
The score is a weighted, binned MAE (W-bMAE), computed per recording and then averaged.

All figures come from the exploratory analysis in [eda_muse/EDA_Muse_va_phuong_phap.md](eda_muse/EDA_Muse_va_phuong_phap.md) (sections 1-10),
unless noted. The dataset has 540 recordings from 203 people (157.5 hours). Of its 113,375 windows, 80,975 lie before N2 and are scored.

## Summary

| # | Issue | Why it matters | What we do about it |
|---|---|---|---|
| 1 | One label per recording; file length reveals it | very little supervision; easy to leak the answer | strict streaming; no whole-file statistics |
| 2 | The metric's weight sits on the scarcest windows | 5% of the windows carry 53% of the score | weighted loss; distributional head with a "≥ 600 s" class |
| 3 | Noisy signal, frequent channel failures, dirty start | features and CNN input are often corrupted | low-pass 45 Hz; artifact features; start-of-recording handling (E4) |
| 4 | Weak precursors of N2 | single windows separate the last bins poorly | minutes of context (GRU); realistic error floor |
| 5 | Large differences between people and nights | features shift by several dB from one night to another | augmentation; recording-relative context |
| 6 | The public test is small and unrepresentative | public scores are noisy and can disagree with CV | person-grouped CV; require agreement of CV and public |

## 1. Few labels, and a leaky structure

- **One label per recording:** the onset of the first N2 epoch (`n2_onset`, seconds from the start). There is no hypnogram, no N1 annotation,
  and no age or sex (`participants.tsv` holds only `n/a`).
- **The file length gives the answer away:** `recording_duration − n2_onset` equals exactly 300 s for all 540 recordings, and
  `sample_count = recording_duration × 128`. Anything derived from the length of the file is a label leak. Therefore:
  - no normalisation by whole-file statistics;
  - no "windows remaining" or "relative position in the file" as an input;
  - local evaluation in the same streaming mode as the platform (one window at a time, the end unknown).
- **The "off-grid" group (77 recordings):**
  - 85.7% of the onsets lie on the 30 s scoring grid counted from the start of the recording; these 77 do not.
  - All of them have onsets at or after 913.8 s.
  - Their sample counts are not multiples of 640, and none of them starts with the switch-on transient.
  - The most likely explanation (not confirmed by the metadata) is that their beginnings were cut, so "time since the start"
    means something different for them.
  - They are 13.6% of train but 22.5% of the public test.
- **The windows after N2 are not scored**, but about 5 minutes of them are present in every file. They are usable for training (for example
  auxiliary tasks), never for features at inference.

## 2. The metric puts its weight on the scarcest data

| Target bin | Share of windows | Share of the score weight |
|---|---|---|
| 0-40 s | 5.3% | **52.9%** |
| 40-90 s | 6.7% | 26.5% |
| 90-300 s | 27.3% | 15.8% |
| 300-600 s | 60.7% | 4.8% |

- A window in the 0-40 s bin weighs about 100 times as much as one in the 300-600 s bin, so the model must be best where examples are rarest.
- **30.2% of the targets equal exactly 600 s** because of the cap, and 43.8% are at least 450 s. The capped mass is a separate class,
  not an ordinary regression value. This is why the solvers use a 121-class distributional head with a "≥ 600 s" class, a loss weighted
  by the metric, and median decoding.
- A recording has a median of 138 scored windows. 39.4% of the recordings never reach the cap (onset < 600 s); 7.0% have an onset
  under 300 s.

## 3. The signal is noisy and channels often fail

| Window statistic (before N2) | TP9 | AF7 | AF8 | TP10 |
|---|---|---|---|---|
| Median SD (µV) | 9.9 | 5.5 | 5.2 | 10.0 |
| 95th percentile SD (µV) | 162 | 92 | 77 | 149 |

- **Saturation:** 519 of 540 recordings hit about 780 µV (the ADC limit) on at least one channel.
- **Bad channels:**
  - Definition: a channel-window is bad when max |x| > 200 µV or the 30-45 Hz RMS exceeds 5 µV.
  - 21.7% of the scored windows have at least one bad channel, and 5.2% have all four bad.
  - The temporal channels fail more often than the frontal ones (22.6% against 14.2% far from N2).
- **Dirty start of the recordings:**
  - 67.8% of the recordings begin with a near-zero sample on all channels (the switch-on transient of the device filter).
  - The share of bad channel-windows is 41.4% in the first 30 s, 29.5% at 1-2 minutes and 13.4% at 2-5 minutes.
- **Interference:**
  - A clear 50 Hz peak in 200 recordings and a 60 Hz peak in 330.
  - Narrow lines specific to AF7 at 42.67 Hz (fs/3, 62 recordings) and 21.33 Hz (fs/6, 38 recordings). They can act as a device
    fingerprint for a deep model.
- **Artifacts are themselves informative:** the share of windows with a bad channel falls from 31.3% (more than 600 s before N2) to 3.5%
  (the last 40 s). Artifact features carry real information about wakefulness, but they also contaminate the spectral features.

What we do: a causal 45 Hz low-pass; artifact features computed from the raw signal; and the start-of-recording handling of experiment E4,
which won in CV and is part of the best submission:
- the first second is zeroed;
- the first window gets loss weight 0;
- while training, recordings start at a random point in the first 3 minutes.

## 4. Weak precursors of N2

- **Two families of features:**
  - Slowly drifting features (beta, gamma, alpha, amplitude, artifacts) change over 5-10 minutes. They separate "far" from "fairly close".
  - Late features (sigma/beta, 1/f slope, theta/alpha) stay flat and change only in the last 1.5-2.5 minutes.
- **The last two bins are hard to tell apart:** the best single feature separates 0-40 s from 40-90 s with an AUC of only 0.65-0.67
  (0.5 is chance). The late features change along a nearly straight line, with no step at the 40 s mark.
- **Delta power is U-shaped:** it falls as eye movements and artifacts disappear, then rises again about 90 s before N2. Part of the
  broadband power decrease near N2 is simply the disappearance of artifacts.
- **Spread between recordings:** the spread between recordings is about as large as the change itself, so a single window is not
  enough; minutes of context and some form of within-recording normalisation are needed.
- **Realistic floor:** an error of roughly ±20-30 s in the last 90 s is a natural limit of this data. The larger gains are in separating
  40-90 s, 90-300 s and the cap correctly.

## 5. Differences between people and between nights

| Baseline feature | ICC(1) | SD between people | SD between nights of the same person |
|---|---|---|---|
| alpha 8-11 Hz, temporal | 0.595 | 2.76 dB | 2.28 dB |
| amplitude (log SD), temporal | 0.443 | 5.3 dB | 5.9 dB |
| beta, temporal | 0.406 | 2.08 dB | 2.52 dB |
| delta, temporal | 0.351 | 4.27 dB | 5.80 dB |

- Baseline levels are partly personal, but they move almost as much from night to night as from person to person.
- **The onset itself is barely predictable from identity:** ICC 0.38. The SD between people is about 216 s; between nights of the same person
  it is about 275 s. Knowing who the person is helps little, and identity may not be used at inference anyway.
- Remedies: normalisation relative to the running recording, and augmentations such as random gain or hemisphere swap.

## 6. The public test is small and does not represent the sealed test well

- **Size and composition:**
  - The public test has only 40 recordings from 40 people, and every one of them has another night in train.
  - It therefore measures only "a new night of a known person". The sealed score is the average of a seen and an unseen group, and the
    public test says nothing about the unseen half.
  - With 40 recordings the standard error of the public score is about 4 s, and a single training run varies by about ±2 s.
- **The test is cleaner at the start:** the median share of windows with a bad channel in the first minute is 0.75 in train but 0.42 in
  test (Mann-Whitney p = 0.0014). The off-grid group is also over-represented in test (point 1).
- **The clock:**
  - Elapsed time has a Spearman correlation of −0.44 with the target, stronger than any EEG feature.
  - This is largely a property of how the public set was curated: every recording ends within about 25 minutes, and the per-minute hazard
    of N2 rises to 100% at minute 24.
  - If the sealed set is cut differently, a model that counts windows since the start will fail. We check this with two probes:
    - crop-start: the beginning of each held-out recording is removed;
    - extension: up to 5 minutes of another recording's wake start are prepended.
- **Consequence for model selection:** CV and the public test can disagree. Mixup augmentation, for example, improved the public score of
  a 5-model ensemble by 1.3 s but was neutral in CV (see [AUGMENT_EXPERIMENTS.md](AUGMENT_EXPERIMENTS.md)).
  - Our validation uses 5 participant-grouped folds, each with a seen and an unseen group.
  - A change counts as an improvement only when CV and the public test agree.

## Rules that follow from these issues

1. Never use the file length, the number of samples, whole-file statistics, the recording order, or participant identity.
2. Weight the loss like the metric and model the capped mass explicitly.
3. Expect 1-3 missing channels per window and a dirty first minute; handle both explicitly.
4. Give the model minutes of causal context; do not expect single windows to be decisive.
5. Validate on people the model has never seen, average over several seeds, and treat the 40-recording public test as a sanity check,
   not as the selection criterion.
