# Raw EEG-sAAD dataset EDA

Generated from `/root/disk_3tb/EEG-sAAD-Grand-Challenge-Training-Dataset/data/superhuge/raw` on 2026-09-24 by `run_eda_raw.py`.

## Scope and method

This analysis scans the complete SuperHuge `raw` stage: metadata and all paired EEG/envelope `.npy` files. Signals are read by memory map in chunks; the analysis does not modify source data. Trial inventory and descriptive metrics are exact where calculated over all samples. Distribution plots use 64 reproducible random time points per trial. Example traces show the first 10 seconds of the first eligible trial per dataset and are standardized only for display.

## Corpus overview

- Metadata entries: **1,979**; paired usable trials: **1,979**.
- Subjects across dataset-specific IDs: **95**.
- Total recording duration: **77.0 hours**.
- Sampling rates found: **128 Hz**.
- Channel-count groups found: **64, 253**.
- Dataset trial-level labels: `{'R': 784, 'L': 776, 'left': 210, 'right': 209}`. These are metadata labels, not continuous attention-switch labels.

![Corpus overview](01_corpus_overview.png)

| Dataset | Subjects | Trials | Hours | Channels | fs | Trial duration median (s) | Label counts |
|---|---:|---:|---:|---|---|---:|---|
| KUL-AV-GC_raw | 13 | 99 | 16.5 | 64 | 128.0 | 600.0 | left:50; right:49 |
| DTU_raw | 18 | 1080 | 15.0 | 64 | 128.0 | 50.0 | L:536; R:544 |
| KUL_raw | 16 | 320 | 20.5 | 64 | 128.0 | 123.9 | left:160; right:160 |
| KUL-HighDensity_raw | 30 | 120 | 13.0 | 253 | 128.0 | 391.0 | L:60; R:60 |
| USTC_raw | 18 | 360 | 12.0 | 64 | 128.0 | 120.0 | L:180; R:180 |

## Sampling, duration, and montage

![Trial duration](02_trial_duration.png)

The raw stage is already represented on its stored sample grid. The converter documents 128 Hz as the target for raw data; original recordings with another rate may have been resampled during CND preparation. This EDA reports actual metadata rates instead of assuming a common rate. EEG channel montage differs: the 64-channel datasets and the high-density dataset have separate channel-count groups. Channel names are preserved in metadata; compare `channel_signal_summary.csv` before pooling/mapping channels.

![Channel RMS](05_channel_rms_heatmap.png)

## Signal distributions and examples

![Signal distributions](03_signal_distributions.png)

EEG values are stored in the source numeric scale (the converter writes NumPy arrays from CND; this report does not infer physical units unless calibration metadata establishes them). Histograms are based on reproducible random samples across trials. Per-channel full-corpus mean, standard deviation, RMS and extrema are in `channel_signal_summary.csv`.

![Example traces](06_example_raw_traces.png)

The trace figure standardizes each displayed channel/envelope separately to show temporal shape. Use the CSV summaries for scale comparisons. Envelope files have time × feature × stream layout; inspect `envelope_summary_by_trial_stream.csv`. Per-trial correlation between the two envelopes is in `trial_inventory.csv`.

## Labels and task interpretation

The SuperHuge metadata `label` field is a trial-level category where available (often a direction/azimuth-derived left/right label). It does not provide a time-varying attention-switch sequence. The envelope tensor contains two concurrent streams; the converter convention places the majority-attended stream in column 0 and preserves the competing stream in column 1. For switch trials, that convention is trial-level and must not be mistaken for per-sample switch ground truth. Dataset/subject/trial IDs are organization keys and should not be used as predictive inputs in the blind evaluation.

![Trial metadata label counts](04_trial_label_counts.png)

## Integrity checks

- Missing paired EEG/envelope files: **0**.
- EEG length differs from metadata: **0**.
- Envelope time length differs from EEG: **0**.
- Trials with any nonfinite EEG values: **0**.
- Trials with any nonfinite envelope values: **0**.
- Additional structural issues recorded: **0**.

See `data_integrity_issues.csv` for any flagged entries, and `trial_inventory.csv` for every trial's shape, duration, storage dtype/order, file pairing, finite-value counts and signal summaries.

## Files in this EDA folder

- `dataset_summary.csv`: dataset-level counts, durations, labels and signal summaries.
- `subject_summary.csv`: per-dataset/per-subject recording coverage.
- `trial_inventory.csv`: one row per trial with metadata, shape, quality and signal statistics.
- `channel_signal_summary.csv`: exact pooled channel statistics across samples.
- `envelope_summary_by_trial_stream.csv`: per-trial/per-stream envelope statistics.
- `data_integrity_issues.csv`: missing or structurally inconsistent data.
- `01_...png` through `06_...png`: overview, distributions and example plots.
- `run_eda_raw.py`: reproducible analysis script.

## Interpretation caveats

1. Numeric amplitude units and reference montage are not inferred from values alone; use each source dataset's acquisition documentation for physical interpretation.
2. Dataset IDs and subject IDs are assigned by the unified conversion and need not match original releases.
3. The trial-level `label` and envelope column ordering do not replace time-varying attended-speaker or attended-direction ground truth for switch detection.
4. This is a descriptive EDA, not a validation that the attention labels are behaviorally correct.
