Skip to content

feat(whoop5): decode v18 sleep_state @81 + status_word @75; rename @63 -> activity_class#481

Open
j0b-dev wants to merge 6 commits into
NoopApp:mainfrom
j0b-dev:feat/v18-sleep-state-swift
Open

feat(whoop5): decode v18 sleep_state @81 + status_word @75; rename @63 -> activity_class#481
j0b-dev wants to merge 6 commits into
NoopApp:mainfrom
j0b-dev:feat/v18-sleep-state-swift

Conversation

@j0b-dev

@j0b-dev j0b-dev commented Jun 16, 2026

Copy link
Copy Markdown

What this does

Surfaces the WHOOP 5/MG strap's own on-device sleep + activity signals from the v18 type-47 record, which were previously left in the "unmapped" region (or mislabeled). It decodes sleep_state (@81) and status_word (@75) in decodeWhoop5Historical, and renames @63 to its real meaning (activity_class) on both Swift and Android.

These come from reverse-engineering the strap v18 record builder, then validating against real captures.

What it fixes / unlocks

1. Sleep that doesn't collapse to "all light" (relates to #469).
The current sleep-stage breakdown is an app-side actigraphy approximation, and on some nights it reports nearly 100% light with almost no deep/REM despite a healthy night. The strap RE explains why: deep/REM/light are not on the strap at all — WHOOP computes them in the cloud. What the band does emit, in the clear, is its own per-second 4-state determination:

sleep_state = (frame[81] >> 4) & 30 wake / 1 still / 2 asleep / 3 up (arousal)

Decoding it gives the app a trustworthy, device-sourced wake/asleep/arousal timeline to anchor the sleep view on — instead of inferring everything from motion. Validated against a real night: it tracks evening wind-down → asleep (resting HR drops) → brief up arousals → morning wake. It won't invent deep/REM (those genuinely require the cloud), but it stops the "everything is light" failure mode by grounding wake vs asleep in the band's own call.

2. Real activity classification, not a mislabeled field (relates to #132).
@63 was decoded as motion_wear_quality ("poor contact"). The strap shows it's actually the ICM-45686 APEX activity class: 0 still / 1 walk / 2 run. Renaming it to activity_class makes it usable for activity features (e.g. walk/run minutes) and corrects a wrong label. Real-data distribution fits: mostly still, a chunk of walking, rare running. (This complements the already-upstream @57 step counter.)

3. Prevents a wrong "deep sleep" reading.
@75 looks like it could be a deep-sleep flag (values like 0x50/0xA0), but across ~258k real records its low nibble is always 0 and the value occurs as often awake as asleep — it's a packed status word, not a depth metric. Decoding it explicitly as status_word (with that note) keeps anyone from shipping 80=deep as a sleep stage.

Why it makes the features better

  • 5/MG sleep view can show the strap's real wake/asleep/arousal state — a reliable coarse signal — rather than a pure-actigraphy guess that can misfire.
  • Activity gets a genuine still/walk/run class from the device.
  • Correctness: replaces an "unmapped (…/sleep-state)" region + a mislabeled @63 with honest, strap-grounded fields, and forecloses a plausible-but-wrong deep-sleep misread.

Scope & test plan

  • Decode-only (fields land in ParsedFrame / the decode-features store, like step_motion_counter/skin_temp_raw); no UI or aggregation changes.
  • swift test (Packages/WhoopProtocol) — full suite green, 157 tests, 0 failures.
  • sleep_state asserted on the existing real fixture (wake=0) + the (b>>4)&3 mapping (states 0/1/2/3, high-nibble-only) via an in-memory @81 override — no new device frames committed.
  • status_word asserted on the existing fixture; @63 test updated to activity_class.
  • Android: decodes @63 rename + cardiac block + status_word/sleep_state at parity; covered by the Android unit suite.

Notes

  • @81/@75/cardiac decode is now on both Swift and Android (the Android decoder reaches parity in HistoricalStreams.kt; the sleep-state nibble-mapping test re-stamps the CRC32 since Android gates on it, unlike Swift). Full Android unit suite green.
  • Deep/REM/light remain cloud-only by design — this PR is honest about that rather than approximating them.

Cardiac fields (added)

Also decodes the v18 @33-40 cardiac block in decodeWhoop5Historical:

  • hr_fixed_8_8 (@36, u16) — HR in 8.8 fixed-point, bpm = value/256 — a higher-precision heart rate than the integer hr@22 (validated: fixture 25997 -> 101.55 vs hr 102; corr 0.989 over 257k frames). Produced by the SIGPROC PPG->HR DSP.
  • rr_packed (@38), cardiac_flags (@33), cardiac_status (@40) — emitted raw with honest "DSP-gated/unvalidated" notes (same pattern as status_word). HeartKey/ECG is NOT the 1 Hz record producer.

Late/header fields (added)

Also decodes three more v18 fields in decodeWhoop5Historical on both Swift and Android, with identical keys:

  • record_index (u32 LE @11) — monotonic per-record counter (low byte = byte@11; same field the v20/v21 layouts expose, iOS Sync Issue: Historical records use firmware layout v20/v21 #344). Validated on two devices.
  • apex_cadence (u8 @59) — ICM-45686 APEX samples-per-step, the byte between step_motion_counter@57 and activity_class@63; raw (scale chip-gated).
  • dsp_metric (f32 LE @113) — DSP telemetry float (−5.31…0.0; 0.0 = unset); raw, meaning DSP-gated.

Doc rows for these are in #483; the cadence row is in #474.

…59, dsp_metric @113 (Swift + Android)

Adds three v18 type-47 fields to decodeWhoop5Historical on both platforms,
keeping the parsed key set identical:

- record_index (u32 LE @11): monotonic per-record counter (low byte = byte@11;
  same field the v20/v21 bulk layouts expose, NoopApp#344). Validated on the real worn
  fixture (25443699) and off-wrist (25443642).
- apex_cadence (u8 @59): ICM-45686 APEX samples-per-step, between
  step_motion_counter @57 and activity_class @63; raw (scale chip-gated).
- dsp_metric (f32 LE @113): DSP telemetry float (worn -5.2307; off-wrist 0.0 =
  unset); raw, meaning DSP-gated.

Tests assert exact values off the same real frames already in the suite (no new
device captures). swift test 158 green; Android testFullDebugUnitTest green.
@j0b-dev j0b-dev force-pushed the feat/v18-sleep-state-swift branch from c45fbfd to afe8cb4 Compare June 16, 2026 14:54
…p_state @81 (Swift parity)

Brings the Android v18 decoder to parity with Swift decodeWhoop5Historical:
- cardiac block @33-40 (cardiac_flags, hr_fixed_8_8 = HR 8.8 fixed-point,
  rr_packed, cardiac_status)
- status_word @75 (packed; not a deep-sleep marker)
- sleep_state @81 = (byte >> 4) & 3 (band SLEEPFLAG; deep/REM/light are cloud-side)

Tests assert the exact values off the same real frame the Swift suite uses; the
sleep-state nibble-mapping test re-stamps the CRC32 (Android gates on CRC, unlike
Swift) via the existing Crc.crc32 over frame[8..len-4]. Full Android unit suite
691 green.
@NoopApp

NoopApp commented Jun 16, 2026

Copy link
Copy Markdown
Owner

@j0b-dev — first, genuinely: the validation here is some of the best observational work the project has had. Two devices, 96/96 HR-timestamp matches, the 0.989 HR correlation, the 725–900× step-inflation tables, tracking @81 against a real night's wind-down → asleep → arousals → wake. That rigor is exactly how we want format facts established, and the behaviours you've found are real and valuable.

But I can't merge this cluster as it stands, and I want to be completely straight about why — it's a hard line for NOOP specifically, not a knock on the work or your intent.

The blocker is provenance, not correctness. The PRs describe the map as coming from "reverse-engineering the strap v18 record builder" / a "firmware-grounded field map," and the docs name internal parts (AS6221, the ICM-45686 APEX class, the SIGPROC DSP) that can't be observed from BLE frames — they're firmware identifiers. So the discovery ran through WHOOP's firmware, with captures used to validate.

NOOP's entire defensibility — the literal promise in our README/FAQ, and the thing that lets an anonymous interop project exist — is "no WHOOP proprietary code, firmware, or assets, and no knowledge derived from disassembling the firmware." Observing the data your own strap emits is interoperability; disassembling the firmware that produces it is a different act, and the moment our code or docs carry "firmware-grounded" provenance (or WHOOP's internal part names), that shield has a hole. "It was validated with captures" doesn't close the hole if the map came from firmware, and "a contributor sent it in" isn't a defense either. I have to hold this consistently or it's worth nothing — I applied the same line to some firmware images that landed in my lap earlier today.

Here's the good news, and the path I'd love to take with you — most of these fields look independently establishable from observation alone, and you've already done the hard validation:

  • @81 sleep-state already has a clean, independent origin on the board: @Extazian hypothesised the @81 high-nibble as a sleep state on 🙏 Roadmap & help wanted — specific things we need (grab one you can do) #132 from captures + Fitbit ground-truth, before any firmware was in the picture. We can pursue that on its own merits — cross-validate the @81 nibble against scored nights — and it stands without a single firmware reference.
  • @57 steps (cumulative counter → wrap-aware diff) and @36 HR (the /256 value that correlates 0.989 with @22) are pure behaviours — discoverable by anyone scanning the record against ground truth, no firmware needed.

So the ask: can you re-present the map grounded only in observation — "here's the capture, here's how @81's nibble tracks a scored night, here's the counter behaviour" — with the firmware terminology and the AS6221/APEX/SIGPROC names stripped (describe them by what they do, as observed: "a skin-temp sensor," "an activity class," "a PPG signal-quality block")? If the behaviour holds from observation alone (your data says it does), I'll reimplement the decode as a NoopApp commit with full credit to you for the capture analysis and validation. We get 5/MG wake/asleep/arousal and correct steps — I want those wins — just on footing that can't ever be turned against the project.

Holding this + the companions (#476 / #472 / #474 / #483) pending that re-grounding. If a given field genuinely can't be gotten without the firmware, we leave that one out rather than launder it. Thank you for the depth — this is the work that gets 5/MG sleep over the line; I just need it to come in the front door. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants