Split app layer into open_health - #7
Conversation
A single-page, local-first health dashboard following notes/dashboard-v2-brainstorm.md. - Rust (crates/oura-cli/src/dashboard.rs) owns the DB + non-model calcs: per-night HRV/RHR/skin-temp, SpO2 % via Oura's R→% calibration, baselines + deltas, the Device & Data Health panel, and the digest. It serves the page + /api/summary. - The torch models (sleep hypnogram, activity, CVA) run via the Python runners, shelled out like `oura sessions`. Added `--json` to run_sleep_model.py and run_cva_model.py for machine-readable output. - Frontend in a separate dashboard/web/ folder: vanilla HTML/CSS/JS, no build, no external deps. Auto light/dark via prefers-color-scheme, skeleton loading, reduced-motion safe, one accent + one radius, system + mono type. Activity grouped by day, per-night hypnograms, vitals tiles with sparklines. - New `oura dashboard --port --tz-offset --sex/--age/--height/--weight` subcommand (loopback-only). Default build includes it; models still run via Python. Verified in Playwright: light, dark, and mobile all render correctly.
- Display serif (system-only, offline) for the digest 'what changed' statement and the big hero numbers (vascular age, SpO2); data/tables stay mono. - Small UPPERCASE letter-spaced labels for panel headings, tile labels, day headers (the scientific/editorial meta voice). - Italic serif sub-lines under the hero numbers; more air on panels. Verified in Playwright (light, dark, mobile). No external/Google fonts — keeps the dashboard fully offline.
Functional: - Sync the ring from the header (POST /api/sync runs `oura sync` as a subprocess, using the dashboard's --name/--key-file), then auto-refreshes. CSRF-guarded. - Battery in the header + Device panel, read offline from stored `battery_level_changed` debug events (battery % + voltage). - Editable user profile (age/sex/height/weight/ring) the ring can't measure, stored in a gitignored profile.json next to oura.db and used by the CVA model + runners (GET/POST /api/profile, CSRF-guarded; run_cva_model.py reads it as defaults). Fixes the wrong "-11.2 yr vs your age" — now driven by your real age. Visual: - Vendored Phosphor icons (MIT, offline) on panel headers, actions, chips. - Leaner palette: neutral outline tags, subtle status dots instead of filled pills, monochrome thin sparklines with a faint area fill. Verified in Playwright (light/dark): battery, icons, profile edit → CVA re-run.
- Device & data health now shows the ring's identity, read offline from the
`device` table: Ring ID (serial), firmware, API version, MAC. Added
Store::device_info() (device row joined with sync_state).
- "Last sync" now uses the real sync_state.last_sync_unix instead of the event
anchor, so it reflects the actual last drain.
- Sync errors are surfaced in a visible toast with an actionable hint
("Couldn't find your ring. Take it off the charger…") instead of only a
tooltip that read "Failed". Success shows the synced summary line.
Verified in Playwright: identity renders, sync toast shows the real reason.
… fixes
- estimate active kcal per detected session; show it in the session popover
- cache build_summary (invalidate on db/profile mtime); run sleep/CVA/activity
models concurrently; batch all nights through one sleep-model process
- resolve --db to an absolute path so Rust and the Python runners open the same
file (Bugbot: Python models use wrong DB)
- HRV digest %: divide the delta by the prior-night mean baseline, matching the
vitals tile (Bugbot: HRV digest percent wrong)
- recovery line keys off HRV direction (RHR-down as fallback), not the raw sign
of an HR-rise fragment (Bugbot: recovery text ignores resting HR)
- surface profile-save errors instead of closing the dialog on a 200+{error}
response (Bugbot: profile save ignores server errors)
- vivid-but-glassy hypnogram colors; dot-style toasts; activity icons
…nvert, decode notes - `oura sleep-score`: SleepNet hypnogram → durations/efficiency + calibrated contributor curves & combiner weights (tools/score_sleep.py, fit_*.py) - score-weights doc + data-recovery-map: the app (not cloud) is the analytics tier - run_stress_model.py runner - Ring Runner: persist horizontal/vertical invert toggles across restarts - real_steps decode: byte-layout notes confirmed vs the native parser
- route API on the path only, stripping any query string, so /api/summary?cb=… no longer 404s (Bugbot: API path ignores query string) - guard the vitals trend against a zero baseline → no Inf/NaN in the JSON (Bugbot: vitals delta divides by zero) - "Real steps" capability tile checks feature_1 OR feature_2, matching the event the Steps stream actually counts (Bugbot: real steps toggle wrong event) - last-sync fields come only from a real recorded sync timestamp; absent → null (UI shows "—") instead of faking freshness from the latest event's capture time (Bugbot: wrong fallback for last sync)
When the scored night can't be matched to a stored bedtime, fall back to the night's own midpoint (start_ds, end_ds) instead of indexing bts[-1] — which used the wrong night's end and raised IndexError when bts was empty (e.g. --start/--end with no bedtime_period rows). timing_features now takes end_ds.
A session crossing midnight had its bar sized from full duration_min while end was clamped to 1440, so the bar ran wider than its lane and the tooltip/popover time range didn't match the duration. Keep a true uncapped end (endTrue) for the duration/range label, and size the bar from the day-clamped (end - start).
…ror state
- score_sleep: compute the 7-day midpoint regularity from a circular mean so it's a
true circular delta — frame-invariant (independent of --tz / ds→clock phase) and
correct across the midnight wrap, where the naive arithmetic mean broke
(Bugbot: sleep timing mixes timezones)
- dashboard: on an /api/summary {error} or fetch failure, clear every panel's
loading skeleton (not just the digest) so the page reads as errored instead of
stuck mid-load (Bugbot: summary errors leave skeletons)
- cached_summary re-stats oura.db / profile.json after the (slow) build and only stores the result if they didn't change underneath it, so a sync that lands during a build can't be masked by a stale cache entry; also guards against a slower concurrent build clobbering a fresher entry (Bugbot: summary cache stale after sync) - startup line now reads "open_oura dashboard running — open http://… in your browser" instead of the arrow-y debug-looking string (Bugbot: wrong dashboard startup message)
…atest, matching %) Both the vitals tiles and the digest now read a single vital_stat: latest is the *most recent night's* value (None if that night had no sample, so an older night is never shown as current), baseline is the mean of the prior nights, and the percent change comes from one shared helper. - latest no longer slides to an older night when the newest night lacks samples (Bugbot: vitals "latest" skips empty nights) - the digest HRV % is now the exact value the tile shows, including the zero-baseline guard (no fragment), instead of d/base.max(1.0) (Bugbot: digest HRV percent mismatches tiles)
- Advanced & debugging panel: export the 16-byte ring key (copy / .key / QR) to set up another device without re-pairing, or import one (paste / .key / QR), written to --key-file (0600, lowercased, validated as 32 hex chars). New GET/POST /api/ring-key behind the existing CSRF + loopback guards. - capability toggles now reflect the *real* on-ring feature mode: each sync snapshots SetFeatureMode status to a gitignored feature_modes.json next to oura.db; the panel falls back to "events seen recently" until a mode is captured. - docs: document the advanced panel, key export/import, the feature-modes snapshot, the JSON API + CSRF/loopback guards, and the cached/parallel/batched summary.
…etrics tools/fit_scores_all.py extends the Sleep two-layer approach (weights × contributor curves) to Readiness and Activity, engineering lag-1 / trailing-mean features for the "today vs personal baseline" contributors. Recovered weights confirmed for all three (ceiling R²=0.84–0.998); end-to-end Sleep R²=0.97, with the Readiness/Activity gap isolated to baseline-relative / multi-day-load contributors that need accumulated history. docs/algorithms written up accordingly.
…ock end - security: GET /api/ring-key discloses the ring auth key, so it now requires the same-origin X-Oura-Dash header like the mutating endpoints — a cross-origin page (or DNS-rebind attempt, already blocked by the Host guard) can't read the key; only the dashboard's own page can. fetchRingKey() sends the header. - actogram: display the API's wall-clock `end` string (which already wraps past midnight) instead of formatting a numeric endTrue that produced invalid "24:xx" times; the day-clamped endMin is kept only for bar geometry.
Compute the 7-day midpoint regularity with a circular mean + circular distance on the 24h clock, like tools/score_sleep.py, so a schedule that straddles a wrap isn't scored as wildly irregular. Equivalent to the previous arithmetic mean on normal (contiguous) bedtimes, so the calibration is unchanged.
…n them) A successful /api/feature toggle now writes the new mode (0 off / 1 automatic) into feature_modes.json right away, and the summary cache keys on that file's mtime too — so a reload shows the new capability state instead of the pre-toggle one held until the next sync re-snapshots. (Bugbot: capability toggles stale until sync)
- the "Steps" stream now counts both real_step_event_feature_1 and _2 (paired halves), and "Skin temp" counts sleep_temp_event alongside temp_event — matching the events the per-night stats already consume (Bugbot: steps/skin-temp streams undercount) - load() carries a sequence id and drops a response if a newer load() has started, so a slow earlier /api/summary can't overwrite fresher vitals/CVA/digest after a sync or profile save (Bugbot: overlapping summary loads race)
…ura-ffi Move build_summary and its helpers out of the CLI into a reusable `oura-summary` crate behind a `ModelRunner` trait (the dashboard supplies a Python runner; a native client can supply an on-device one). Add `oura-core` (UniFFI-exported protocol/summary surface) and `oura-ffi` (C-ABI) so the same tested Rust logic backs the native app. dashboard.rs now just wires the Python runner + HTTP server; no behavior change.
apps/ios: SwiftUI client (OuraApp) + an on-device model spike, built on OuraCore.xcframework generated from oura-core via UniFFI. Build outputs (the xcframework, Xcode build/ dirs, bundled libtorch, *.db) are gitignored and regenerated by the build scripts — only source + generated bindings are tracked.
…lines, docs Add score_readiness.py and the calibrate_scores/export_mobile/build_daily tooling that turns raw metrics into the daily Sleep/Readiness/Activity summaries with personal baselines, and document the approach (daily-summaries-and-baselines.md).
Put `pub mod ble` and the btleplug `From` impl behind `#[cfg(feature = "ble")]` so oura-link (and oura-core/oura-ffi on top of it) can build without btleplug — BLE isn't available on iOS, where transport is supplied natively. The CLI keeps `ble` on by default, so its behavior is unchanged.
- write() now awaits the GATT write-with-response ack via a continuation resumed in didWriteValueFor, so the sequential Rust client knows a frame landed before sending the next (Bugbot: BLE write never awaits response) - connect(timeout:) actually uses the timeout — fails with .timedOut instead of hanging when no ring advertises or Bluetooth is off (Bugbot: connect timeout unused) - resume connect/write continuations on disconnect too, and take/resume them under a lock so the success and timeout paths can't double-resume (a crash). Class marked @unchecked Sendable accordingly.
…ript - BLETransport: store the connect-timeout as a DispatchWorkItem and cancel it in finishConnect, so a stale timer from a finished/replaced attempt can't abort a newer connection with a false timeout (Bugbot: stale BLE connect timers) - BLETransport: only report "connected" once every notify subscription is confirmed (didUpdateNotificationStateFor), so Rust sync can't start before inbound frames flow and miss early ring responses (Bugbot: connect OK before notify ready) - OuraApp: pass the phone's real UTC offset to summaryJson instead of a hardcoded 1, so labels/windows/digest match the wearer's clock (Bugbot: iOS timezone mismatch) - resume_libtorch_ios.sh: drive CMake directly like build_libtorch_ios.sh — PyTorch 2.9 removed scripts/build_ios.sh (Bugbot: resume script uses removed build)
- BLETransport: reject a second connect() while one is in flight (.busy) instead of overwriting the continuation and stranding the earlier caller (Bugbot) - BLETransport: honor the CoreBluetooth error in didDiscoverServices / didDiscoverCharacteristicsFor — fail the connect with it rather than proceeding on an incomplete characteristic set (Bugbot) - Info.plist: add NSBluetoothAlways/PeripheralUsageDescription, required for CoreBluetooth on device (Bugbot) - OuraApp: round the device UTC offset to the nearest whole hour (the offset is integer-hours across the web CLI, Python runners, and FFI) (Bugbot)
Connect fixes (iPhone scan stalled while desktop worked): - accept any device matching the proprietary Oura service filter even before its name arrives — the ADV packet is full, so the name only comes in the scan response, which a worn ring answers lazily; the old name gate could reject the ring's single coalesced report - scan with allow-duplicates so a late scan response is re-delivered - raise the connect budget 20s -> 50s (desktop allows 25s scan + 30s GATT) - fail fast when Bluetooth is off/unauthorized instead of burning the whole timeout; serial (not concurrent) CoreBluetooth delegate queue - timeouts now name the stage that stalled (scanning / GATT / subscribe) Diagnostics: RingDiag transcript (timestamped, tagged, 4000-line ring buffer, cleared per attempt) fed by every pipeline step — state changes, adv reports (deduped), characteristics + properties, subscribe errors, full frame hex both ways — with a live tail + "copy logs" button in the sync screen. os.log mirror uses privacy: .public so Console shows values instead of <private> on an untethered device. Also fix backwards pairing advice: the ring advertises reliably ON its charger, and the phone running the official app holds its single link.
…c refinements
DNA explorer — new `oura-dna` crate reads genome VCF/gVCF (`.vcf.gz`) and scores
curated single-SNP traits (`dna/catalog.json`) plus polygenic scores: illustrative
built-ins and real PGS Catalog scoring files fetched on demand from EBI. Rigorous:
strict effect+other-allele matching with strand-flip resolution and palindromic
exclusion, `weight_type` (OR/HR→ln), coverage stats; gVCF reference-block resolution
(single-pass per-chromosome cursor) so 30x WGS gets full coverage; strand-aware trait
interpretation. Served by `crates/oura-cli/src/dna.rs` → `dashboard/web/dna.{html,js,css}`.
Genomes read from a configurable directory (`--dna-files` / `$OURA_DNA_FILES`) and are
never committed; only the curated catalog is tracked.
blood panel — `/blood` lab-marker tracking (`crates/oura-cli/src/blood.rs` +
`dashboard/web/blood.{html,js,css}`).
Also: autonomic-recovery-by-stage (`oura-summary` + iOS `Reports.swift`), ring-sync /
BLE-discovery refinements (`oura-link`, `RingSync.swift`), epoch-aware time mapping
(`tools/epoch_time.py`), regenerated iOS FFI bindings.
Personal data — genomes, `oura.db` backups, and working plan drafts — stays gitignored.
Unfiltered advertisement scan that lists every nearby BLE advertiser (Oura ones flagged). `oura scan` filters on the ring's service and only says "not found", which can't tell apart: Bluetooth off (radio sees nothing), the ring's single link held elsewhere / asleep / flat (radio sees other devices), or the ring present via its charging case but itself silent. This example pins the failure mode — it's what identified the official Android app holding the link during the sync-debug session. Uses the shared OURA_SERVICE constant; current-thread runtime.
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dbf256dcb5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let _ = self | ||
| .request_tag(&protocol::req_get_event_ack(start), 0x11) | ||
| .await; |
There was a problem hiding this comment.
If SQLite fails while cmd_sync is inserting a batch, its on_event callback records db_err and then stops storing later events, and on_batch returns without persisting the cursor; however drain_events still sends this ring-side ack before the caller can see that error. In a disk-full/DB-locked mid-batch sync, the ring is told that all events through start were consumed while they were not written to SQLite and the local cursor remains old, so a retry can permanently miss data the ring has already acknowledged.
Useful? React with 👍 / 👎.
| let ms = match self.last_ms { | ||
| Some(prev) if encoded_ts < 100_000_000 => prev.saturating_add(encoded_ts), | ||
| _ => encoded_ts, |
There was a problem hiding this comment.
For ExtGetEvent syncs on a recently rebooted ring, absolute timestamps can be below 100_000_000 ms, but this branch treats every such value after the first packet as a delta from the previous event. Since the parser's own format says each completed envelope starts with an absolute timestamp and only subsequent events in that envelope are deltas, carrying last_ms across envelopes inflates ring_timestamp and the sync cursor for those boots, corrupting event ordering and later resume ranges.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d409f9ecad
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| REPO="${CI_PRIMARY_REPOSITORY_PATH:-$PWD}" | ||
|
|
||
| # OuraCore.xcframework (device + sim) from the committed UniFFI bindings | ||
| bash "$REPO/apps/ios/build-xcframework.sh" |
There was a problem hiding this comment.
If this repo is built by Xcode Cloud, adding ci_scripts/ci_post_clone.sh makes the post-clone phase fail because this split removes the apps/ tree, yet the script still invokes apps/ios/build-xcframework.sh and later cds into apps/ios/OuraApp. I checked the tree with rg --files | rg '^apps/' and there are no matching files, so the CI hook is now guaranteed to exit before any Rust checks run.
Useful? React with 👍 / 👎.
| if csv: | ||
| cmd += ["--csv", csv] |
There was a problem hiding this comment.
When build_daily.py is run with its documented --csv option, this appends --csv <file> to the score_sleep.py subprocess, but score_sleep.py's argparse setup does not register a --csv argument. That makes every sleep-score subprocess exit with an argparse error, sleep_metrics() returns None, and the daily summary is silently rebuilt with zero nights.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9cc3170050
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| print(f"daily_summary: {len(rows)} night(s) written to {db.name}") | ||
| for d in rows: | ||
| rb = f"hrvσ={d['hrv_avg']:.0f}/{d['hrv_baseline']:.0f}" if d['hrv_baseline'] else f"hrv={d['hrv_avg']:.0f}/—" |
There was a problem hiding this comment.
If a synced night has no decoded HRV/IBI-derived hr_bpm data, hrv_avg remains None (the code explicitly stores None when rmssd is empty), but this status line formats it with :.0f. In that valid sparse-data case the script writes daily_summary and then exits with TypeError, making tools/build_daily.py look failed and breaking callers that require a zero exit status.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe27a2db48
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
@codex review for serious correctness regressions in Ring 5 history-envelope decoding, cursor checkpointing, and the physical regression-vector behavior. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e3ca846981
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// gives no progress reporting. A few thousand events (~1 min of transfer) keeps | ||
| /// the per-batch round-trip overhead negligible while bounding what a drop can | ||
| /// lose and yielding regular `bytes_left` progress updates. | ||
| const EXT_BATCH_MAX_EVENTS: u16 = 4096; |
There was a problem hiding this comment.
With Ring 5 extended sync this new 4096-event cap makes cursor checkpointing lossy when the ring cuts a batch between events that share the same decisecond. The physical 0x7e/0x7f real-step pairs in the added regression vector are only 1 ms apart, so if 0x7e is the last event in a capped batch, validate_batch advances to max_ts + 1 and the next ExtGetEvent starts at (T+1)*100 ms, skipping the 0x7f event at T*100+1 ms before it is ever fetched. Either avoid partial extended batches or checkpoint with enough precision/overlap to include the rest of the current decisecond.
Useful? React with 👍 / 👎.
|
|
||
| def build_epochs(pairs): | ||
| """pairs: iterable of (ds, captured_unix). Returns list of [min_ds, max_ds, anchor_unix].""" | ||
| order = sorted((cu, ds) for ds, cu in pairs) |
There was a problem hiding this comment.
For reboot detection this helper needs the real sync order, but sorting by (captured_unix, ds) reorders all rows captured in the same second by timestamp. In a physical drain with thousands of rows sharing captured_unix, a reboot reset places the low post-reboot ds values before the high pre-reboot values, so the backward jump is hidden and the model runners merge distinct boot epochs under one wall-clock anchor. Include the SQLite insertion id/order (or accept an already captured_unix, id ordered stream) instead of sorting ties by ds.
Useful? React with 👍 / 👎.
|
@codex review Fixes the exact iOS Ring 5 sync trace in c6674ed:
Validated with cargo test --workspace and clippy -D warnings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c6674ede0d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 5. SyncTime using the app-style counter packet (`12 09 ... f6`) where supported. | ||
| 6. (optional) firmware / product / battery for metadata. | ||
| 7. DataFlush, then drain history events from the persisted cursor; persist each | ||
| event, ack with `GetEvent(max_events=0)`, and advance the cursor; stop when |
There was a problem hiding this comment.
When a client follows this minimal recipe on Ring 5 with the extended ExtGetEvent path, it will still send the legacy GetEvent(max_events=0) ACK after persisting events. That is the ACK pattern the implementation now avoids because it can make late history frames race with the next flush; the documentation should distinguish that only the legacy GetEvent path is ACKed, matching the !use_extended guard in OuraClient::drain_events.
Useful? React with 👍 / 👎.
Readers (iOS models, summary_json) and the per-event sync writer share oura.db; the default rollback journal made every autocommit insert take an exclusive lock, so a concurrent whole-table read could come back truncated or kill the writer. WAL gives readers a consistent snapshot during the drain, and a 5 s busy_timeout absorbs the remaining contention. The pragma failure is tolerated for the read-only seed DB bundled with the iOS app.
1 participant
Summary
Splits the app/product layer out of
open_ouranow thatopen_healthexists at https://github.com/Th0rgal/open_health.open_ourakeeps the reusable Oura protocol/library surface:oura-protocoloura-linkoura-storeoura-analysisouraCLI commandsThe moved surfaces now live in
open_health:Sync correctness and maintainability
(captured_unix, insertion id)order so ring reboots remain detectable even when thousands of rows share a capture secondNotes
The
open_healthrepository was created with history preserved from this repository usinggit-filter-repo, then bootstrapped to consume core crates fromopen_ouraas Git dependencies.Validation
cargo test --workspaceopen_healthbuild 8 in device and simulator builds