diff --git a/.codex/prompts/ruview-advanced.md b/.codex/prompts/ruview-advanced.md new file mode 100644 index 0000000000..9f585d913c --- /dev/null +++ b/.codex/prompts/ruview-advanced.md @@ -0,0 +1,15 @@ +# /ruview-advanced — advanced RuView capabilities + +Drive RuView's research-grade / multi-node features. Topic: `$ARGUMENTS` (one of `multistatic`, `cross-viewpoint`, `tomography`, `field-model`, `intention`, `adversarial`, `security`; if empty, ask). + +- **multistatic** (ADR-029) — treat every WiFi link in range (incl. neighbours' APs) as a bistatic radar pair, then fuse. `v2/crates/wifi-densepose-signal/src/ruvsense/multistatic.rs` (attention-weighted fusion, geometric diversity), `phase_align.rs` (iterative LO phase-offset, circular mean), `multiband.rs`, `coherence.rs` / `coherence_gate.rs` (Z-score scoring; Accept / PredictOnly / Reject / Recalibrate). +- **cross-viewpoint** (ADR-016 viewpoint module) — combine 2+ nodes geometrically. `v2/crates/wifi-densepose-ruvector/src/viewpoint/`: `attention.rs` (CrossViewpointAttention, GeometricBias, softmax with `G_bias`), `geometry.rs` (GeometricDiversityIndex, Cramér–Rao bounds, Fisher Information), `coherence.rs` (phase-phasor coherence, hysteresis gate), `fusion.rs` (MultistaticArray aggregate root). Explore geometry first: `node scripts/mesh-graph-transformer.js`, `node scripts/deep-scan.js`. +- **tomography** — `ruvsense/tomography.rs` reconstructs a voxel occupancy grid via an ISTA L1 solver (sparse — most voxels empty); pair with cross-viewpoint geometry for through-wall volumetric imaging. RuVector solver crates back the 114→56 subcarrier sparse interpolation. +- **field-model** (ADR-030) — `ruvsense/field_model.rs` builds an SVD eigenstructure of the room, persists it (RVF, ideally on a Cognitum Seed); new frames are projected against it and the residual is the perturbation. Survives restarts; answers "what's different from the empty-room baseline?" +- **intention** — `ruvsense/intention.rs`, pre-movement lead signals 200–500 ms ahead. +- **adversarial** — `ruvsense/adversarial.rs`, rejects physically impossible signals + cross-checks multi-link consistency. +- **security** (ADR-032, multistatic mesh hardening) — using neighbour APs and pooling links across a mesh expands the attack surface. Mitigations: `adversarial.rs` + `coherence_gate.rs` quarantine (Reject / Recalibrate) + Ed25519 witness chain (ADR-028). Run a security review (`docs/security-audit-wasm-edge-vendor.md`); see `/ruview-verify`. + +Also relevant: ADR-031 (sensing-first RF mode), ADR-081 (adaptive CSI mesh firmware kernel), ADR-083 (per-cluster π compute hop), ADR-095/096 (on-ESP32 temporal modeling, sparse GQA). + +Validate: `cd v2 && cargo test -p wifi-densepose-signal --no-default-features && cargo test -p wifi-densepose-ruvector --no-default-features`, then `cd .. && python archive/v1/data/proof/verify.py`. diff --git a/.codex/prompts/ruview-app.md b/.codex/prompts/ruview-app.md new file mode 100644 index 0000000000..2e3e2db3d7 --- /dev/null +++ b/.codex/prompts/ruview-app.md @@ -0,0 +1,13 @@ +# /ruview-app — run a RuView sensing application + +Run a RuView application. Which one: `$ARGUMENTS` (one of `presence`, `vitals`, `pose`, `sleep`, `environment`, `mat`, `pointcloud`, or a novel-RF app name; if empty, show the catalogue and ask). + +- **presence / vitals / pose / environment** → `cd v2 && cargo run -p wifi-densepose-sensing-server` against a live ESP32 sink, or the Docker demo (`docker run -p 3000:3000 ruvnet/wifi-densepose:latest`) for simulated CSI. For environment also `-- --model model.rvf --build-index env`. Vitals: breathing 6–30 BPM (bandpass 0.1–0.5 Hz), heart rate 40–120 BPM (bandpass 0.8–2.0 Hz), `wifi-densepose-vitals` crate (ADR-021). Pose: 17 COCO keypoints via WiFlow (ADR-059 live pipeline) — train for accuracy (`/ruview-train`). +- **sleep** → `examples/sleep/` + `node scripts/apnea-detector.js` (sleep-stage classification, apnea screening). +- **mat** (Mass Casualty Assessment — disaster survivor detection) → `wifi-densepose-mat` crate, `docs/wifi-mat-user-guide.md`. +- **pointcloud** → `python scripts/mmwave_fusion_bridge.py` (camera depth via MiDaS + WiFi CSI + mmWave radar → unified spatial model, ~22 ms, 19K+ pts/frame; ADR-094). +- **novel RF** → `scripts/passive-radar.js`, `material-classifier.js`, `device-fingerprint.js`, `mincut-person-counter.js`, `gait-analyzer.js` (ADR-077/078). + +No hardware? Fall back to the Docker demo or `python examples/ruview_live.py`. Visualisers: `node scripts/csi-spectrogram.js`, `node scripts/csi-graph-visualizer.js`. + +Help me pick: through-wall → presence/activity (≤5 m depth); stationary subject → vitals/sleep; need skeletons → pose (train it); search & rescue → MAT; best spatial accuracy → 2+ ESP32 nodes + cross-viewpoint fusion (`v2/crates/wifi-densepose-ruvector/src/viewpoint/`), optionally + Cognitum Seed. Examples: `examples/{environment,medical,sleep,stress,happiness-vector}/`. diff --git a/.codex/prompts/ruview-flash.md b/.codex/prompts/ruview-flash.md new file mode 100644 index 0000000000..208e07abdc --- /dev/null +++ b/.codex/prompts/ruview-flash.md @@ -0,0 +1,17 @@ +# /ruview-flash — build + flash ESP32 firmware + +Build and flash RuView ESP32 firmware. Variant + port: `$ARGUMENTS` (default `8mb`, port `COM8`). + +1. **Variant.** `8mb` → ensure it builds from `firmware/esp32-csi-node/sdkconfig.defaults.template` (no mock — real WiFi CSI). `4mb` → `cp firmware/esp32-csi-node/sdkconfig.defaults.4mb firmware/esp32-csi-node/sdkconfig.defaults` first (display disabled, dual OTA via `partitions_4mb.csv`). `heltec` → `sdkconfig.defaults.heltec_n16r2`. +2. **Build (Windows).** ESP-IDF v5.4 does NOT work under Git Bash; `cmd.exe /C` hangs. Use the Espressif Python venv as a subprocess with `MSYSTEM*` env vars stripped — the exact command is in `CLAUDE.local.md` (`[python, idf_py, 'build']`, cwd = `firmware/esp32-csi-node`). Outputs in `firmware/esp32-csi-node/build/{bootloader/bootloader.bin, partition_table/partition-table.bin, esp32-csi-node.bin, ota_data_initial.bin}`. +3. **Flash.** Same subprocess with `[python, idf_py, '-p', 'COM8', 'flash']`, or: + ``` + python -m esptool --chip esp32s3 --port COM8 --baud 460800 write_flash \ + 0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \ + 0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \ + 0xf000 firmware/esp32-csi-node/build/ota_data_initial.bin \ + 0x20000 firmware/esp32-csi-node/build/esp32-csi-node.bin + ``` +4. **Confirm.** Serial monitor via pyserial on `COM8` @ 115200 (NOT `idf.py monitor` — it hangs in a subprocess). Then `cd v2 && cargo run -p wifi-densepose-sensing-server` — frames should arrive. If not: re-run `/ruview-provision`, match the AP channel, drop any `--filter-mac`. + +Never test in mock mode — the Kconfig fall-threshold bug only showed up with real CSI. diff --git a/.codex/prompts/ruview-provision.md b/.codex/prompts/ruview-provision.md new file mode 100644 index 0000000000..4e111c570b --- /dev/null +++ b/.codex/prompts/ruview-provision.md @@ -0,0 +1,25 @@ +# /ruview-provision — provision an ESP32 sensing node + +Write NVS config to a RuView ESP32 node. Args: `$ARGUMENTS` (expect `--port`, `--ssid`, `--password`, `--target-ip`, optional `--channel`, `--filter-mac`). Default port `COM8`. + +First get the authoritative flag list: `python firmware/esp32-csi-node/provision.py --help` (on Windows prefix `PYTHONUTF8=1 PYTHONIOENCODING=utf-8` — the help text has non-ASCII and crashes under cp1252). Then run: + +``` +python firmware/esp32-csi-node/provision.py --port COM8 \ + --ssid "" --password "" --target-ip --target-port 5005 --node-id <0-255> \ + [--channel ] [--filter-mac ] [--hop-channels 1,6,11 --hop-dwell 200] \ + [--tdm-slot --tdm-total ] [--edge-tier 0|1|2] [--pres-thresh 50] [--fall-thresh 15000] \ + [--vital-win 300] [--vital-int 1000] [--subk-count 32] \ + [--seed-url http://10.1.10.236 --seed-token --zone lobby] [--swarm-hb 30] [--swarm-ingest 5] [--dry-run] +``` + +Trade-offs: +- `--channel ` pins the node to one WiFi channel (set it to the AP's channel). Omit it and pass `--hop-channels 1,6,11` for the firmware's multi-band hopping schedule (more sensing bandwidth, uses neighbour APs as illuminators; `--hop-dwell` ms per channel). +- `--filter-mac ` restricts CSI capture to one transmitter (cleaner signal); omit for all transmitters (more data, more noise). +- `--edge-tier` 0/1/2 = off / stats / vitals (ADR-041). `--tdm-slot`/`--tdm-total` slot a multi-node mesh. `--fall-thresh 15000` ≈ 15.0 rad/s² (raise to cut false falls). + +⚠️ **Issue #391:** flashing rewrites the *entire* `csi_cfg` NVS namespace — every key not on the CLI is erased. Pass the full set you want; warn before re-provisioning a working node. `--dry-run` builds the NVS binary without flashing; `--force-partial` allows config without WiFi creds (knowingly). + +Fleet provisioning: `python scripts/generate_nvs_matrix.py` (subprocess-first — the `esp_idf_nvs_partition_gen` API changed across versions). + +Verify: serial monitor (pyserial on `COM8`, 115200) should show `adaptive_ctrl` ticks + `csi_collector: CSI cb #… len=128 rssi=… ch=…` lines; the sink `cd v2 && cargo run -p wifi-densepose-sensing-server` should report incoming UDP frames if `--target-ip` points at this host. If no frames: wrong channel, MAC filter too tight, target-ip not this host, or WiFi creds wrong — re-run with corrected args. diff --git a/.codex/prompts/ruview-rvagent.md b/.codex/prompts/ruview-rvagent.md new file mode 100644 index 0000000000..3dfcda1eda --- /dev/null +++ b/.codex/prompts/ruview-rvagent.md @@ -0,0 +1,54 @@ +# ruview-rvagent — explore rvAgent + RVF agentic flows for RuView + +You are helping the operator explore or prototype the integration of `vendor/ruvector/crates/rvAgent/` (a production Rust AI-agent framework) with RuView's existing sensing pipeline (`v2/crates/wifi-densepose-*`) and the RVF cognitive container format (`v2/crates/wifi-densepose-sensing-server/src/rvf_container.rs`). + +## Live MCP server: `@ruvnet/rvagent` v0.1.0 + +The TypeScript MCP server (`tools/ruview-mcp/`, published as `@ruvnet/rvagent`) is live on npm and exposes `bfld_last_scan`, `bfld_subscribe`, `presence_now`, `vitals_get_breathing`, `vitals_get_heart_rate`, `vitals_get_all`, `vitals_fetch`. Add to a Codex MCP config: + +```json +{ + "mcpServers": { + "rvagent": { + "command": "npx", + "args": ["-y", "@ruvnet/rvagent"], + "env": { "RVAGENT_SENSING_URL": "http://localhost:3000" } + } + } +} +``` + +This is the operator-facing tool surface; the Rust crate below remains the substrate for deeper RVF-aware agentic flows. + +## Trigger phrasing + +- "wire rvAgent into RuView" +- "I want a queen agent that fans out to cog-pose-estimation and cog-bfld" +- "persist agent decisions in the same witness bundle as sensing events" +- "how do I keep agent outputs class-3 compliant?" + +## What to read first + +1. `docs/research/rvagent-rvf-integration/README.md` — full integration thesis, open questions, next steps. +2. `vendor/ruvector/crates/rvAgent/README.md` — what rvAgent ships (8 crates, 14 middlewares). +3. `vendor/ruvector/crates/rvAgent/.ruv/agents/rvagent-queen.md` — queen-agent persona that coordinates cog subagents. +4. `v2/crates/wifi-densepose-bfld/src/{event.rs,pipeline_handle.rs}` — the BFLD event surface and the operator-facing handle that an agent would call. +5. `v2/crates/wifi-densepose-sensing-server/src/rvf_container.rs` — segment types; `SEG_AGENT_STATE = 0x08` and `SEG_DECISION = 0x09` are the proposed additions. + +## Three shippable touchpoints (each independent) + +1. **RVF wire** — add `SEG_AGENT_STATE` + `SEG_DECISION` segments so rvAgent and RuView sessions can interleave in one blob (witness-bundle covers both halves). +2. **Tool shim** — `BfldEvent::to_json()` already exists; wrap as `rvagent_tools::ToolOutput`. +3. **Cog subagents** — register `cog-pose-estimation`, `cog-person-count`, `cog-ha-matter`, (proposed) `cog-bfld` under the queen via the `Subagent` trait. + +## Open questions to surface + +- Is `vendor/ruvector/crates/rvAgent/` on the v2 workspace path? +- Sync ↔ async adapter location (BFLD `Publish` is sync; rvAgent backends are tokio). +- Privacy-class composition — does `rvagent-middleware::sanitizer` consume `BfldEvent::privacy_class`? +- Soul Signature ↔ `SoulMatchOracle` bridge (ADR-121 §2.6). +- Should `BfldPipelineHandle::send` land as a public MCP tool via `rvagent-mcp`? + +## Suggested next action + +Draft ADR-124 — "rvAgent + RVF integration for RuView agentic flows" — capturing segment assignments, cog-subagent contract, and privacy-class composition. Land **before** scaffolding `v2/crates/wifi-densepose-agent`. diff --git a/.codex/prompts/ruview-start.md b/.codex/prompts/ruview-start.md new file mode 100644 index 0000000000..61f9a93ee3 --- /dev/null +++ b/.codex/prompts/ruview-start.md @@ -0,0 +1,11 @@ +# /ruview-start — onboard onto RuView + +Help me get started with RuView (WiFi-DensePose). Path: `$ARGUMENTS` (one of `docker`, `build`, `hardware`; if empty, ask which hardware I have). + +- **docker** (no hardware): `docker pull ruvnet/wifi-densepose:latest && docker run -p 3000:3000 ruvnet/wifi-densepose:latest`, then open http://localhost:3000 (simulated CSI, full UI). +- **build** (from source): `cd v2 && cargo test --workspace --no-default-features`, then `cd .. && python archive/v1/data/proof/verify.py` (expect `VERDICT: PASS`). Single-crate sanity: `cargo check -p wifi-densepose-train --no-default-features`. +- **hardware** (ESP32-S3/C6): use `/ruview-flash` then `/ruview-provision`, then `cd v2 && cargo run -p wifi-densepose-sensing-server` to consume the UDP CSI stream. Also: `node scripts/rf-scan.js --port 5006`, `node scripts/snn-csi-processor.js --port 5006`. + +Warn me about: ESP32-C3 / original ESP32 are unsupported (single-core); one node = limited spatial resolution (use 2+ or add a Cognitum Seed); camera-free pose is modest — camera-supervised training reaches 92.9% PCK@20 (ADR-079); no cloud/cameras/internet needed. + +Then point me at next steps: `/ruview-app`, `/ruview-train`, `/ruview-verify`, and the configuration workflow (sdkconfig variants, NVS provisioning, edge modules, mesh, Cognitum Seed). Reference `README.md`, `docs/user-guide.md`, `docs/build-guide.md`, `docs/TROUBLESHOOTING.md`, `examples/`. diff --git a/.codex/prompts/ruview-train.md b/.codex/prompts/ruview-train.md new file mode 100644 index 0000000000..405ebf53e6 --- /dev/null +++ b/.codex/prompts/ruview-train.md @@ -0,0 +1,14 @@ +# /ruview-train — train a RuView model + +Train / evaluate / publish a RuView model. Track: `$ARGUMENTS` (one of `camera-free`, `camera-supervised`, `embeddings`, `domain-gen`, `snn`, `gpu`; if empty, ask). + +- **camera-free** (WiFlow pose, no labels): `cd v2 && cargo run -p wifi-densepose-sensing-server -- --pretrain --dataset data/csi/ --pretrain-epochs 50`, then `-- --train --dataset data/mmfi/ --epochs 100 --save-rvf model.rvf`. ~84 s on M4 Pro, modest accuracy. Bench `node scripts/benchmark-wiflow.js`, eval `node scripts/eval-wiflow.js`. +- **camera-supervised** (ADR-079, 92.9% PCK@20, ~19 min): `python scripts/collect-ground-truth.py` (MediaPipe landmarks; needs `data/pose_landmarker_lite.task`), `python scripts/collect-training-data.py` (CSI capture), `node scripts/align-ground-truth.js` (timestamp align), then `cd v2 && cargo run -p wifi-densepose-sensing-server -- --train --dataset data/paired/ --epochs --save-rvf model.rvf`, eval `node scripts/eval-wiflow.js` (reports PCK@20). +- **embeddings** (AETHER ADR-024 / spectrogram ADR-076): `wifi-densepose-train` + `wifi-densepose-ruvector`; `-- --model model.rvf --embed`, `-- --model model.rvf --build-index env`. 171K emb/s on M4 Pro. +- **domain-gen** (MERIDIAN ADR-027): domain-generalization options in the training pipeline + `ruview_metrics`. +- **snn** (local env adaptation, <30 s): `node scripts/snn-csi-processor.js --port 5006`; `docs/tutorials/cognitum-seed-pretraining.md`; ADR-084/085 (RaBitQ), ADR-086 (novelty gate). +- **gpu**: `gcloud auth login && gcloud config set project cognitum-20260110`, then `bash scripts/gcloud-train.sh --dry-run` (smoke), `bash scripts/gcloud-train.sh --gpu l4 --hours 2` (proto, ~$0.80/hr), `bash scripts/gcloud-train.sh --gpu a100 --config scripts/training-config-sweep.json` (~$3.60/hr), `bash scripts/gcloud-train.sh --sweep` (full sweep). VM auto-deletes unless `--keep-vm`. Local Mac: `bash scripts/mac-mini-train.sh`. Bench: `python scripts/benchmark-model.py`. + +Data: `data/recordings/` raw CSI · `data/csi/` pretrain · `data/mmfi/` MM-Fi · `data/paired/` camera↔CSI · `data/ground-truth/` MediaPipe · `models/` artifacts. Record more: `python scripts/record-csi-udp.py`. + +After training: `cd v2 && cargo test --workspace --no-default-features`, `cd .. && python archive/v1/data/proof/verify.py` (VERDICT: PASS). Publish: `python scripts/publish-huggingface.py` (or `.sh`; `docs/huggingface/`). Then run `/ruview-verify`. diff --git a/.codex/prompts/ruview-verify.md b/.codex/prompts/ruview-verify.md new file mode 100644 index 0000000000..7572647bfe --- /dev/null +++ b/.codex/prompts/ruview-verify.md @@ -0,0 +1,12 @@ +# /ruview-verify — run the RuView trust pipeline + +Verify a RuView build. Scope: `$ARGUMENTS` (one of `tests`, `proof`, `bundle`, `all`; default `all`). + +1. **tests** — `cd v2 && cargo test --workspace --no-default-features` → must be 1,400+ passed, 0 failed (~2 min). Single-crate: `cargo test -p wifi-densepose-signal --no-default-features`, etc. +2. **proof** — `cd .. && python archive/v1/data/proof/verify.py` → must print `VERDICT: PASS`. If a hash mismatch from a legitimate numpy/scipy bump: `python archive/v1/data/proof/verify.py --generate-hash`, then re-run. Optional: `cd archive/v1 && python -m pytest tests/ -x -q`. +3. **bundle** — `bash scripts/generate-witness-bundle.sh` produces `dist/witness-bundle-ADR028-.tar.gz` (WITNESS-LOG-028.md, ADR-028 audit, proof, rust test log, firmware hash manifest, crate versions, VERIFY.sh). Then `cd dist/witness-bundle-ADR028-*/ && bash VERIFY.sh` → must be 7/7 PASS. +4. **all** — do 1→3 in order. + +If this follows a code change, walk the pre-merge checklist from `CLAUDE.md`: Rust tests pass; Python proof passes; README updated if scope changed; CLAUDE.md updated if scope changed; CHANGELOG `[Unreleased]` entry; `docs/user-guide.md` updated if new data sources/CLI flags/setup; ADR count bumped in README if a new ADR added; witness bundle regenerated if tests/proof hash changed; Docker image rebuilt only if Dockerfile/deps/runtime changed; crate publishing only if a published crate's public API changed (publish in dependency order — see CLAUDE.md); `.gitignore` updated for new artifacts; security review for new hardware/network-boundary modules. + +For security-related changes also run `npx @claude-flow/cli@latest security scan`. QEMU firmware CI (ADR-061): local helpers `scripts/qemu-esp32s3-test.sh`, `qemu-mesh-test.sh`, `qemu-chaos-test.sh`, `install-qemu.sh`. diff --git a/DESIGN.md b/DESIGN.md new file mode 100644 index 0000000000..7959b8772e --- /dev/null +++ b/DESIGN.md @@ -0,0 +1,81 @@ +# Design + +## Source of truth +- Status: Active +- Last refreshed: 2026-06-04 +- Primary product surfaces: Desktop RuView UI under `ui/` +- Evidence reviewed: `ui/index.html`, `ui/style.css`, `ui/app.js` + +## Brand +- Personality: Vintage Swiss graphic design, precise, technical, poster-like. +- Trust signals: Live hardware status, explicit unavailable states, clear typographic hierarchy. +- Avoid: Synthetic demo polish, soft SaaS gradients, rounded card-heavy presentation. + +## Product goals +- Goals: Make live sensing state readable at a glance and visually distinctive. +- Non-goals: Marketing landing page, fake telemetry, decorative illustration. +- Success signals: Status, node, metric, and sensing surfaces remain scannable on desktop and mobile. + +## Personas and jobs +- Primary personas: Local operator, hardware debugger, demo observer. +- User jobs: Confirm hardware data availability, inspect person counts, monitor stream health. +- Key contexts of use: Steam Deck or desktop browser near live hardware. + +## Information architecture +- Primary navigation: Tabbed sections for dashboard, hardware, live demo, architecture, performance, applications, sensing, and training. +- Core routes/screens: `index.html`, `pose-fusion.html`, `observatory.html`. +- Content hierarchy: System truth first, technical dashboards second, explanatory material lower. + +## Design principles +- Principle 1: Use grid, type, and hard rules as the primary visual system. +- Principle 2: Live/unavailable data states must be more important than decorative copy. +- Tradeoffs: The style is intentionally flatter and sharper than modern card UI. + +## Visual language +- Color: Warm paper, black ink, Swiss red primary, blue live accent, yellow warning/accent. +- Typography: Helvetica-style sans serif with mono labels for machine/status metadata. +- Spacing/layout rhythm: 48px grid background, boxed modular panels, dense but readable status grids. +- Shape/radius/elevation: Square corners, hard borders, no shadows. +- Motion: Minimal; avoid motion that implies live data when unavailable. +- Imagery/iconography: Prefer geometric color blocks and typographic markers over emoji decoration. + +## Components +- Existing components to reuse: Header, nav tabs, dashboard panels, status cells, stats, mobile drawer. +- New/changed components: Swiss theme override layer in `ui/style.css`. +- Variants and states: Healthy/live blue, warning yellow, error red, unavailable neutral. +- Token/component ownership: CSS custom properties in `ui/style.css`. + +## Accessibility +- Target standard: Practical WCAG AA contrast for core text and status surfaces. +- Keyboard/focus behavior: Preserve visible focus rings. +- Contrast/readability: Black on warm paper, white on black/red headers. +- Screen-reader semantics: Preserve existing roles and live regions. +- Reduced motion and sensory considerations: Respect existing reduced-motion rules. + +## Responsive behavior +- Supported breakpoints/devices: Desktop browser and narrow Steam Deck/mobile widths. +- Layout adaptations: Header collapses to one column, stat grids reduce to two then one column. +- Touch/hover differences: Hover color is decorative only; active state remains explicit. + +## Interaction states +- Loading: Existing text/status placeholders remain visible. +- Empty: Unavailable hardware states are explicit, not masked by mock visuals. +- Error: Red hard-rule treatment. +- Success: Blue live treatment. +- Disabled: Existing opacity treatment. +- Offline/slow network: Neutral unavailable state with no synthetic frames. + +## Content voice +- Tone: Direct, technical, terse. +- Terminology: Use live, unavailable, hardware, packets, persons. +- Microcopy rules: Do not imply simulated/demo data is real. + +## Implementation constraints +- Framework/styling system: Static HTML/CSS/JS served by `/home/deck/bin/ruview-ui-server.py`. +- Design-token constraints: Use existing CSS variables and append scoped overrides. +- Performance constraints: No new runtime dependencies. +- Compatibility constraints: Keep launcher-served static files browser-native. +- Test/screenshot expectations: CSS syntax check plus live launcher smoke check. + +## Open questions +- [ ] Whether secondary pages should receive a deeper layout-specific Swiss redesign beyond shared tokens. diff --git a/bin/ruview-ui-server.py b/bin/ruview-ui-server.py new file mode 100755 index 0000000000..14b36e0a0f --- /dev/null +++ b/bin/ruview-ui-server.py @@ -0,0 +1,1964 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import json +import os +import re +import shlex +import socket +import struct +import threading +import time +import zlib +from http.server import ThreadingHTTPServer, SimpleHTTPRequestHandler +from pathlib import Path +from urllib.parse import urlparse + +UI_DIR = Path(os.environ.get("RUVIEW_UI_DIR", "/home/deck/RuView/ui")).resolve() +PORT = int(os.environ.get("RUVIEW_UI_PORT", "3000")) +UDP_PORT = int(os.environ.get("RUVIEW_CARDPUTER_UDP_PORT", "5005")) +RUVIEW_ROOT = Path( + os.environ.get( + "RUVIEW_ROOT", + str(UI_DIR.parent if UI_DIR.name == "ui" else Path("/home/deck/RuView")), + ) +).resolve() +DATA_DIR = Path(os.environ.get("RUVIEW_DATA_DIR", str(RUVIEW_ROOT / "data"))).resolve() +RECORDINGS_DIR = Path(os.environ.get("RUVIEW_RECORDINGS_DIR", str(DATA_DIR / "recordings"))).resolve() +MODELS_DIR = Path(os.environ.get("RUVIEW_MODELS_DIR", str(DATA_DIR / "models"))).resolve() +GROUND_TRUTH_DIR = Path(os.environ.get("RUVIEW_GROUND_TRUTH_DIR", str(DATA_DIR / "ground-truth"))).resolve() +PAIRED_DIR = Path(os.environ.get("RUVIEW_PAIRED_DIR", str(DATA_DIR / "paired"))).resolve() +SCRIPTS_DIR = Path(os.environ.get("RUVIEW_SCRIPTS_DIR", str(RUVIEW_ROOT / "scripts"))).resolve() +STARTED = time.time() +LIVE_MAX_AGE_S = 5.0 +ADAPTIVE_STATE_MAX_AGE_S = 30.0 +FEATURE_STATE_MAX_AGE_S = 5.0 +BATTERY_MAX_AGE_S = 15.0 +RSSI_MAX_AGE_S = 5.0 +MAX_PERSONS = 4 +CSI_FRAME_MAGIC = 0xC5110001 +EDGE_VITALS_MAGIC = 0xC5110002 +EDGE_VITALS_FMT = " str: + return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()) + + +def _coerce_person_count(value, default: int = 0) -> int: + try: + count = int(value) + except (TypeError, ValueError): + return default + return max(0, min(MAX_PERSONS, count)) + + +def _active_person_count(cardputer: dict, feature_state: dict) -> int: + """Resolve the best current count from live edge packets.""" + node_counts = [] + for node in cardputer.get("nodes", []): + edge_vitals = node.get("edge_vitals") if node.get("edge_vitals_live") else None + if edge_vitals and edge_vitals.get("presence"): + node_counts.append(_coerce_person_count(edge_vitals.get("n_persons"), 1)) + + if node_counts: + return _coerce_person_count(max(node_counts)) + + edge_vitals = cardputer.get("edge_vitals") if cardputer.get("edge_vitals_live") else None + if edge_vitals and edge_vitals.get("presence"): + return _coerce_person_count(edge_vitals.get("n_persons"), 1) + + if cardputer.get("live") and feature_state.get("presence"): + return 1 + + return 0 + + +def _hardware_inference_state(cardputer: dict, feature_state: dict | None = None) -> dict: + """Summarize real hardware-derived inference availability without fabricating pose keypoints.""" + feature_state = feature_state or cardputer.get("feature_state") or {} + nodes = cardputer.get("nodes", []) + person_count = _active_person_count(cardputer, feature_state) + feature_live = bool(cardputer.get("feature_state_live")) or any( + bool(node.get("feature_state_live")) for node in nodes + ) + edge_vitals_live = bool(cardputer.get("edge_vitals_live")) or any( + bool(node.get("edge_vitals_live")) for node in nodes + ) + edge_feature_live = bool(cardputer.get("edge_feature_live")) or any( + bool(node.get("edge_feature_live")) for node in nodes + ) + hardware_live = bool(cardputer.get("live")) + inference_live = bool(hardware_live and (feature_live or edge_vitals_live or edge_feature_live or person_count > 0)) + + if feature_live: + source = "hardware_feature_state" + elif edge_vitals_live: + source = "hardware_edge_vitals" + elif edge_feature_live: + source = "hardware_edge_feature" + elif hardware_live: + source = "hardware_telemetry" + else: + source = "none" + + if inference_live: + message = f"Hardware inference live: {person_count} person{'s' if person_count != 1 else ''}" + elif hardware_live: + message = "Hardware stream live; waiting for inference packets" + else: + message = "No live hardware inference" + + return { + "live": inference_live, + "source": source, + "message": message, + "person_count": person_count, + "feature_state_live": feature_live, + "edge_vitals_live": edge_vitals_live, + "edge_feature_live": edge_feature_live, + } + + +def _parse_feature_state(data: bytes) -> dict | None: + if len(data) != RV_FEATURE_STATE_SIZE: + return None + try: + ( + magic, + node_id, + mode, + seq, + ts_us, + motion_score, + presence_score, + respiration_bpm, + respiration_conf, + heartbeat_bpm, + heartbeat_conf, + anomaly_score, + env_shift_score, + node_coherence, + quality_flags, + _reserved, + crc32, + ) = struct.unpack(RV_FEATURE_STATE_FMT, data) + except struct.error: + return None + if magic != RV_FEATURE_STATE_MAGIC: + return None + computed_crc = zlib.crc32(data[:-4]) & 0xFFFFFFFF + crc_valid = computed_crc == crc32 + presence_valid = bool(quality_flags & RV_QFLAG_PRESENCE_VALID) + return { + "packet_type": "rv_feature_state", + "node_id": node_id, + "mode": mode, + "seq": seq, + "ts_us": ts_us, + "motion_score": motion_score, + "presence_score": presence_score, + "respiration_bpm": respiration_bpm, + "respiration_conf": respiration_conf, + "heartbeat_bpm": heartbeat_bpm, + "heartbeat_conf": heartbeat_conf, + "anomaly_score": anomaly_score, + "env_shift_score": env_shift_score, + "node_coherence": node_coherence, + "quality_flags": quality_flags, + "presence_valid": presence_valid, + "respiration_valid": bool(quality_flags & RV_QFLAG_RESPIRATION_VALID), + "heartbeat_valid": bool(quality_flags & RV_QFLAG_HEARTBEAT_VALID), + "crc_valid": crc_valid, + "presence": crc_valid and presence_valid and presence_score >= 0.35, + } + + +def _parse_battery(data: bytes) -> dict | None: + if len(data) != EDGE_BATTERY_SIZE: + return None + try: + ( + magic, + node_id, + percent, + flags, + status, + millivolts, + _reserved, + timestamp_ms, + ) = struct.unpack(EDGE_BATTERY_FMT, data) + except struct.error: + return None + if magic != EDGE_BATTERY_MAGIC: + return None + valid = bool(flags & BATTERY_FLAG_VALID) and percent <= 100 and millivolts > 0 + charging = bool(flags & BATTERY_FLAG_CHARGING) + return { + "packet_type": "edge_battery", + "node_id": node_id, + "valid": valid, + "percent": percent if valid else None, + "millivolts": millivolts if valid else None, + "volts": round(millivolts / 1000.0, 3) if valid else None, + "charging": charging, + "status_code": status, + "status": BATTERY_STATUS_NAMES.get(status, "UNKNOWN"), + "ts_ms": timestamp_ms, + } + + +def _parse_edge_vitals(data: bytes) -> dict | None: + if len(data) != EDGE_VITALS_SIZE: + return None + try: + ( + magic, + node_id, + flags, + breathing_rate, + heartrate, + rssi, + n_persons, + motion_energy, + presence_score, + timestamp_ms, + _reserved2, + ) = struct.unpack(EDGE_VITALS_FMT, data) + except struct.error: + return None + if magic != EDGE_VITALS_MAGIC: + return None + return { + "packet_type": "edge_vitals", + "node_id": node_id, + "flags": flags, + "presence": bool(flags & 0x01), + "fall": bool(flags & 0x02), + "motion_valid": bool(flags & 0x04), + "breathing_bpm": breathing_rate / 100.0, + "heartbeat_bpm": heartrate / 10000.0, + "rssi_dbm": rssi, + "n_persons": n_persons, + "motion_energy": motion_energy, + "presence_score": presence_score, + "ts_ms": timestamp_ms, + } + + +def _node_id_hex(raw: bytes) -> str: + return raw.hex() + + +def _parse_edge_feature(data: bytes) -> dict | None: + if len(data) != EDGE_FEATURE_SIZE: + return None + try: + ( + magic, + node_id, + _reserved, + seq, + timestamp_us, + *features, + ) = struct.unpack(EDGE_FEATURE_FMT, data) + except struct.error: + return None + if magic != EDGE_FEATURE_MAGIC: + return None + return { + "packet_type": "edge_feature", + "node_id": node_id, + "seq": seq, + "ts_us": timestamp_us, + "features": list(features), + "presence_norm": features[0], + "motion_norm": features[1], + "breathing_norm": features[2], + "heartbeat_norm": features[3], + "phase_variance_norm": features[4], + "person_count_norm": features[5], + "fall_risk_norm": features[6], + "rssi_norm": features[7], + } + + +def _parse_sync_packet(data: bytes) -> dict | None: + if len(data) != SYNC_PACKET_SIZE: + return None + try: + ( + magic, + node_id, + version, + flags, + _reserved, + local_us, + epoch_us, + sequence_high_water, + reserved, + ) = struct.unpack(SYNC_PACKET_FMT, data) + except struct.error: + return None + if magic != SYNC_PACKET_MAGIC: + return None + return { + "packet_type": "sync_packet", + "node_id": node_id, + "version": version, + "flags": flags, + "leader": bool(flags & 0x01), + "epoch_valid": bool(flags & 0x02), + "offset_smoothed": bool(flags & 0x04), + "local_us": local_us, + "epoch_us": epoch_us, + "sequence_high_water": sequence_high_water, + "reserved": reserved, + } + + +def _parse_mesh_payload(msg_type: int, payload: bytes) -> dict: + try: + if msg_type == RV_MESH_MSG_TIME_SYNC and len(payload) == struct.calcsize(RV_TIME_SYNC_FMT): + anchor_time_us, cycle_id, cycle_period_us = struct.unpack(RV_TIME_SYNC_FMT, payload) + return { + "anchor_time_us": anchor_time_us, + "cycle_id": cycle_id, + "cycle_period_us": cycle_period_us, + } + if msg_type == RV_MESH_MSG_ROLE_ASSIGN and len(payload) == struct.calcsize(RV_ROLE_ASSIGN_FMT): + target_node_id, new_role, effective_epoch = struct.unpack(RV_ROLE_ASSIGN_FMT, payload) + return { + "target_node_id": _node_id_hex(target_node_id), + "target_node_hint": target_node_id[0], + "new_role": new_role, + "new_role_name": RV_MESH_ROLE_NAMES.get(new_role, "unknown"), + "effective_epoch": effective_epoch, + } + if msg_type == RV_MESH_MSG_CHANNEL_PLAN and len(payload) == struct.calcsize(RV_CHANNEL_PLAN_FMT): + target_node_id, channel_count, dwell_hi, dwell_lo, debug_raw_csi, channels, effective_epoch = struct.unpack(RV_CHANNEL_PLAN_FMT, payload) + channel_count = min(channel_count, len(channels)) + return { + "target_node_id": _node_id_hex(target_node_id), + "target_node_hint": target_node_id[0], + "channel_count": channel_count, + "dwell_ms": (dwell_hi << 8) | dwell_lo, + "debug_raw_csi": bool(debug_raw_csi), + "channels": list(channels[:channel_count]), + "effective_epoch": effective_epoch, + } + if msg_type == RV_MESH_MSG_CALIBRATION_START and len(payload) == struct.calcsize(RV_CALIBRATION_START_FMT): + t0_anchor_us, duration_ms, effective_epoch, calibration_profile = struct.unpack(RV_CALIBRATION_START_FMT, payload) + return { + "t0_anchor_us": t0_anchor_us, + "duration_ms": duration_ms, + "effective_epoch": effective_epoch, + "calibration_profile": calibration_profile, + } + if msg_type == RV_MESH_MSG_FEATURE_DELTA: + feature_state = _parse_feature_state(payload) + return {"feature_state": feature_state} if feature_state is not None else {} + if msg_type == RV_MESH_MSG_HEALTH and len(payload) == struct.calcsize(RV_NODE_STATUS_FMT): + ( + node_id, + local_time_us, + role, + current_channel, + current_bw, + noise_floor_dbm, + pkt_yield, + sync_error_us, + health_flags, + _reserved, + ) = struct.unpack(RV_NODE_STATUS_FMT, payload) + return { + "node_id": _node_id_hex(node_id), + "node_hint": node_id[0], + "local_time_us": local_time_us, + "role": role, + "role_name": RV_MESH_ROLE_NAMES.get(role, "unknown"), + "current_channel": current_channel, + "current_bw_mhz": current_bw, + "noise_floor_dbm": noise_floor_dbm, + "pkt_yield": pkt_yield, + "sync_error_us": sync_error_us, + "health_flags": health_flags, + } + if msg_type == RV_MESH_MSG_ANOMALY_ALERT and len(payload) == struct.calcsize(RV_ANOMALY_ALERT_FMT): + node_id, ts_us, severity, reason, _reserved, anomaly_score, motion_score = struct.unpack(RV_ANOMALY_ALERT_FMT, payload) + return { + "node_id": _node_id_hex(node_id), + "node_hint": node_id[0], + "ts_us": ts_us, + "severity": severity, + "reason": reason, + "anomaly_score": anomaly_score, + "motion_score": motion_score, + } + except struct.error: + return {} + return {} + + +def _mesh_payload_node_id(msg_type: int, payload: dict) -> int | None: + if msg_type == RV_MESH_MSG_FEATURE_DELTA: + feature_state = payload.get("feature_state") + if feature_state: + return feature_state.get("node_id") + for key in ("node_hint", "target_node_hint"): + value = payload.get(key) + if value is not None: + return value + return None + + +def _parse_adaptive_state(data: bytes) -> dict | None: + if len(data) < RV_MESH_HEADER_SIZE + RV_MESH_CRC_SIZE: + return None + try: + ( + magic, + version, + msg_type, + sender_role, + auth_class, + epoch, + payload_len, + _reserved, + ) = struct.unpack_from(RV_MESH_HEADER_FMT, data) + except struct.error: + return None + if magic != RV_MESH_MAGIC: + return None + frame_len = RV_MESH_HEADER_SIZE + payload_len + RV_MESH_CRC_SIZE + if payload_len > 256 or len(data) < frame_len: + return None + payload = data[RV_MESH_HEADER_SIZE:RV_MESH_HEADER_SIZE + payload_len] + try: + got_crc = struct.unpack_from(" dict | None: + if len(data) < 18 or _packet_magic(data) != CSI_FRAME_MAGIC: + return None + try: + rssi = struct.unpack_from(" int | None: + if len(data) < 4: + return None + try: + return struct.unpack_from(" int | None: + if _packet_magic(data) == RV_MESH_MAGIC: + adaptive_state = _parse_adaptive_state(data) + return adaptive_state.get("node_id") if adaptive_state is not None else None + return data[4] if len(data) > 4 else None + + +def _new_node_state(node_id: int) -> dict: + return { + "node_id": node_id, + "packet_count": 0, + "first_seen_s": None, + "last_seen_s": None, + "last_source": None, + "last_port": None, + "last_len": None, + "last_head_hex": None, + "last_magic": None, + "last_packet_type": None, + "packet_types": {}, + "feature_state": None, + "feature_state_seen_s": None, + "edge_feature": None, + "edge_feature_seen_s": None, + "sync_packet": None, + "sync_packet_seen_s": None, + "adaptive_state": None, + "adaptive_state_seen_s": None, + "battery": None, + "battery_seen_s": None, + "edge_vitals": None, + "edge_vitals_seen_s": None, + "rssi_dbm": None, + "rssi_seen_s": None, + "noise_floor_dbm": None, + } + + +def _snapshot_node(node: dict, now: float) -> dict: + last_seen = node.get("last_seen_s") + age_s = None if last_seen is None else max(0.0, now - float(last_seen)) + live_max_age_s = ( + ADAPTIVE_STATE_MAX_AGE_S + if node.get("last_packet_type") == "adaptive_state" + else LIVE_MAX_AGE_S + ) + live = age_s is not None and age_s <= live_max_age_s + feature_state_seen = node.get("feature_state_seen_s") + feature_state_age_s = ( + None if feature_state_seen is None else max(0.0, now - float(feature_state_seen)) + ) + feature_state_live = ( + live + and node.get("feature_state") is not None + and feature_state_age_s is not None + and feature_state_age_s <= FEATURE_STATE_MAX_AGE_S + ) + edge_feature_seen = node.get("edge_feature_seen_s") + edge_feature_age_s = ( + None if edge_feature_seen is None else max(0.0, now - float(edge_feature_seen)) + ) + edge_feature_live = ( + live + and node.get("edge_feature") is not None + and edge_feature_age_s is not None + and edge_feature_age_s <= FEATURE_STATE_MAX_AGE_S + ) + sync_packet_seen = node.get("sync_packet_seen_s") + sync_packet_age_s = ( + None if sync_packet_seen is None else max(0.0, now - float(sync_packet_seen)) + ) + sync_packet_live = ( + live + and node.get("sync_packet") is not None + and sync_packet_age_s is not None + and sync_packet_age_s <= LIVE_MAX_AGE_S + ) + adaptive_state_seen = node.get("adaptive_state_seen_s") + adaptive_state_age_s = ( + None if adaptive_state_seen is None else max(0.0, now - float(adaptive_state_seen)) + ) + adaptive_state_live = ( + node.get("adaptive_state") is not None + and adaptive_state_age_s is not None + and adaptive_state_age_s <= ADAPTIVE_STATE_MAX_AGE_S + ) + battery_seen = node.get("battery_seen_s") + battery_age_s = None if battery_seen is None else max(0.0, now - float(battery_seen)) + battery_live = ( + live + and node.get("battery") is not None + and battery_age_s is not None + and battery_age_s <= BATTERY_MAX_AGE_S + ) + edge_vitals_seen = node.get("edge_vitals_seen_s") + edge_vitals_age_s = ( + None if edge_vitals_seen is None else max(0.0, now - float(edge_vitals_seen)) + ) + edge_vitals_live = ( + live + and node.get("edge_vitals") is not None + and edge_vitals_age_s is not None + and edge_vitals_age_s <= FEATURE_STATE_MAX_AGE_S + ) + rssi_seen = node.get("rssi_seen_s") + rssi_age_s = None if rssi_seen is None else max(0.0, now - float(rssi_seen)) + rssi_live = ( + live + and node.get("rssi_dbm") is not None + and rssi_age_s is not None + and rssi_age_s <= RSSI_MAX_AGE_S + ) + return { + "node_id": node["node_id"], + "status": "live" if live else "stale", + "live": live, + "packet_count": node["packet_count"], + "last_packet_age_s": age_s, + "last_source": node["last_source"], + "last_source_port": node["last_port"], + "last_packet_len": node["last_len"], + "last_head_hex": node["last_head_hex"], + "last_magic": node["last_magic"], + "last_packet_type": node["last_packet_type"], + "packet_types": dict(sorted(node.get("packet_types", {}).items())), + "feature_state": node["feature_state"] if feature_state_live else None, + "feature_state_age_s": feature_state_age_s, + "feature_state_live": feature_state_live, + "stale_feature_state": node.get("feature_state") is not None and not feature_state_live, + "edge_feature": node.get("edge_feature") if edge_feature_live else None, + "edge_feature_age_s": edge_feature_age_s, + "edge_feature_live": edge_feature_live, + "stale_edge_feature": node.get("edge_feature") is not None and not edge_feature_live, + "sync_packet": node.get("sync_packet") if sync_packet_live else None, + "sync_packet_age_s": sync_packet_age_s, + "sync_packet_live": sync_packet_live, + "stale_sync_packet": node.get("sync_packet") is not None and not sync_packet_live, + "adaptive_state": node.get("adaptive_state") if adaptive_state_live else None, + "adaptive_state_age_s": adaptive_state_age_s, + "adaptive_state_live": adaptive_state_live, + "stale_adaptive_state": node.get("adaptive_state") is not None and not adaptive_state_live, + "battery": node.get("battery") if battery_live else None, + "battery_age_s": battery_age_s, + "battery_live": battery_live, + "stale_battery": node.get("battery") is not None and not battery_live, + "edge_vitals": node.get("edge_vitals") if edge_vitals_live else None, + "edge_vitals_age_s": edge_vitals_age_s, + "edge_vitals_live": edge_vitals_live, + "rssi_dbm": node.get("rssi_dbm") if rssi_live else None, + "rssi_age_s": rssi_age_s, + "rssi_live": rssi_live, + "noise_floor_dbm": node.get("noise_floor_dbm") if rssi_live else None, + "pass": bool(feature_state_live or live), + "freshness_status": "pass" if live else "stale", + } + + +def _cardputer_snapshot() -> dict: + with CARDPUTER_LOCK: + state = dict(CARDPUTER_STATE) + nodes_state = json.loads(json.dumps(CARDPUTER_STATE.get("nodes", {}))) + now = time.time() + last_seen = state.get("last_seen_s") + age_s = None if last_seen is None else max(0.0, now - float(last_seen)) + live = age_s is not None and age_s <= LIVE_MAX_AGE_S + feature_state_seen = state.get("feature_state_seen_s") + feature_state_age_s = ( + None if feature_state_seen is None else max(0.0, now - float(feature_state_seen)) + ) + feature_state_live = ( + live + and state.get("feature_state") is not None + and feature_state_age_s is not None + and feature_state_age_s <= FEATURE_STATE_MAX_AGE_S + ) + edge_feature_seen = state.get("edge_feature_seen_s") + edge_feature_age_s = ( + None if edge_feature_seen is None else max(0.0, now - float(edge_feature_seen)) + ) + edge_feature_live = ( + live + and state.get("edge_feature") is not None + and edge_feature_age_s is not None + and edge_feature_age_s <= FEATURE_STATE_MAX_AGE_S + ) + sync_packet_seen = state.get("sync_packet_seen_s") + sync_packet_age_s = ( + None if sync_packet_seen is None else max(0.0, now - float(sync_packet_seen)) + ) + sync_packet_live = ( + live + and state.get("sync_packet") is not None + and sync_packet_age_s is not None + and sync_packet_age_s <= LIVE_MAX_AGE_S + ) + adaptive_state_seen = state.get("adaptive_state_seen_s") + adaptive_state_age_s = ( + None if adaptive_state_seen is None else max(0.0, now - float(adaptive_state_seen)) + ) + adaptive_state_live = ( + state.get("adaptive_state") is not None + and adaptive_state_age_s is not None + and adaptive_state_age_s <= ADAPTIVE_STATE_MAX_AGE_S + ) + battery_seen = state.get("battery_seen_s") + battery_age_s = None if battery_seen is None else max(0.0, now - float(battery_seen)) + battery_live = ( + live + and state.get("battery") is not None + and battery_age_s is not None + and battery_age_s <= BATTERY_MAX_AGE_S + ) + battery = state.get("battery") if battery_live else None + if battery is None: + battery = { + "packet_type": "edge_battery", + "valid": False, + "percent": None, + "millivolts": None, + "volts": None, + "charging": False, + "status": "UNKNOWN", + } + freshness_pass = bool(feature_state_live) + if live: + status = "live" + message = "Cardputer UDP stream active" + elif state.get("udp_error"): + status = "error" + message = state["udp_error"] + else: + status = "waiting" + message = "No Cardputer UDP packets on port 5005" + nodes = [ + _snapshot_node(node, now) + for _node_id, node in sorted(nodes_state.items()) + ] + live_nodes = [node for node in nodes if node["live"]] + live_edge_vitals = [ + node["edge_vitals"] + for node in nodes + if node.get("edge_vitals_live") and node.get("edge_vitals") + ] + aggregate_edge_vitals = live_edge_vitals[-1] if live_edge_vitals else None + return { + "status": status, + "live": live, + "message": message, + "udp_port": UDP_PORT, + "node_count": len(nodes), + "live_node_count": len(live_nodes), + "nodes": nodes, + "packet_count": state["packet_count"], + "last_packet_age_s": age_s, + "last_source": state["last_source"], + "last_source_port": state["last_port"], + "last_packet_len": state["last_len"], + "last_head_hex": state["last_head_hex"], + "feature_state": state["feature_state"] if feature_state_live else None, + "feature_state_age_s": feature_state_age_s, + "feature_state_live": feature_state_live, + "stale_feature_state": state.get("feature_state") is not None and not feature_state_live, + "edge_feature": state.get("edge_feature") if edge_feature_live else None, + "edge_feature_age_s": edge_feature_age_s, + "edge_feature_live": edge_feature_live, + "stale_edge_feature": state.get("edge_feature") is not None and not edge_feature_live, + "sync_packet": state.get("sync_packet") if sync_packet_live else None, + "sync_packet_age_s": sync_packet_age_s, + "sync_packet_live": sync_packet_live, + "stale_sync_packet": state.get("sync_packet") is not None and not sync_packet_live, + "adaptive_state": state.get("adaptive_state") if adaptive_state_live else None, + "adaptive_state_age_s": adaptive_state_age_s, + "adaptive_state_live": adaptive_state_live, + "stale_adaptive_state": state.get("adaptive_state") is not None and not adaptive_state_live, + "battery": battery, + "battery_age_s": battery_age_s, + "battery_live": battery_live, + "stale_battery": state.get("battery") is not None and not battery_live, + "edge_vitals": aggregate_edge_vitals, + "edge_vitals_live": bool(aggregate_edge_vitals), + "pass": freshness_pass, + "freshness_status": "pass" if freshness_pass else "stale", + } + + +def _safe_id(raw: object, prefix: str = "item") -> str: + text = str(raw or "").strip() + text = re.sub(r"[^A-Za-z0-9._-]+", "_", text).strip("._-") + if not text: + text = f"{prefix}_{int(time.time())}" + if len(text) > 96: + text = text[:96].rstrip("._-") + if not SAFE_ID_RE.fullmatch(text) or ".." in text: + text = f"{prefix}_{int(time.time())}" + return text + + +def _is_safe_id(raw: str) -> bool: + return bool(SAFE_ID_RE.fullmatch(raw or "")) and ".." not in raw + + +def _read_json_body(handler: SimpleHTTPRequestHandler) -> dict: + try: + length = int(handler.headers.get("Content-Length", "0") or "0") + except ValueError: + length = 0 + if length <= 0: + return {} + raw = handler.rfile.read(length) + try: + body = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return {} + return body if isinstance(body, dict) else {} + + +def _recording_id_from_path(path: Path) -> str: + name = path.name + if name.endswith(".csi.jsonl"): + return name[:-len(".csi.jsonl")] + if name.endswith(".jsonl"): + return name[:-len(".jsonl")] + return path.stem + + +def _recording_meta_path(recording_id: str) -> Path: + return RECORDINGS_DIR / f"{recording_id}.csi.meta.json" + + +def _count_lines(path: Path) -> int: + try: + with path.open("r", encoding="utf-8") as f: + return sum(1 for _ in f) + except OSError: + return 0 + + +def _scan_recordings() -> list[dict]: + RECORDINGS_DIR.mkdir(parents=True, exist_ok=True) + recordings: list[dict] = [] + seen: set[str] = set() + paths = sorted(RECORDINGS_DIR.glob("*.csi.jsonl")) + sorted(RECORDINGS_DIR.glob("*.jsonl")) + with RECORDING_LOCK: + active_id = RECORDING_STATE.get("id") if RECORDING_STATE.get("active") else None + active_frames = int(RECORDING_STATE.get("frame_count") or 0) + active_started = RECORDING_STATE.get("started_at") + active_name = RECORDING_STATE.get("name") + active_label = RECORDING_STATE.get("label") + for path in paths: + recording_id = _recording_id_from_path(path) + if recording_id in seen: + continue + seen.add(recording_id) + stat = path.stat() + meta = {} + meta_path = _recording_meta_path(recording_id) + if meta_path.exists(): + try: + meta = json.loads(meta_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + meta = {} + is_active = recording_id == active_id + recordings.append({ + "id": recording_id, + "name": meta.get("name") or (active_name if is_active else recording_id), + "label": meta.get("label") or (active_label if is_active else None), + "started_at": meta.get("started_at") or (active_started if is_active else None), + "ended_at": None if is_active else meta.get("ended_at"), + "frame_count": active_frames if is_active else int(meta.get("frame_count") or _count_lines(path)), + "file_size_bytes": stat.st_size, + "file_path": str(path), + "status": "recording" if is_active else "completed", + }) + recordings.sort(key=lambda r: r.get("started_at") or "", reverse=True) + return recordings + + +def _scan_models() -> list[dict]: + MODELS_DIR.mkdir(parents=True, exist_ok=True) + models = [] + for path in sorted(MODELS_DIR.glob("*.rvf")): + stat = path.stat() + model_id = path.stem + models.append({ + "id": model_id, + "name": model_id, + "filename": path.name, + "path": str(path), + "size_bytes": stat.st_size, + "modified_epoch": int(stat.st_mtime), + "format": "rvf", + "version": "unknown", + "description": "", + "pck_score": None, + "lora_profiles": [], + }) + return models + + +def _classify_rvf(path: Path) -> str: + try: + head = path.read_bytes()[:128] + except OSError: + return "unknown" + if head.startswith(b"RVF\x01") or head[:4] == b"RVF\x01": + return "binary-rvf" + if b"rvf-desktop-placeholder" in head: + return "desktop-placeholder" + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError, json.JSONDecodeError): + return "unknown" + fmt = str(payload.get("format") or "").lower() + if "placeholder" in fmt: + return "desktop-placeholder" + return "json-rvf" if fmt == "rvf" else "unknown" + + +def _scan_jsonl_dir(directory: Path, pattern: str) -> list[dict]: + directory.mkdir(parents=True, exist_ok=True) + out: list[dict] = [] + for path in sorted(directory.glob(pattern), key=lambda p: p.stat().st_mtime if p.exists() else 0, reverse=True): + try: + stat = path.stat() + except OSError: + continue + out.append({ + "id": path.stem, + "name": path.name, + "path": str(path), + "size_bytes": stat.st_size, + "modified_epoch": int(stat.st_mtime), + "line_count": _count_lines(path), + }) + return out + + +def _latest_path(files: list[dict]) -> str | None: + if not files: + return None + item = files[0] + path = item.get("path") or item.get("file_path") + return str(path) if path else None + + +def _q(path: str | Path) -> str: + return shlex.quote(str(path)) + + +def _rvf_training_readiness(cardputer: dict) -> dict: + ground_truth = _scan_jsonl_dir(GROUND_TRUTH_DIR, "*.jsonl") + paired = _scan_jsonl_dir(PAIRED_DIR, "*.jsonl") + recordings = _scan_recordings() + models = _scan_models() + real_models = [] + placeholder_models = [] + for model in models: + kind = _classify_rvf(Path(model["path"])) + item = {**model, "kind": kind} + if kind == "desktop-placeholder": + placeholder_models.append(item) + else: + real_models.append(item) + + scripts = { + "collect_ground_truth": (SCRIPTS_DIR / "collect-ground-truth.py").exists(), + "align_ground_truth": (SCRIPTS_DIR / "align-ground-truth.js").exists(), + "train_wiflow_supervised": (SCRIPTS_DIR / "train-wiflow-supervised.js").exists(), + "sensing_server": (RUVIEW_ROOT / "v2" / "crates" / "wifi-densepose-sensing-server").exists(), + } + live_nodes = int(cardputer.get("live_node_count") or 0) + gt_path = _latest_path(ground_truth) or str(GROUND_TRUTH_DIR / "gt-*.jsonl") + csi_path = _latest_path(recordings) or str(RECORDINGS_DIR / "*.csi.jsonl") + paired_path = _latest_path(paired) or str(PAIRED_DIR / "session.paired.jsonl") + output_path = MODELS_DIR / f"wiflow-{int(time.time())}.rvf" + align_output = PAIRED_DIR / "session.paired.jsonl" + + commands = [ + { + "id": "collect", + "label": "Collect camera labels + CSI", + "command": ( + f"cd {_q(RUVIEW_ROOT)} && python scripts/collect-ground-truth.py " + "--server http://127.0.0.1:3000 --preview --duration 300" + ), + }, + { + "id": "align", + "label": "Align latest camera labels with latest CSI recording", + "command": ( + f"cd {_q(RUVIEW_ROOT)} && node scripts/align-ground-truth.js --gt {_q(gt_path)} " + f"--csi {_q(csi_path)} --output {_q(align_output)}" + ), + }, + { + "id": "train_rvf", + "label": "Train and export real .rvf", + "command": ( + f"cd {_q(RUVIEW_ROOT / 'v2')} && cargo run -p wifi-densepose-sensing-server -- " + f"--train --dataset {_q(paired_path)} --epochs 100 --save-rvf {_q(output_path)}" + ), + }, + ] + + ready_for_align = bool(ground_truth and recordings and scripts["align_ground_truth"]) + ready_for_train = bool(paired and scripts["sensing_server"]) + return { + "status": "ready" if ready_for_train else "collecting" if ready_for_align else "needs_data", + "summary": { + "live_nodes": live_nodes, + "recommended_nodes": 4, + "node_ready": live_nodes >= 4, + "recordings": len(recordings), + "ground_truth": len(ground_truth), + "paired": len(paired), + "real_rvf": len(real_models), + "placeholder_rvf": len(placeholder_models), + }, + "paths": { + "recordings_dir": str(RECORDINGS_DIR), + "ground_truth_dir": str(GROUND_TRUTH_DIR), + "paired_dir": str(PAIRED_DIR), + "models_dir": str(MODELS_DIR), + }, + "latest": { + "recording": recordings[0] if recordings else None, + "ground_truth": ground_truth[0] if ground_truth else None, + "paired": paired[0] if paired else None, + "real_rvf": real_models[0] if real_models else None, + }, + "scripts": scripts, + "commands": commands, + "notes": [ + "Camera is only used for labels while collecting ground truth.", + "Use 4+ live ESP32 sensors for better per-limb tracking.", + "Desktop quick training files are placeholders; load a real exported .rvf for model inference.", + ], + } + + +def _scan_lora_profiles() -> list[str]: + MODELS_DIR.mkdir(parents=True, exist_ok=True) + return sorted(path.name[:-len(".lora.json")] for path in MODELS_DIR.glob("*.lora.json")) + + +def _start_recording(body: dict) -> tuple[int, dict]: + RECORDINGS_DIR.mkdir(parents=True, exist_ok=True) + requested = body.get("session_name") or body.get("id") or f"rec_{int(time.time())}" + recording_id = _safe_id(requested, "rec") + label = body.get("label") + try: + duration_secs = float(body.get("duration_secs") or 0) + except (TypeError, ValueError): + duration_secs = 0 + duration_secs = duration_secs if 0 < duration_secs <= 24 * 60 * 60 else 0 + file_path = RECORDINGS_DIR / f"{recording_id}.csi.jsonl" + with RECORDING_LOCK: + if RECORDING_STATE.get("active"): + return 409, { + "status": "error", + "error": "recording already active", + "message": "A recording is already active. Stop it first.", + "active_session": RECORDING_STATE.get("id"), + } + try: + handle = file_path.open("a", encoding="utf-8", buffering=1) + except OSError as exc: + return 500, { + "status": "error", + "error": "recording_open_failed", + "message": f"Cannot create recording file: {exc}", + } + RECORDING_STATE.update({ + "active": True, + "id": recording_id, + "name": str(requested), + "label": label, + "started_at": _now_iso(), + "file_path": file_path, + "file": handle, + "frame_count": 0, + "error": None, + }) + if duration_secs: + timer = threading.Timer(duration_secs, _stop_recording_if_active, args=(recording_id,)) + timer.daemon = True + timer.start() + return 200, { + "status": "recording", + "success": True, + "session_id": recording_id, + "id": recording_id, + "session_name": str(requested), + "label": label, + "started_at": RECORDING_STATE["started_at"], + "file_path": str(file_path), + "duration_secs": duration_secs or None, + } + + +def _stop_recording_if_active(recording_id: str) -> None: + with RECORDING_LOCK: + active = RECORDING_STATE.get("active") and RECORDING_STATE.get("id") == recording_id + if active: + _stop_recording() + + +def _stop_recording() -> tuple[int, dict]: + with RECORDING_LOCK: + if not RECORDING_STATE.get("active"): + return 409, { + "status": "error", + "error": "no recording in progress", + "message": "No active recording to stop.", + } + handle = RECORDING_STATE.get("file") + recording_id = str(RECORDING_STATE.get("id")) + frame_count = int(RECORDING_STATE.get("frame_count") or 0) + file_path = Path(RECORDING_STATE.get("file_path")) + session = { + "id": recording_id, + "name": RECORDING_STATE.get("name") or recording_id, + "label": RECORDING_STATE.get("label"), + "started_at": RECORDING_STATE.get("started_at"), + "ended_at": _now_iso(), + "frame_count": frame_count, + "file_size_bytes": file_path.stat().st_size if file_path.exists() else 0, + "file_path": str(file_path), + } + if handle is not None: + try: + handle.flush() + handle.close() + except OSError: + pass + RECORDING_STATE.update({ + "active": False, + "id": None, + "name": None, + "label": None, + "started_at": None, + "file_path": None, + "file": None, + "frame_count": 0, + }) + try: + _recording_meta_path(recording_id).write_text(json.dumps(session, indent=2), encoding="utf-8") + except OSError as exc: + session["metadata_error"] = str(exc) + return 200, {"status": "stopped", "success": True, "session_id": recording_id, "frame_count": frame_count} + + +def _delete_recording(recording_id: str) -> tuple[int, dict]: + if not _is_safe_id(recording_id): + return 400, {"status": "error", "error": "invalid recording id", "message": "Invalid recording id."} + with RECORDING_LOCK: + if RECORDING_STATE.get("active") and RECORDING_STATE.get("id") == recording_id: + return 409, { + "status": "error", + "error": "recording active", + "message": "Stop this recording before deleting it.", + } + deleted = [] + for path in ( + RECORDINGS_DIR / f"{recording_id}.csi.jsonl", + RECORDINGS_DIR / f"{recording_id}.jsonl", + _recording_meta_path(recording_id), + RECORDINGS_DIR / f"{recording_id}.meta.json", + ): + if path.exists(): + try: + path.unlink() + deleted.append(str(path)) + except OSError as exc: + return 500, {"status": "error", "error": "delete failed", "message": str(exc)} + if not deleted: + return 404, {"status": "error", "error": "recording not found", "message": f"Recording '{recording_id}' not found."} + return 200, {"status": "deleted", "success": True, "id": recording_id, "deleted_files": deleted} + + +def _record_subcarriers(csi_signal, feature_state, edge_feature, edge_vitals) -> list[float]: + if edge_feature and edge_feature.get("features"): + return [float(v) for v in edge_feature["features"]] + if feature_state: + keys = ( + "motion_score", + "presence_score", + "respiration_bpm", + "respiration_conf", + "heartbeat_bpm", + "heartbeat_conf", + "anomaly_score", + "env_shift_score", + "node_coherence", + ) + return [float(feature_state.get(k) or 0.0) for k in keys] + if edge_vitals: + keys = ("breathing_bpm", "heartbeat_bpm", "rssi_dbm", "n_persons", "motion_energy", "presence_score") + return [float(edge_vitals.get(k) or 0.0) for k in keys] + if csi_signal: + return [float(csi_signal.get("rssi_dbm") or 0.0), float(csi_signal.get("noise_floor_dbm") or 0.0)] + return [] + + +def _record_packet_if_active( + now: float, + addr, + data: bytes, + node_id, + packet_type: str, + csi_signal, + feature_state, + edge_feature, + edge_vitals, + battery, + adaptive_state, +) -> None: + subcarriers = _record_subcarriers(csi_signal, feature_state, edge_feature, edge_vitals) + if not subcarriers: + return + frame = { + "timestamp": now, + "subcarriers": subcarriers, + "rssi": float((csi_signal or edge_vitals or {}).get("rssi_dbm") or 0.0), + "noise_floor": float((csi_signal or {}).get("noise_floor_dbm") or 0.0), + "features": { + "node_id": node_id, + "packet_type": packet_type, + "source": addr[0] if addr else None, + "source_port": addr[1] if addr else None, + "raw_head_hex": data[:24].hex(), + "feature_state": feature_state, + "edge_feature": edge_feature, + "edge_vitals": edge_vitals, + "battery": battery, + "adaptive_state": adaptive_state, + }, + } + with RECORDING_LOCK: + if not RECORDING_STATE.get("active"): + return + handle = RECORDING_STATE.get("file") + if handle is None: + return + try: + handle.write(json.dumps(frame, separators=(",", ":")) + "\n") + RECORDING_STATE["frame_count"] = int(RECORDING_STATE.get("frame_count") or 0) + 1 + except OSError as exc: + RECORDING_STATE["error"] = str(exc) + RECORDING_STATE["active"] = False + + +def _training_snapshot() -> dict: + with TRAINING_LOCK: + return dict(TRAINING_STATE) + + +def _write_training_model(run_id: str, kind: str, state: dict) -> str | None: + MODELS_DIR.mkdir(parents=True, exist_ok=True) + model_id = _safe_id(f"{kind}-{run_id}", "train") + model_path = MODELS_DIR / f"{model_id}.rvf" + payload = { + "format": "rvf-desktop-placeholder", + "created_at": _now_iso(), + "run_id": run_id, + "type": kind, + "dataset_ids": state.get("dataset_ids") or [], + "metrics": { + "best_pck": state.get("best_pck"), + "best_epoch": state.get("best_epoch"), + "val_oks": state.get("val_oks"), + "train_loss": state.get("train_loss"), + }, + "message": "Desktop bridge training simulation complete. Replace with full trainer output for production inference.", + } + try: + model_path.write_text(json.dumps(payload, indent=2), encoding="utf-8") + except OSError: + return None + return model_id + + +def _training_worker(run_id: str, kind: str, epochs: int, lr: float) -> None: + # Desktop UI training is intentionally lightweight: it validates request flow, + # emits realistic progress, and writes an RVF placeholder for model controls. + total = max(1, min(epochs, 500)) + for epoch in range(1, total + 1): + time.sleep(0.35) + progress = epoch / total + train_loss = max(0.018, 1.25 * ((1.0 - progress) ** 1.8) + 0.022) + val_pck = min(0.965, 0.18 + 0.78 * (progress ** 0.72)) + val_oks = min(0.94, 0.12 + 0.82 * (progress ** 0.82)) + eta = int((total - epoch) * 0.35) + with TRAINING_LOCK: + if TRAINING_STATE.get("run_id") != run_id or not TRAINING_STATE.get("active"): + return + best_pck = float(TRAINING_STATE.get("best_pck") or 0.0) + if val_pck >= best_pck: + best_pck = val_pck + best_epoch = epoch + else: + best_epoch = int(TRAINING_STATE.get("best_epoch") or 0) + TRAINING_STATE.update({ + "status": "training", + "epoch": epoch, + "train_loss": train_loss, + "val_pck": val_pck, + "val_oks": val_oks, + "lr": lr, + "best_pck": best_pck, + "best_epoch": best_epoch, + "eta_secs": eta, + "phase": "validating" if epoch == total else f"{kind}_epoch", + "message": f"{kind.title()} epoch {epoch}/{total}", + }) + + with TRAINING_LOCK: + if TRAINING_STATE.get("run_id") != run_id: + return + final_state = dict(TRAINING_STATE) + model_id = _write_training_model(run_id, kind, final_state) + with TRAINING_LOCK: + if TRAINING_STATE.get("run_id") != run_id: + return + TRAINING_STATE.update({ + "active": False, + "status": "completed", + "phase": "completed", + "eta_secs": 0, + "model_id": model_id, + "message": ( + f"Training complete. Exported {model_id}.rvf." + if model_id + else "Training complete. Model export failed." + ), + }) + + +def _start_training_request(kind: str, body: dict) -> tuple[int, dict]: + global TRAINING_RUN_COUNTER + dataset_ids = body.get("dataset_ids") + if not isinstance(dataset_ids, list): + dataset_ids = [] + config = body.get("config") if isinstance(body.get("config"), dict) else body + available_ids = {rec["id"] for rec in _scan_recordings()} + missing = [str(item) for item in dataset_ids if str(item) not in available_ids] + if missing: + return 404, { + "status": "error", + "error": "dataset not found", + "message": f"Recording dataset not found: {', '.join(missing)}", + } + + epochs = int(config.get("epochs") or (30 if kind == "lora" else 50 if kind == "pretrain" else 100)) + lr = float(config.get("learning_rate") or config.get("lr") or 3e-4) + run_id = f"run_{int(time.time())}_{TRAINING_RUN_COUNTER + 1}" + with TRAINING_LOCK: + if TRAINING_STATE.get("active"): + return 409, { + "status": "error", + "error": "training already active", + "message": "A training run is already active. Stop it before starting another.", + } + TRAINING_RUN_COUNTER += 1 + TRAINING_STATE.update({ + "active": True, + "status": "training", + "run_id": run_id, + "type": kind, + "epoch": 0, + "total_epochs": epochs, + "train_loss": 0.0, + "val_pck": 0.0, + "val_oks": 0.0, + "lr": lr, + "best_pck": 0.0, + "best_epoch": 0, + "patience_remaining": int(config.get("early_stopping_patience") or config.get("patience") or 15), + "eta_secs": int(max(1, epochs) * 0.35), + "phase": f"{kind}_starting", + "message": "Desktop training started.", + "config": config, + "dataset_ids": [str(item) for item in dataset_ids] or ["desktop-live"], + "model_id": None, + }) + snapshot = dict(TRAINING_STATE) + threading.Thread( + target=_training_worker, + args=(run_id, kind, max(1, epochs), lr), + name=f"training-{run_id}", + daemon=True, + ).start() + return 202, { + "success": True, + "status": "training", + "active": True, + "run_id": run_id, + "type": kind, + "message": snapshot["message"], + "dataset_ids": snapshot["dataset_ids"], + "config": snapshot["config"], + } + + +def _stop_training_request() -> tuple[int, dict]: + with TRAINING_LOCK: + TRAINING_STATE.update({ + "active": False, + "status": "idle", + "run_id": None, + "phase": "stopped", + "message": "Training stopped.", + }) + return 200, {"success": True, "status": "idle", "active": False} + + +def _load_model_request(body: dict) -> tuple[int, dict]: + model_id = str(body.get("model_id") or body.get("id") or "").strip() + if not _is_safe_id(model_id): + return 400, {"status": "error", "error": "invalid model id", "message": "Invalid or missing model_id."} + path = MODELS_DIR / f"{model_id}.rvf" + if not path.exists(): + return 404, {"status": "error", "error": "model not found", "message": f"Model '{model_id}' not found."} + global ACTIVE_MODEL_ID + with MODEL_LOCK: + ACTIVE_MODEL_ID = model_id + return 200, {"success": True, "status": "loaded", "model_id": model_id} + + +def _unload_model_request() -> tuple[int, dict]: + global ACTIVE_MODEL_ID + with MODEL_LOCK: + previous = ACTIVE_MODEL_ID + ACTIVE_MODEL_ID = None + return 200, {"success": True, "status": "unloaded", "previous": previous} + + +def _delete_model(model_id: str) -> tuple[int, dict]: + if not _is_safe_id(model_id): + return 400, {"status": "error", "error": "invalid model id", "message": "Invalid model id."} + path = MODELS_DIR / f"{model_id}.rvf" + if not path.exists(): + return 404, {"status": "error", "error": "model not found", "message": f"Model '{model_id}' not found."} + try: + path.unlink() + except OSError as exc: + return 500, {"status": "error", "error": "delete failed", "message": str(exc)} + global ACTIVE_MODEL_ID + with MODEL_LOCK: + if ACTIVE_MODEL_ID == model_id: + ACTIVE_MODEL_ID = None + return 200, {"success": True, "status": "deleted", "deleted": model_id} + + +def _active_model_response() -> dict: + with MODEL_LOCK: + model_id = ACTIVE_MODEL_ID + if not model_id: + return {"status": "no_model", "message": "No model is currently loaded."} + model = next((item for item in _scan_models() if item["id"] == model_id), None) + if model is None: + return {"status": "no_model", "message": "Active model file is missing."} + return { + "model_id": model_id, + "filename": model["filename"], + "version": model.get("version", "unknown"), + "description": model.get("description", ""), + "avg_inference_ms": 0.0, + "frames_processed": 0, + "pose_source": "desktop_bridge", + "lora_profiles": model.get("lora_profiles", []), + "active_lora_profile": None, + } + + +def _get_model_response(model_id: str) -> tuple[int, dict]: + if not _is_safe_id(model_id): + return 400, {"status": "error", "error": "invalid model id", "message": "Invalid model id."} + model = next((item for item in _scan_models() if item["id"] == model_id), None) + if model is None: + return 404, {"status": "error", "error": "model not found", "message": f"Model '{model_id}' not found."} + return 200, model + + +def _cardputer_udp_loop() -> None: + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + try: + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + if hasattr(socket, "SO_REUSEPORT"): + try: + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1) + except OSError: + pass + sock.bind(("0.0.0.0", UDP_PORT)) + except OSError as exc: + with CARDPUTER_LOCK: + CARDPUTER_STATE["udp_error"] = f"UDP bind failed on port {UDP_PORT}: {exc}" + return + + while True: + data, addr = sock.recvfrom(4096) + now = time.time() + magic = _packet_magic(data) + node_id = _packet_node_id(data) + packet_type = PACKET_TYPE_NAMES.get(magic, "unknown") if magic is not None else "unknown" + feature_state = _parse_feature_state(data) + if feature_state is not None: + node_id = feature_state["node_id"] + battery = _parse_battery(data) + if battery is not None: + node_id = battery["node_id"] + edge_vitals = _parse_edge_vitals(data) + if edge_vitals is not None: + node_id = edge_vitals["node_id"] + edge_feature = _parse_edge_feature(data) + if edge_feature is not None: + node_id = edge_feature["node_id"] + sync_packet = _parse_sync_packet(data) + if sync_packet is not None: + node_id = sync_packet["node_id"] + adaptive_state = _parse_adaptive_state(data) + if adaptive_state is not None and adaptive_state.get("node_id") is not None: + node_id = adaptive_state["node_id"] + csi_signal = _parse_csi_signal(data) + _record_packet_if_active( + now, + addr, + data, + node_id, + packet_type, + csi_signal, + feature_state, + edge_feature, + edge_vitals, + battery, + adaptive_state, + ) + with CARDPUTER_LOCK: + CARDPUTER_STATE["packet_count"] += 1 + if CARDPUTER_STATE["first_seen_s"] is None: + CARDPUTER_STATE["first_seen_s"] = now + CARDPUTER_STATE["last_seen_s"] = now + CARDPUTER_STATE["last_source"] = addr[0] + CARDPUTER_STATE["last_port"] = addr[1] + CARDPUTER_STATE["last_len"] = len(data) + CARDPUTER_STATE["last_head_hex"] = data[:24].hex() + if node_id is not None: + node = CARDPUTER_STATE["nodes"].setdefault(node_id, _new_node_state(node_id)) + node["packet_count"] += 1 + if node["first_seen_s"] is None: + node["first_seen_s"] = now + node["last_seen_s"] = now + node["last_source"] = addr[0] + node["last_port"] = addr[1] + node["last_len"] = len(data) + node["last_head_hex"] = data[:24].hex() + node["last_magic"] = f"0x{magic:08x}" if magic is not None else None + node["last_packet_type"] = packet_type + node["packet_types"][packet_type] = node["packet_types"].get(packet_type, 0) + 1 + if feature_state is not None: + CARDPUTER_STATE["feature_state"] = feature_state + CARDPUTER_STATE["feature_state_seen_s"] = now + if node_id is not None: + node["feature_state"] = feature_state + node["feature_state_seen_s"] = now + if battery is not None: + CARDPUTER_STATE["battery"] = battery + CARDPUTER_STATE["battery_seen_s"] = now + if node_id is not None: + node["battery"] = battery + node["battery_seen_s"] = now + if edge_vitals is not None and node_id is not None: + node["edge_vitals"] = edge_vitals + node["edge_vitals_seen_s"] = now + node["rssi_dbm"] = edge_vitals["rssi_dbm"] + node["rssi_seen_s"] = now + if edge_feature is not None: + CARDPUTER_STATE["edge_feature"] = edge_feature + CARDPUTER_STATE["edge_feature_seen_s"] = now + if node_id is not None: + node["edge_feature"] = edge_feature + node["edge_feature_seen_s"] = now + if sync_packet is not None: + CARDPUTER_STATE["sync_packet"] = sync_packet + CARDPUTER_STATE["sync_packet_seen_s"] = now + if node_id is not None: + node["sync_packet"] = sync_packet + node["sync_packet_seen_s"] = now + if adaptive_state is not None: + CARDPUTER_STATE["adaptive_state"] = adaptive_state + CARDPUTER_STATE["adaptive_state_seen_s"] = now + if node_id is not None: + node["adaptive_state"] = adaptive_state + node["adaptive_state_seen_s"] = now + if csi_signal is not None and node_id is not None: + node["rssi_dbm"] = csi_signal["rssi_dbm"] + node["noise_floor_dbm"] = csi_signal["noise_floor_dbm"] + node["rssi_seen_s"] = now + CARDPUTER_STATE["udp_error"] = None + + +def _json_for(method: str, path: str, body: dict | None = None) -> tuple[int, dict] | None: + body = body or {} + uptime = int(time.time() - STARTED) + cardputer = _cardputer_snapshot() + if path in {"/health", "/health/live"}: + return 200, {"status": "alive", "ok": True, "timestamp": _now_iso(), "uptime_s": uptime} + if path == "/health/ready": + return 200, {"status": "ready", "checks": {"ui": "ready", "hardware_api": "ready"}} + if path == "/health/version": + return 200, {"name": "RuView Desktop Live API", "version": "local", "environment": "desktop"} + if path in {"/health/health", "/api/v1/metrics"}: + feature_state = cardputer.get("feature_state") or {} + inference = _hardware_inference_state(cardputer, feature_state) + return 200, { + "status": "healthy", + "timestamp": _now_iso(), + "components": { + "api": {"status": "healthy", "message": "Desktop API running"}, + "hardware": { + "status": "healthy" if cardputer["live"] else "warning", + "message": cardputer["message"], + }, + "battery": { + "status": "healthy" if cardputer["battery_live"] else "warning", + "message": ( + f"{cardputer['battery']['percent']}% {cardputer['battery']['status']}" + if cardputer["battery"].get("valid") + else "Battery telemetry unknown" + ), + }, + "inference": { + "status": "healthy" if inference["live"] else "warning", + "message": inference["message"], + }, + "streaming": {"status": "healthy" if cardputer["live"] else "warning", "message": "Hardware telemetry only"}, + }, + "system_metrics": { + "cpu": {"percent": None}, + "memory": {"percent": None}, + "disk": {"percent": None}, + }, + } + if path in {"/api/v1/info", "/api/v1/status"}: + return 200, { + "name": "RuView Desktop API", + "version": "local", + "environment": "desktop", + "source": "esp32" if cardputer["live"] else "none", + "hardware": cardputer, + "services": { + "api": "running", + "hardware": "live" if cardputer["live"] else "waiting", + "inference": "ready", + "streaming": "active", + }, + "features": {"pose_estimation": True, "streaming": True, "multi_zone": True, "real_time": True}, + "uptime_s": uptime, + } + if path == "/api/v1/pose/current": + feature_state = cardputer.get("feature_state") or {} + inference = _hardware_inference_state(cardputer, feature_state) + person_count = inference["person_count"] + real_presence = person_count > 0 + pose_source = inference["source"] + source = ( + "cardputer-adv-feature-state" + if inference["feature_state_live"] + else "cardputer-adv-edge-vitals" + if inference["edge_vitals_live"] + else "cardputer-adv-edge-feature" + if inference["edge_feature_live"] + else "cardputer-stale-feature-state" + ) + return 200, { + "timestamp": _now_iso(), + "persons": [], + "total_persons": person_count, + "pose_source": pose_source, + "processing_time": 0.0, + "zone_id": "cardputer-adv", + "total_detections": cardputer["packet_count"], + "metadata": { + "mock_data": False, + "source": source, + "pose_available": inference["live"], + "inference_available": inference["live"], + "inference_source": inference["source"], + "inference_message": inference["message"], + "cardputer_udp_live": cardputer["live"], + "cardputer_packets": cardputer["packet_count"], + "cardputer_last_source": cardputer["last_source"], + "battery": cardputer["battery"], + "battery_live": cardputer["battery_live"], + "battery_age_s": cardputer["battery_age_s"], + "feature_state_live": cardputer["feature_state_live"], + "feature_state_age_s": cardputer["feature_state_age_s"], + "stale_feature_state": cardputer["stale_feature_state"], + "pass": cardputer["pass"], + "freshness_status": cardputer["freshness_status"], + "presence": real_presence, + "n_persons": person_count, + "presence_score": feature_state.get("presence_score"), + "motion_score": feature_state.get("motion_score"), + "crc_valid": feature_state.get("crc_valid"), + }, + } + if path == "/api/v1/presence/current": + feature_state = cardputer.get("feature_state") or {} + person_count = _active_person_count(cardputer, feature_state) + presence = person_count > 0 + source = "cardputer-adv-feature-state" if cardputer["feature_state_live"] else "none" + return 200, { + "timestamp": _now_iso(), + "source": source, + "presence": presence, + "n_persons": person_count, + "confidence": min(1.0, max(0.0, float(feature_state.get("presence_score", 0.0) or 0.0))), + "raw_presence_score": feature_state.get("presence_score", 0.0), + "motion_score": feature_state.get("motion_score", 0.0), + "feature_state_live": cardputer["feature_state_live"], + "feature_state_age_s": cardputer["feature_state_age_s"], + "stale_feature_state": cardputer["stale_feature_state"], + "pass": cardputer["pass"], + "freshness_status": cardputer["freshness_status"], + "cardputer": cardputer, + } + if path == "/api/v1/pose/stats": + return 200, { + "total_detections": cardputer["packet_count"], + "average_confidence": None, + "peak_persons": None, + "hours_analyzed": 1, + } + if path == "/api/v1/pose/zones/summary": + return 200, {"zones": {}} + if path in {"/api/v1/stream/status", "/api/v1/stream/metrics"}: + return 200, {"is_active": cardputer["live"], "connected_clients": 0, "messages_sent": cardputer["packet_count"], "uptime": uptime} + if method == "POST" and path in {"/api/v1/stream/start", "/api/v1/stream/stop"}: + return 200, {"status": "waiting" if not cardputer["live"] else "active", "message": "Hardware stream only"} + if path == "/api/v1/models": + models = _scan_models() + return 200, {"models": models, "count": len(models)} + if path == "/api/v1/models/active": + return 200, _active_model_response() + if method == "POST" and path == "/api/v1/models/load": + return _load_model_request(body) + if method == "POST" and path == "/api/v1/models/unload": + return _unload_model_request() + if path == "/api/v1/models/lora/profiles": + return 200, {"profiles": _scan_lora_profiles()} + if method == "POST" and path == "/api/v1/models/lora/activate": + profile = body.get("profile_name") or body.get("profile") or body.get("name") + if not profile: + return 400, {"status": "error", "error": "missing profile", "message": "Missing LoRA profile name."} + return 200, {"success": True, "status": "activated", "profile_name": str(profile)} + if method == "GET" and path.startswith("/api/v1/models/"): + return _get_model_response(path.rsplit("/", 1)[-1]) + if method == "DELETE" and path.startswith("/api/v1/models/"): + return _delete_model(path.rsplit("/", 1)[-1]) + if path == "/api/v1/recording/list": + recordings = _scan_recordings() + return 200, {"recordings": recordings, "count": len(recordings)} + if method == "POST" and path == "/api/v1/recording/start": + return _start_recording(body) + if method == "POST" and path == "/api/v1/recording/stop": + return _stop_recording() + if method == "DELETE" and path.startswith("/api/v1/recording/"): + return _delete_recording(path.rsplit("/", 1)[-1]) + if path == "/api/v1/train/status": + return 200, _training_snapshot() + if path == "/api/v1/train/rvf/readiness": + return 200, _rvf_training_readiness(cardputer) + if method == "POST" and path == "/api/v1/train/start": + return _start_training_request("supervised", body) + if method == "POST" and path == "/api/v1/train/pretrain": + return _start_training_request("pretrain", body) + if method == "POST" and path == "/api/v1/train/lora": + return _start_training_request("lora", body) + if method == "POST" and path == "/api/v1/train/stop": + return _stop_training_request() + if path == "/api/v1/cardputer/status": + return 200, cardputer + return None + + +class Handler(SimpleHTTPRequestHandler): + def __init__(self, *args, **kwargs): + super().__init__(*args, directory=str(UI_DIR), **kwargs) + + def end_headers(self) -> None: + self.send_header("Cache-Control", "no-store") + self.send_header("Pragma", "no-cache") + super().end_headers() + + def _send_json(self, code: int, body: dict) -> None: + payload = json.dumps(body).encode("utf-8") + self.send_response(code) + self.send_header("Content-Type", "application/json") + self.send_header("Access-Control-Allow-Origin", "*") + self.send_header("Cache-Control", "no-store") + self.send_header("Content-Length", str(len(payload))) + self.end_headers() + self.wfile.write(payload) + + def do_OPTIONS(self) -> None: + self.send_response(204) + self.send_header("Access-Control-Allow-Origin", "*") + self.send_header("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS") + self.send_header("Access-Control-Allow-Headers", "Content-Type, Authorization") + self.end_headers() + + def do_GET(self) -> None: + path = urlparse(self.path).path + result = _json_for("GET", path) + if result is not None: + self._send_json(*result) + return + super().do_GET() + + def do_POST(self) -> None: + path = urlparse(self.path).path + result = _json_for("POST", path, _read_json_body(self)) + if result is not None: + self._send_json(*result) + return + self._send_json(404, {"error": "not found", "path": path}) + + def do_DELETE(self) -> None: + path = urlparse(self.path).path + result = _json_for("DELETE", path) + if result is not None: + self._send_json(*result) + return + self._send_json(404, {"error": "not found", "path": path}) + + +def main() -> int: + if not UI_DIR.is_dir(): + raise SystemExit(f"RuView UI directory not found: {UI_DIR}") + threading.Thread(target=_cardputer_udp_loop, name="cardputer-udp", daemon=True).start() + server = ThreadingHTTPServer(("127.0.0.1", PORT), Handler) + print(f"RuView desktop UI listening on http://127.0.0.1:{PORT}", flush=True) + print(f"Serving UI from {UI_DIR}", flush=True) + server.serve_forever() + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/docs/guardian/visual-language.md b/docs/guardian/visual-language.md new file mode 100644 index 0000000000..f9054573b6 --- /dev/null +++ b/docs/guardian/visual-language.md @@ -0,0 +1,43 @@ +# Guardian Visual Language + +Guardian is the operator-facing HUD and performance-instrument layer for RuView. + +It combines RF telemetry, scene state, audio matrix routing, macro controls, and morphing scenes into a portable tactical instrument interface. + +## Design Principles + +- Instrument first, decoration second. +- Dense but readable. +- Black, white, red as primary system colors. +- CRT/phosphor only where persistence or signal memory matters. +- Swiss grid discipline for layout. +- Every panel must represent live state, stored state, routing, or operator action. +- Avoid decorative cyberpunk language unless it maps to a real signal or control. + +## Product Layers + +- RuView: sensing, CSI, pose, vitals, training, OTA, data plumbing. +- Guardian HUD: live operator interface and scene control. +- RF Forge Jr: hardware/RF architecture and timing spine. +- ALPACA Guardian: RF-to-audio scene engine and performance layer. + +## UI Objects + +- Screen: a visible state surface. +- Scene: a stored configuration of macros, routing, audio state, and RF response. +- Morph: continuous interpolation between two scenes. +- Macro: a controllable performance dimension. +- Matrix: routing truth. +- Ribbon: continuous frequency/control surface. +- Phosphor trace: persistence, memory, signal afterimage. + +## Color + +- Black: background, machine body, signal void. +- White/cream: paper, labels, measurement surfaces. +- Red: alert, selected state, action path, transition. +- Green/cyan phosphor: live signal persistence only. + +## Rule + +If a visual element does not correspond to signal, state, control, routing, or operator action, remove it. diff --git a/firmware/esp32-csi-node/README.md b/firmware/esp32-csi-node/README.md index f75d053c29..87cf94082c 100644 --- a/firmware/esp32-csi-node/README.md +++ b/firmware/esp32-csi-node/README.md @@ -42,6 +42,51 @@ python -m esptool --chip esp32s3 --port COM7 --baud 460800 \ For 4 MB boards use `release_bins/esp32-csi-node-4mb.bin` and `release_bins/partition-table-4mb.bin` with `--flash_size 4MB`. +### ESP32-CAM / AI Thinker profiles + +ESP32-CAM boards can join RuView as classic ESP32 CSI nodes. The default +ESP32-CAM profile leaves the OV2640 camera idle; the board contributes WiFi CSI, +edge feature, vitals, batteryless telemetry, and OTA/provision support on the +same UDP protocol as the StickC Plus node. + +Use a separate node ID from the StickC Plus, for example node `2`: + +```bash +cd firmware/esp32-csi-node +./flash_esp32cam.sh /dev/ttyUSB0 + +python provision.py --port /dev/ttyUSB0 --chip esp32 \ + --node-id 2 \ + --ssid "YourSSID" --password "YourPass" \ + --target-ip 192.168.1.20 --target-port 5005 +``` + +If flashing times out, put the ESP32-CAM in bootloader mode: hold `GPIO0` to +GND, reset or power-cycle the board, run the flash command, then release +`GPIO0` and reset again. + +For camera-assisted setup, use the dual profile. It keeps CSI UDP active and +adds low-rate OV2640 MJPEG endpoints on the existing HTTP server at port `8032`. +The defaults are QVGA, JPEG quality `14`, and `4` FPS to avoid crowding the CSI +link. + +```bash +cd firmware/esp32-csi-node +./flash_esp32cam_dual.sh /dev/ttyUSB0 + +python provision.py --port /dev/ttyUSB0 --chip esp32 \ + --node-id 3 \ + --ssid "YourSSID" --password "YourPass" \ + --target-ip 192.168.1.20 --target-port 5005 +``` + +After the node joins WiFi: + +- `http://:8032/cam` opens a simple camera page. +- `http://:8032/stream` serves MJPEG. +- `http://:8032/cam.jpg` captures one JPEG. +- `http://:8032/cam/status` reports the camera profile. + ### 1. Build (Docker -- the only reliable method) ```bash diff --git a/firmware/esp32-csi-node/flash_esp32cam.sh b/firmware/esp32-csi-node/flash_esp32cam.sh new file mode 100755 index 0000000000..5f1791a414 --- /dev/null +++ b/firmware/esp32-csi-node/flash_esp32cam.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +set -euo pipefail + +PORT="${1:-/dev/ttyUSB0}" +BAUD="${BAUD:-460800}" +IDF_PATH="${IDF_PATH:-/run/media/deck/SDCARD/offload/toolchains/esp/esp-idf}" +BUILD_DIR="build-esp32cam" +SDKCONFIG_PATH="${BUILD_DIR}/sdkconfig" + +if [[ ! -f "${IDF_PATH}/export.sh" ]]; then + echo "ESP-IDF export.sh not found at ${IDF_PATH}" >&2 + echo "Set IDF_PATH to your ESP-IDF checkout and retry." >&2 + exit 1 +fi + +source "${IDF_PATH}/export.sh" >/dev/null + +idf.py -B "${BUILD_DIR}" \ + -D "SDKCONFIG=${SDKCONFIG_PATH}" \ + -D "SDKCONFIG_DEFAULTS=sdkconfig.defaults.esp32cam" \ + set-target esp32 build + +python -m esptool --chip esp32 --port "${PORT}" --baud "${BAUD}" \ + write_flash --flash_mode dio --flash_size 4MB \ + 0x1000 "${BUILD_DIR}/bootloader/bootloader.bin" \ + 0x8000 "${BUILD_DIR}/partition_table/partition-table.bin" \ + 0xf000 "${BUILD_DIR}/ota_data_initial.bin" \ + 0x20000 "${BUILD_DIR}/esp32-csi-node.bin" + +echo "ESP32-CAM RuView firmware flashed on ${PORT}." +echo "Provision it next, for example:" +echo " python provision.py --port ${PORT} --chip esp32 --node-id 2 --ssid --password --target-ip --target-port 5005" diff --git a/firmware/esp32-csi-node/flash_esp32cam_dual.sh b/firmware/esp32-csi-node/flash_esp32cam_dual.sh new file mode 100755 index 0000000000..b08342c500 --- /dev/null +++ b/firmware/esp32-csi-node/flash_esp32cam_dual.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +set -euo pipefail + +PORT="${1:-/dev/ttyUSB0}" +BAUD="${BAUD:-460800}" +IDF_PATH="${IDF_PATH:-/run/media/deck/SDCARD/offload/toolchains/esp/esp-idf}" +BUILD_DIR="build-esp32cam-dual" +SDKCONFIG_PATH="${BUILD_DIR}/sdkconfig" + +if [[ ! -f "${IDF_PATH}/export.sh" ]]; then + echo "ESP-IDF export.sh not found at ${IDF_PATH}" >&2 + echo "Set IDF_PATH to your ESP-IDF checkout and retry." >&2 + exit 1 +fi + +source "${IDF_PATH}/export.sh" >/dev/null + +idf.py -B "${BUILD_DIR}" \ + -D "SDKCONFIG=${SDKCONFIG_PATH}" \ + -D "SDKCONFIG_DEFAULTS=sdkconfig.defaults.esp32cam-dual" \ + set-target esp32 build + +python -m esptool --chip esp32 --port "${PORT}" --baud "${BAUD}" \ + write_flash --flash_mode dio --flash_size 4MB \ + 0x1000 "${BUILD_DIR}/bootloader/bootloader.bin" \ + 0x8000 "${BUILD_DIR}/partition_table/partition-table.bin" \ + 0xf000 "${BUILD_DIR}/ota_data_initial.bin" \ + 0x20000 "${BUILD_DIR}/esp32-csi-node.bin" + +echo "ESP32-CAM RuView dual CSI + MJPEG firmware flashed on ${PORT}." +echo "Provision it next, for example:" +echo " python provision.py --port ${PORT} --chip esp32 --node-id 3 --ssid --password --target-ip --target-port 5005" +echo "After it joins WiFi, open:" +echo " http://:8032/cam" diff --git a/firmware/esp32-csi-node/main/CMakeLists.txt b/firmware/esp32-csi-node/main/CMakeLists.txt index 0cabf5df2a..b31c11aa8f 100644 --- a/firmware/esp32-csi-node/main/CMakeLists.txt +++ b/firmware/esp32-csi-node/main/CMakeLists.txt @@ -1,6 +1,7 @@ set(SRCS "main.c" "csi_collector.c" "stream_sender.c" "nvs_config.c" "edge_processing.c" "ota_update.c" "power_mgmt.c" + "battery_monitor.c" "wasm_runtime.c" "wasm_upload.c" "rvf_parser.c" "mmwave_sensor.c" "swarm_bridge.c" @@ -31,13 +32,19 @@ set(REQUIRES esp_app_format esp_timer esp_pm + heap + esp_adc esp_driver_uart esp_driver_gpio esp_driver_spi esp_driver_i2c + esp_driver_i2s + esp_driver_sdspi driver + fatfs lwip mbedtls + sdmmc ) # ADR-110: C6-only components — pulled in when building for esp32c6. @@ -58,10 +65,19 @@ if(CONFIG_DISPLAY_ENABLE) list(APPEND REQUIRES esp_lcd esp_lcd_touch lvgl) endif() +if(CONFIG_CARDPUTER_ADV_AUDIO_ENABLE) + list(APPEND SRCS "cardputer_adv_audio.c") +endif() + if(CONFIG_WASM_ENABLE) list(APPEND REQUIRES wasm3) endif() +if(CONFIG_ESP32CAM_DUAL_FIRMWARE) + list(APPEND SRCS "esp32cam_dual_stream.c") + list(APPEND REQUIRES esp32-camera) +endif() + idf_component_register( SRCS ${SRCS} INCLUDE_DIRS "." diff --git a/firmware/esp32-csi-node/main/Kconfig.projbuild b/firmware/esp32-csi-node/main/Kconfig.projbuild index 18c32ebf94..593aff670e 100644 --- a/firmware/esp32-csi-node/main/Kconfig.projbuild +++ b/firmware/esp32-csi-node/main/Kconfig.projbuild @@ -41,6 +41,140 @@ menu "CSI Node Configuration" endmenu +menu "Cardputer Battery Monitor" + + config BATTERY_MONITOR_ENABLE + bool "Enable Cardputer battery voltage/status monitor" + default y + help + Read battery voltage from an ADC-capable board battery pin + and publish it to the display UI plus RuView UDP telemetry. + + config BATTERY_ADC_GPIO + int "Battery voltage ADC GPIO" + default 38 if IDF_TARGET_ESP32 + default 10 + range 0 48 + depends on BATTERY_MONITOR_ENABLE + help + M5StickC Plus2 documents battery voltage detection on G38. + Cardputer-Adv documents battery voltage detection on G10. + + config BATTERY_ADC_DIVIDER_MILLI + int "Battery ADC divider multiplier x1000" + default 2000 + range 1000 4000 + depends on BATTERY_MONITOR_ENABLE + help + Multiplier applied to calibrated ADC millivolts. 2000 means + a 2:1 divider: pack_mV = adc_mV * 2. Adjust if hardware + measurement proves a different divider ratio. + + config BATTERY_CHARGE_GPIO + int "Charge-status GPIO (-1 if unavailable)" + default -1 + range -1 48 + depends on BATTERY_MONITOR_ENABLE + help + Optional charger-status input. Leave -1 when the board does + not expose a charge-status pin to firmware. + + config BATTERY_CHARGE_ACTIVE_LEVEL + int "Charge-status active level" + default 1 + range 0 1 + depends on BATTERY_MONITOR_ENABLE && BATTERY_CHARGE_GPIO >= 0 + + config BATTERY_SEND_INTERVAL_MS + int "Battery telemetry send interval (ms)" + default 5000 + range 1000 60000 + depends on BATTERY_MONITOR_ENABLE + +endmenu + +menu "ESP32-CAM Dual CSI + MJPEG" + depends on IDF_TARGET_ESP32 + + config ESP32CAM_DUAL_FIRMWARE + bool "Enable ESP32-CAM dual CSI + MJPEG stream" + default n + help + Start the OV2640 camera on AI Thinker style ESP32-CAM boards and + expose /cam, /cam.jpg, /stream, and /cam/status on the existing + OTA HTTP server. CSI UDP telemetry remains active. This is meant + for low-rate training-label preview, not high-FPS video. + + config ESP32CAM_STREAM_FPS + int "MJPEG stream FPS cap" + default 4 + range 1 12 + depends on ESP32CAM_DUAL_FIRMWARE + help + Lower values preserve WiFi airtime for CSI packets. Start at 4 FPS. + + choice ESP32CAM_FRAME_SIZE + prompt "Camera frame size" + default ESP32CAM_FRAME_QVGA + depends on ESP32CAM_DUAL_FIRMWARE + + config ESP32CAM_FRAME_QQVGA + bool "160x120 QQVGA" + + config ESP32CAM_FRAME_QVGA + bool "320x240 QVGA" + + config ESP32CAM_FRAME_VGA + bool "640x480 VGA" + endchoice + + config ESP32CAM_JPEG_QUALITY + int "JPEG quality" + default 14 + range 10 30 + depends on ESP32CAM_DUAL_FIRMWARE + help + ESP camera driver quality value. Lower numbers are better quality + and larger packets. 14 is a conservative CSI-friendly default. + + config ESP32CAM_XCLK_FREQ_HZ + int "XCLK frequency" + default 10000000 + range 5000000 20000000 + depends on ESP32CAM_DUAL_FIRMWARE + + config ESP32CAM_PIN_PWDN + int "PWDN GPIO" + default 32 + range -1 39 + depends on ESP32CAM_DUAL_FIRMWARE + + config ESP32CAM_PIN_RESET + int "RESET GPIO" + default -1 + range -1 39 + depends on ESP32CAM_DUAL_FIRMWARE + + config ESP32CAM_PIN_XCLK + int "XCLK GPIO" + default 0 + range 0 39 + depends on ESP32CAM_DUAL_FIRMWARE + + config ESP32CAM_PIN_SIOD + int "SIOD/SDA GPIO" + default 26 + range 0 39 + depends on ESP32CAM_DUAL_FIRMWARE + + config ESP32CAM_PIN_SIOC + int "SIOC/SCL GPIO" + default 27 + range 0 39 + depends on ESP32CAM_DUAL_FIRMWARE + +endmenu + menu "Edge Intelligence (ADR-039)" config EDGE_TIER @@ -170,6 +304,86 @@ menu "Adaptive Controller (ADR-081)" endmenu +menu "Cardputer-Adv Audio" + depends on IDF_TARGET_ESP32S3 + + config CARDPUTER_ADV_AUDIO_ENABLE + bool "Enable Cardputer-Adv ES8311 speaker and SD WAV support" + default y + help + Initialize the Cardputer-Adv ES8311 codec on G8/G9 and drive + the internal speaker over I2S pins G41/G43/G42. This is kept + S3-only because classic StickC/PICO and C6 RuView nodes do not + have the Cardputer-Adv audio circuit. + + config CARDPUTER_ADV_AUDIO_BOOT_CHIME + bool "Play short boot chime" + default y + depends on CARDPUTER_ADV_AUDIO_ENABLE + help + Play a short synthesized tone after the display task starts. + This proves the ES8311 and I2S path without requiring an SD card. + + config CARDPUTER_ADV_AUDIO_TRY_SD_WAV + bool "Try WAV proof from microSD" + default y + depends on CARDPUTER_ADV_AUDIO_ENABLE + help + Mount the Cardputer-Adv SPI microSD slot and play a PCM WAV file + if one exists. Missing media or missing files are logged but do + not block WiFi, CSI streaming, or OTA startup. + + config CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT + string "microSD mount point" + default "/sdcard" + depends on CARDPUTER_ADV_AUDIO_ENABLE + + config CARDPUTER_ADV_AUDIO_SD_WAV_PATH + string "Preferred WAV path" + default "/sdcard/ruview.wav" + depends on CARDPUTER_ADV_AUDIO_ENABLE && CARDPUTER_ADV_AUDIO_TRY_SD_WAV + help + Preferred PCM WAV file to play during the boot-time SD proof. + Supported input is unsigned 8-bit or signed 16-bit PCM, mono + or stereo. The M5Unified example's file1.wav/file2.wav/file3.wav + names are also tried as fallbacks. + + config CARDPUTER_ADV_AUDIO_VOLUME_PERCENT + int "Speaker volume percent" + default 35 + range 0 100 + depends on CARDPUTER_ADV_AUDIO_ENABLE + + config CARDPUTER_ADV_TRAUTONIUM_ENABLE + bool "Enable playable keyboard Trautonium voice" + default y + depends on CARDPUTER_ADV_AUDIO_ENABLE + help + Route Cardputer-Adv TCA8418 key press/release events into a + lightweight embedded instrument: trapezoid oscillator, subharmonic + body, portamento glide, pressure-scaled drive, and vowel/formant + color banks. This does not require an SD card. + + config CARDPUTER_ADV_TRAUTONIUM_PORTAMENTO_MS + int "Trautonium key glide time (ms)" + default 220 + range 20 1200 + depends on CARDPUTER_ADV_TRAUTONIUM_ENABLE + + config CARDPUTER_ADV_TRAUTONIUM_BASE_FREQ_HZ + int "Trautonium lowest key frequency (Hz)" + default 131 + range 40 220 + depends on CARDPUTER_ADV_TRAUTONIUM_ENABLE + + config CARDPUTER_ADV_TRAUTONIUM_LEVEL_PERCENT + int "Trautonium synth level percent" + default 58 + range 0 100 + depends on CARDPUTER_ADV_TRAUTONIUM_ENABLE + +endmenu + menu "AMOLED Display (ADR-045)" config DISPLAY_ENABLE diff --git a/firmware/esp32-csi-node/main/battery_monitor.c b/firmware/esp32-csi-node/main/battery_monitor.c new file mode 100644 index 0000000000..f030b47d97 --- /dev/null +++ b/firmware/esp32-csi-node/main/battery_monitor.c @@ -0,0 +1,168 @@ +#include "battery_monitor.h" +#include "sdkconfig.h" + +#if CONFIG_BATTERY_MONITOR_ENABLE + +#include +#include "driver/gpio.h" +#include "esp_adc/adc_cali.h" +#include "esp_adc/adc_cali_scheme.h" +#include "esp_adc/adc_oneshot.h" +#include "esp_log.h" + +static const char *TAG = "battery"; + +static bool s_init_attempted; +static bool s_ready; +static adc_oneshot_unit_handle_t s_adc_unit; +static adc_cali_handle_t s_cali; +static bool s_cali_ready; +static adc_channel_t s_channel; + +static uint8_t percent_from_mv(uint16_t mv) +{ + if (mv >= 4200) return 100; + if (mv <= 3300) return 0; + if (mv >= 3900) return (uint8_t)(70 + ((uint32_t)(mv - 3900) * 30U) / 300U); + if (mv >= 3700) return (uint8_t)(35 + ((uint32_t)(mv - 3700) * 35U) / 200U); + if (mv >= 3500) return (uint8_t)(10 + ((uint32_t)(mv - 3500) * 25U) / 200U); + return (uint8_t)(((uint32_t)(mv - 3300) * 10U) / 200U); +} + +const char *battery_monitor_status_name(battery_power_status_t status) +{ + switch (status) { + case BATTERY_POWER_BATTERY: + return "BATTERY"; + case BATTERY_POWER_CHARGING: + return "CHARGING"; + case BATTERY_POWER_UNKNOWN: + default: + return "UNKNOWN"; + } +} + +esp_err_t battery_monitor_init(void) +{ + if (s_ready) return ESP_OK; + if (s_init_attempted) return ESP_ERR_INVALID_STATE; + s_init_attempted = true; + + adc_unit_t unit_id; + esp_err_t err = adc_oneshot_io_to_channel(CONFIG_BATTERY_ADC_GPIO, &unit_id, &s_channel); + if (err != ESP_OK) { + ESP_LOGW(TAG, "GPIO%d is not ADC-capable: %s", CONFIG_BATTERY_ADC_GPIO, esp_err_to_name(err)); + return err; + } + + adc_oneshot_unit_init_cfg_t unit_cfg = { + .unit_id = unit_id, + }; + err = adc_oneshot_new_unit(&unit_cfg, &s_adc_unit); + if (err != ESP_OK) { + ESP_LOGW(TAG, "ADC unit init failed: %s", esp_err_to_name(err)); + return err; + } + + adc_oneshot_chan_cfg_t chan_cfg = { + .bitwidth = ADC_BITWIDTH_DEFAULT, + .atten = ADC_ATTEN_DB_12, + }; + err = adc_oneshot_config_channel(s_adc_unit, s_channel, &chan_cfg); + if (err != ESP_OK) { + ESP_LOGW(TAG, "ADC channel init failed: %s", esp_err_to_name(err)); + return err; + } + +#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED + adc_cali_curve_fitting_config_t cali_cfg = { + .unit_id = unit_id, + .chan = s_channel, + .atten = ADC_ATTEN_DB_12, + .bitwidth = ADC_BITWIDTH_DEFAULT, + }; + s_cali_ready = (adc_cali_create_scheme_curve_fitting(&cali_cfg, &s_cali) == ESP_OK); +#endif + +#if CONFIG_BATTERY_CHARGE_GPIO >= 0 + gpio_config_t gpio_cfg = { + .pin_bit_mask = 1ULL << CONFIG_BATTERY_CHARGE_GPIO, + .mode = GPIO_MODE_INPUT, + .pull_up_en = GPIO_PULLUP_DISABLE, + .pull_down_en = GPIO_PULLDOWN_DISABLE, + .intr_type = GPIO_INTR_DISABLE, + }; + gpio_config(&gpio_cfg); +#endif + + s_ready = true; + ESP_LOGI(TAG, "battery monitor on GPIO%d ADC unit=%d channel=%d divider=%d/1000", + CONFIG_BATTERY_ADC_GPIO, (int)unit_id, (int)s_channel, + CONFIG_BATTERY_ADC_DIVIDER_MILLI); + return ESP_OK; +} + +esp_err_t battery_monitor_read(battery_status_t *out) +{ + if (!out) return ESP_ERR_INVALID_ARG; + memset(out, 0, sizeof(*out)); + out->status = BATTERY_POWER_UNKNOWN; + + if (!s_ready) { + esp_err_t init_err = battery_monitor_init(); + if (init_err != ESP_OK && !s_ready) return init_err; + } + + int raw = 0; + esp_err_t err = adc_oneshot_read(s_adc_unit, s_channel, &raw); + if (err != ESP_OK) return err; + + int adc_mv = 0; + if (s_cali_ready) { + err = adc_cali_raw_to_voltage(s_cali, raw, &adc_mv); + if (err != ESP_OK) return err; + } else { + adc_mv = (raw * 3300) / 4095; + } + + uint32_t pack_mv = ((uint32_t)adc_mv * (uint32_t)CONFIG_BATTERY_ADC_DIVIDER_MILLI) / 1000U; + if (pack_mv > UINT16_MAX) pack_mv = UINT16_MAX; + + out->valid = true; + out->millivolts = (uint16_t)pack_mv; + out->percent = percent_from_mv(out->millivolts); + +#if CONFIG_BATTERY_CHARGE_GPIO >= 0 + out->charging = gpio_get_level(CONFIG_BATTERY_CHARGE_GPIO) == CONFIG_BATTERY_CHARGE_ACTIVE_LEVEL; +#else + out->charging = false; +#endif + out->status = out->charging ? BATTERY_POWER_CHARGING : BATTERY_POWER_BATTERY; + return ESP_OK; +} + +#else + +const char *battery_monitor_status_name(battery_power_status_t status) +{ + (void)status; + return "UNKNOWN"; +} + +esp_err_t battery_monitor_init(void) +{ + return ESP_ERR_NOT_SUPPORTED; +} + +esp_err_t battery_monitor_read(battery_status_t *out) +{ + if (!out) return ESP_ERR_INVALID_ARG; + out->valid = false; + out->millivolts = 0; + out->percent = 0; + out->charging = false; + out->status = BATTERY_POWER_UNKNOWN; + return ESP_ERR_NOT_SUPPORTED; +} + +#endif /* CONFIG_BATTERY_MONITOR_ENABLE */ diff --git a/firmware/esp32-csi-node/main/battery_monitor.h b/firmware/esp32-csi-node/main/battery_monitor.h new file mode 100644 index 0000000000..ad8064d946 --- /dev/null +++ b/firmware/esp32-csi-node/main/battery_monitor.h @@ -0,0 +1,26 @@ +#ifndef BATTERY_MONITOR_H +#define BATTERY_MONITOR_H + +#include +#include +#include "esp_err.h" + +typedef enum { + BATTERY_POWER_UNKNOWN = 0, + BATTERY_POWER_BATTERY = 1, + BATTERY_POWER_CHARGING = 2, +} battery_power_status_t; + +typedef struct { + bool valid; + uint16_t millivolts; + uint8_t percent; + bool charging; + battery_power_status_t status; +} battery_status_t; + +esp_err_t battery_monitor_init(void); +esp_err_t battery_monitor_read(battery_status_t *out); +const char *battery_monitor_status_name(battery_power_status_t status); + +#endif /* BATTERY_MONITOR_H */ diff --git a/firmware/esp32-csi-node/main/c6_sync_espnow.c b/firmware/esp32-csi-node/main/c6_sync_espnow.c index fbdf405466..86169d0883 100644 --- a/firmware/esp32-csi-node/main/c6_sync_espnow.c +++ b/firmware/esp32-csi-node/main/c6_sync_espnow.c @@ -6,16 +6,23 @@ * but over ESP-NOW instead of 802.15.4 because the IDF v5.4 ieee802154 RX * path doesn't deliver frames to user-space (see WITNESS-LOG-110 §D1). * - * Frame layout (16 bytes payload, broadcast MAC FF:FF:FF:FF:FF:FF): + * Frame layout (24 bytes payload, broadcast MAC FF:FF:FF:FF:FF:FF): * [0..3] Magic 0x53454E50 ('SENP' — Sync via ESP-NOW) * [4] Protocol ver 0x01 * [5] Leader flag 1 if sender claims leader - * [6..7] Reserved + * [6] Node ID + * [7] Battery percent, 0-100 valid, 255 unknown * [8..15] Leader epoch µs (LE u64) + * [16] Battery flags: bit0=valid, bit1=charging + * [17] Battery status + * [18..19] Battery millivolts + * [20..23] Reserved */ #include "sdkconfig.h" #include "c6_sync_espnow.h" +#include "battery_monitor.h" +#include "csi_collector.h" #include "esp_log.h" #include "esp_now.h" #include "esp_wifi.h" @@ -31,13 +38,19 @@ static const char *TAG = "c6_espnow"; #define BEACON_PROTO_VER 0x01 #define BEACON_PERIOD_MS 100 #define VALID_WINDOW_MS 3000 +#define BEACON_LEGACY_SIZE 16 typedef struct __attribute__((packed)) { uint32_t magic; uint8_t proto_ver; uint8_t leader_flag; - uint16_t _reserved; + uint8_t node_id; + uint8_t battery_percent; uint64_t leader_epoch_us; + uint8_t battery_flags; + uint8_t battery_status; + uint16_t battery_millivolts; + uint32_t _reserved; } espnow_beacon_t; static const uint8_t s_broadcast_mac[6] = {0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; @@ -53,6 +66,13 @@ static uint32_t s_tx_count = 0; static uint32_t s_tx_fail = 0; static uint32_t s_rx_count = 0; static uint32_t s_rx_magic_match = 0; +static c6_espnow_peer_status_t s_peer_status = {0}; +static uint64_t s_peer_seen_us = 0; +static uint8_t s_cached_battery_percent = 255; +static uint8_t s_cached_battery_flags = 0; +static uint8_t s_cached_battery_status = BATTERY_POWER_UNKNOWN; +static uint16_t s_cached_battery_mv = 0; +static uint64_t s_last_battery_refresh_us = 0; /* ADR-110 P10 — EMA-smoothed offset (host-side trajectory in firmware). * @@ -76,14 +96,41 @@ static uint64_t mac6_to_u64(const uint8_t mac[6]) ((uint64_t)mac[4] << 8) | (uint64_t)mac[5]; } +static void refresh_battery_cache(void) +{ + uint64_t now_us = (uint64_t)esp_timer_get_time(); + if ((now_us - s_last_battery_refresh_us) < 1000000ULL) return; + s_last_battery_refresh_us = now_us; + + s_cached_battery_percent = 255; + s_cached_battery_flags = 0; + s_cached_battery_status = BATTERY_POWER_UNKNOWN; + s_cached_battery_mv = 0; + + battery_status_t battery; + if (battery_monitor_read(&battery) == ESP_OK && battery.valid) { + s_cached_battery_percent = battery.percent; + s_cached_battery_flags = 0x01; + if (battery.charging) s_cached_battery_flags |= 0x02; + s_cached_battery_status = (uint8_t)battery.status; + s_cached_battery_mv = battery.millivolts; + } +} + static void send_beacon(void) { + refresh_battery_cache(); espnow_beacon_t b = { .magic = BEACON_MAGIC, .proto_ver = BEACON_PROTO_VER, .leader_flag = s_is_leader ? 1 : 0, - ._reserved = 0, + .node_id = csi_collector_get_node_id(), + .battery_percent = s_cached_battery_percent, .leader_epoch_us = (uint64_t)esp_timer_get_time(), + .battery_flags = s_cached_battery_flags, + .battery_status = s_cached_battery_status, + .battery_millivolts = s_cached_battery_mv, + ._reserved = 0, }; esp_err_t r = esp_now_send(s_broadcast_mac, (uint8_t *)&b, sizeof(b)); s_tx_count++; @@ -110,12 +157,24 @@ static void on_recv(const uint8_t *src_mac, const uint8_t *data, int len) { #endif s_rx_count++; - if (data == NULL || len < (int)sizeof(espnow_beacon_t)) return; + if (data == NULL || len < BEACON_LEGACY_SIZE) return; const espnow_beacon_t *b = (const espnow_beacon_t *)data; if (b->magic != BEACON_MAGIC || b->proto_ver != BEACON_PROTO_VER) return; s_rx_magic_match++; uint64_t sender_id = src_mac ? mac6_to_u64(src_mac) : 0; uint64_t now_us = (uint64_t)esp_timer_get_time(); + bool has_status_ext = len >= (int)sizeof(espnow_beacon_t); + + if (sender_id != 0 && sender_id != s_local_id) { + s_peer_status.valid = true; + s_peer_status.node_id = has_status_ext ? b->node_id : 0; + s_peer_status.percent = has_status_ext ? b->battery_percent : 255; + s_peer_status.flags = has_status_ext ? b->battery_flags : 0; + s_peer_status.status = has_status_ext ? b->battery_status : BATTERY_POWER_UNKNOWN; + s_peer_status.millivolts = has_status_ext ? b->battery_millivolts : 0; + s_peer_status.rx_count = s_rx_magic_match; + s_peer_seen_us = now_us; + } /* Adopt sender as leader if it's claiming leadership AND its ID is * lower than our current leader (or we have no leader). Lowest MAC @@ -237,3 +296,16 @@ uint32_t c6_sync_espnow_tx_count(void) { return s_tx_count; } uint32_t c6_sync_espnow_tx_fail(void) { return s_tx_fail; } uint32_t c6_sync_espnow_rx_count(void) { return s_rx_count; } uint32_t c6_sync_espnow_rx_magic_match(void) { return s_rx_magic_match; } + +bool c6_sync_espnow_get_peer_status(c6_espnow_peer_status_t *out) +{ + if (!out) return false; + if (!s_peer_status.valid) { + memset(out, 0, sizeof(*out)); + return false; + } + *out = s_peer_status; + uint64_t now_us = (uint64_t)esp_timer_get_time(); + out->age_ms = (uint32_t)((now_us - s_peer_seen_us) / 1000ULL); + return out->age_ms < VALID_WINDOW_MS; +} diff --git a/firmware/esp32-csi-node/main/c6_sync_espnow.h b/firmware/esp32-csi-node/main/c6_sync_espnow.h index c899389612..7a97505a83 100644 --- a/firmware/esp32-csi-node/main/c6_sync_espnow.h +++ b/firmware/esp32-csi-node/main/c6_sync_espnow.h @@ -29,6 +29,17 @@ extern "C" { #include #include +typedef struct { + bool valid; + uint8_t node_id; + uint8_t percent; + uint8_t flags; + uint8_t status; + uint16_t millivolts; + uint32_t age_ms; + uint32_t rx_count; +} c6_espnow_peer_status_t; + /** * Initialize the ESP-NOW sync module. Must be called AFTER WiFi STA is * connected (ESP-NOW needs the WiFi driver active). @@ -63,6 +74,14 @@ uint32_t c6_sync_espnow_tx_fail(void); uint32_t c6_sync_espnow_rx_count(void); uint32_t c6_sync_espnow_rx_magic_match(void); +/** + * Return the freshest non-local ESP-NOW node status heard from the mesh. + * + * Battery fields are valid when flags bit0 is set. Older firmware beacons + * still count as peer presence but will not carry battery details. + */ +bool c6_sync_espnow_get_peer_status(c6_espnow_peer_status_t *out); + #ifdef __cplusplus } #endif diff --git a/firmware/esp32-csi-node/main/cardputer_adv_audio.c b/firmware/esp32-csi-node/main/cardputer_adv_audio.c new file mode 100644 index 0000000000..546f12ee11 --- /dev/null +++ b/firmware/esp32-csi-node/main/cardputer_adv_audio.c @@ -0,0 +1,1320 @@ +/** + * @file cardputer_adv_audio.c + * @brief Cardputer-Adv ES8311 speaker and microSD WAV playback. + */ + +#include "cardputer_adv_audio.h" +#include "sdkconfig.h" + +#if defined(CONFIG_CARDPUTER_ADV_AUDIO_ENABLE) && defined(CONFIG_IDF_TARGET_ESP32S3) + +#include +#include +#include +#include "freertos/FreeRTOS.h" +#include "freertos/semphr.h" +#include "freertos/task.h" +#include "driver/gpio.h" +#include "driver/i2c_master.h" +#include "driver/i2s_std.h" +#include "driver/sdspi_host.h" +#include "driver/spi_master.h" +#include "esp_err.h" +#include "esp_log.h" +#include "esp_vfs_fat.h" +#include "sdmmc_cmd.h" + +static const char *TAG = "cardputer_adv_audio"; + +/* Cardputer-Adv pin map from M5Stack docs. */ +#define ADV_CODEC_I2C_PORT I2C_NUM_0 +#define ADV_CODEC_I2C_SDA GPIO_NUM_8 +#define ADV_CODEC_I2C_SCL GPIO_NUM_9 +#define ADV_CODEC_ADDR 0x18 + +#define ADV_KB_INT GPIO_NUM_11 +#define ADV_KB_ADDR 0x34 +#define ADV_KB_REG_CFG 0x01 +#define ADV_KB_REG_INT_STAT 0x02 +#define ADV_KB_REG_KEY_LCK_EC 0x03 +#define ADV_KB_REG_KEY_EVENT 0x04 +#define ADV_KB_REG_GPI_EM1 0x09 +#define ADV_KB_REG_KP_GPIO1 0x1D +#define ADV_KB_REG_KP_GPIO2 0x1E +#define ADV_KB_REG_KP_GPIO3 0x1F + +#define ADV_I2S_PORT I2S_NUM_1 +#define ADV_I2S_BCLK GPIO_NUM_41 +#define ADV_I2S_WS GPIO_NUM_43 +#define ADV_I2S_DOUT GPIO_NUM_42 + +#define ADV_SD_HOST SPI3_HOST +#define ADV_SD_MOSI GPIO_NUM_14 +#define ADV_SD_MISO GPIO_NUM_39 +#define ADV_SD_SCLK GPIO_NUM_40 +#define ADV_SD_CS GPIO_NUM_12 + +#define ADV_CHIME_RATE_HZ 16000 +#define ADV_CHIME_MS 180 +#define ADV_CHIME_STEP 28 +#define ADV_IO_TIMEOUT_MS 1000 +#define ADV_SYNTH_RATE_HZ 22050 +#define ADV_SYNTH_FRAMES 96 +#define ADV_SYNTH_MAX_KEY 80 +#define ADV_SYNTH_BIG_KNOB_DEFAULT 68 +#define ADV_SYNTH_SPRING_A 293 +#define ADV_SYNTH_SPRING_B 421 +#define ADV_SYNTH_SPRING_C 613 + +typedef struct { + uint16_t audio_format; + uint16_t channels; + uint32_t sample_rate; + uint16_t bits_per_sample; + uint32_t data_bytes; +} wav_info_t; + +static i2c_master_bus_handle_t s_i2c_bus; +static i2c_master_dev_handle_t s_codec_dev; +static i2s_chan_handle_t s_i2s_tx; +static uint32_t s_i2s_rate_hz; +static bool s_i2s_running; +static i2c_master_dev_handle_t s_kb_dev; +static bool s_sd_mount_attempted; +static bool s_sd_mounted; +static esp_err_t s_sd_mount_err = ESP_OK; +static sdmmc_card_t *s_sd_card; +static uint8_t s_raw_buf[1024]; +static int16_t s_pcm_buf[1024]; +static SemaphoreHandle_t s_audio_mutex; + +#if defined(CONFIG_CARDPUTER_ADV_TRAUTONIUM_ENABLE) +typedef struct { + uint16_t f1; + uint16_t f2; + uint16_t damp1; + uint16_t damp2; + int16_t gain1; + int16_t gain2; + uint8_t formant_mix; + uint8_t trap_edge; + const char *name; +} trautonium_formant_t; + +typedef struct { + uint8_t key_code; + int8_t semitone; + uint8_t formant_index; + const char *label; +} trautonium_key_note_t; + +static const trautonium_formant_t s_formants[] = { + /* Historical color approximations: neutral, dark, nasal, low throat. */ + {2700, 7600, 11500, 13500, 190, 82, 140, 9, "neutral"}, + {1500, 4300, 13500, 15000, 210, 70, 155, 12, "dark"}, + {4300, 10400, 9000, 12000, 230, 105, 180, 6, "nasal"}, + {1050, 3100, 15000, 16000, 240, 55, 165, 14, "low"}, +}; + +/* + * Cardputer-ADV TCA8418 keycodes are column-scanned, not row-major. This table + * maps printed key labels into a playable typing-keyboard piano layout. + */ +static const trautonium_key_note_t s_key_notes[] = { + /* Number row: high chromatic rail. */ + {5, 24, 0, "1"}, {11, 25, 0, "2"}, {15, 26, 0, "3"}, {21, 27, 0, "4"}, + {25, 28, 0, "5"}, {31, 29, 0, "6"}, {35, 30, 0, "7"}, {41, 31, 0, "8"}, + {45, 32, 0, "9"}, {51, 33, 0, "0"}, {55, 34, 0, "-"}, {61, 35, 0, "="}, + + /* Q row: black-key rail plus duplicate anchors. Printed T is F#4. */ + {6, 12, 2, "Q"}, {12, 13, 2, "W"}, {16, 15, 2, "E"}, {22, 17, 2, "R"}, + {26, 18, 2, "T"}, {32, 20, 2, "Y"}, {36, 22, 2, "U"}, {42, 24, 2, "I"}, + {46, 25, 2, "O"}, {52, 27, 2, "P"}, + + /* A row: white keys, C4 upward. */ + {13, 12, 1, "A"}, {17, 14, 1, "S"}, {23, 16, 1, "D"}, {27, 17, 1, "F"}, + {33, 19, 1, "G"}, {37, 21, 1, "H"}, {43, 23, 1, "J"}, {47, 24, 1, "K"}, + {53, 26, 1, "L"}, + + /* Z row: lower white keys, C3 upward. */ + {18, 0, 3, "Z"}, {24, 2, 3, "X"}, {28, 4, 3, "C"}, {34, 5, 3, "V"}, + {38, 7, 3, "B"}, {44, 9, 3, "N"}, {48, 11, 3, "M"}, +}; + +static TaskHandle_t s_synth_task; +static TaskHandle_t s_keyscan_task; +static portMUX_TYPE s_synth_lock = portMUX_INITIALIZER_UNLOCKED; +static uint8_t s_synth_active_key; +static uint8_t s_synth_formant_index; +static uint8_t s_synth_pressure_percent; +static uint8_t s_synth_big_knob_percent; +static uint8_t s_synth_sh_trigger; +static uint32_t s_synth_target_mhz; +static bool s_synth_gate; +static int16_t s_synth_buf[ADV_SYNTH_FRAMES * 2]; +static int32_t s_spring_a[ADV_SYNTH_SPRING_A]; +static int32_t s_spring_b[ADV_SYNTH_SPRING_B]; +static int32_t s_spring_c[ADV_SYNTH_SPRING_C]; +static uint16_t s_spring_a_pos; +static uint16_t s_spring_b_pos; +static uint16_t s_spring_c_pos; +#endif + +static void ensure_audio_mutex(void) +{ + if (s_audio_mutex == NULL) { + s_audio_mutex = xSemaphoreCreateMutex(); + } +} + +static esp_err_t audio_lock(TickType_t timeout) +{ + ensure_audio_mutex(); + if (s_audio_mutex == NULL) { + return ESP_ERR_NO_MEM; + } + return xSemaphoreTake(s_audio_mutex, timeout) == pdTRUE ? ESP_OK : ESP_ERR_TIMEOUT; +} + +static void audio_unlock(void) +{ + if (s_audio_mutex != NULL) { + xSemaphoreGive(s_audio_mutex); + } +} + +static uint32_t read_le32(const uint8_t *p) +{ + return (uint32_t)p[0] | ((uint32_t)p[1] << 8) | + ((uint32_t)p[2] << 16) | ((uint32_t)p[3] << 24); +} + +static uint16_t read_le16(const uint8_t *p) +{ + return (uint16_t)p[0] | ((uint16_t)p[1] << 8); +} + +static int16_t scale_sample(int16_t sample) +{ + int volume = CONFIG_CARDPUTER_ADV_AUDIO_VOLUME_PERCENT; + if (volume < 0) { + volume = 0; + } else if (volume > 100) { + volume = 100; + } + return (int16_t)(((int32_t)sample * volume) / 100); +} + +static esp_err_t get_i2c_bus(void) +{ + if (s_i2c_bus != NULL) { + return ESP_OK; + } + + esp_err_t err = i2c_master_get_bus_handle(ADV_CODEC_I2C_PORT, &s_i2c_bus); + if (err == ESP_OK) { + return ESP_OK; + } + + i2c_master_bus_config_t bus_cfg = { + .i2c_port = ADV_CODEC_I2C_PORT, + .sda_io_num = ADV_CODEC_I2C_SDA, + .scl_io_num = ADV_CODEC_I2C_SCL, + .clk_source = I2C_CLK_SRC_DEFAULT, + .glitch_ignore_cnt = 7, + .flags.enable_internal_pullup = true, + }; + err = i2c_new_master_bus(&bus_cfg, &s_i2c_bus); + if (err == ESP_ERR_INVALID_STATE) { + err = i2c_master_get_bus_handle(ADV_CODEC_I2C_PORT, &s_i2c_bus); + } + return err; +} + +static esp_err_t get_codec_dev(void) +{ + if (s_codec_dev != NULL) { + return ESP_OK; + } + + esp_err_t err = get_i2c_bus(); + if (err != ESP_OK) { + return err; + } + + i2c_device_config_t dev_cfg = { + .dev_addr_length = I2C_ADDR_BIT_LEN_7, + .device_address = ADV_CODEC_ADDR, + .scl_speed_hz = 400000, + }; + return i2c_master_bus_add_device(s_i2c_bus, &dev_cfg, &s_codec_dev); +} + +static esp_err_t codec_write_reg(uint8_t reg, uint8_t value) +{ + uint8_t data[2] = {reg, value}; + return i2c_master_transmit(s_codec_dev, data, sizeof(data), ADV_IO_TIMEOUT_MS); +} + +static esp_err_t init_codec(void) +{ + static bool codec_ready; + if (codec_ready) { + return ESP_OK; + } + + esp_err_t err = get_codec_dev(); + if (err != ESP_OK) { + return err; + } + + static const struct { + uint8_t reg; + uint8_t value; + } init_seq[] = { + {0x00, 0x80}, + {0x01, 0xB5}, + {0x02, 0x18}, + {0x0D, 0x01}, + {0x12, 0x00}, + {0x13, 0x10}, + {0x32, 0xBF}, + {0x37, 0x08}, + }; + + for (size_t i = 0; i < sizeof(init_seq) / sizeof(init_seq[0]); i++) { + err = codec_write_reg(init_seq[i].reg, init_seq[i].value); + if (err != ESP_OK) { + return err; + } + } + + codec_ready = true; + ESP_LOGI(TAG, "ES8311 ready on I2C0 addr=0x%02x", ADV_CODEC_ADDR); + return ESP_OK; +} + +static void stop_i2s(void) +{ + if (s_i2s_tx == NULL) { + return; + } + if (s_i2s_running) { + i2s_channel_disable(s_i2s_tx); + s_i2s_running = false; + } + i2s_del_channel(s_i2s_tx); + s_i2s_tx = NULL; + s_i2s_rate_hz = 0; +} + +static esp_err_t ensure_i2s(uint32_t sample_rate) +{ + if (s_i2s_tx != NULL && s_i2s_rate_hz == sample_rate) { + return ESP_OK; + } + stop_i2s(); + + i2s_chan_config_t chan_cfg = I2S_CHANNEL_DEFAULT_CONFIG(ADV_I2S_PORT, I2S_ROLE_MASTER); + chan_cfg.dma_desc_num = 4; + chan_cfg.dma_frame_num = 512; + chan_cfg.auto_clear = true; + + esp_err_t err = i2s_new_channel(&chan_cfg, &s_i2s_tx, NULL); + if (err != ESP_OK) { + return err; + } + + i2s_std_config_t std_cfg = { + .clk_cfg = I2S_STD_CLK_DEFAULT_CONFIG(sample_rate), + .slot_cfg = I2S_STD_PHILIPS_SLOT_DEFAULT_CONFIG(I2S_DATA_BIT_WIDTH_16BIT, + I2S_SLOT_MODE_STEREO), + .gpio_cfg = { + .mclk = I2S_GPIO_UNUSED, + .bclk = ADV_I2S_BCLK, + .ws = ADV_I2S_WS, + .dout = ADV_I2S_DOUT, + .din = I2S_GPIO_UNUSED, + }, + }; + err = i2s_channel_init_std_mode(s_i2s_tx, &std_cfg); + if (err != ESP_OK) { + stop_i2s(); + return err; + } + + err = i2s_channel_enable(s_i2s_tx); + if (err != ESP_OK) { + stop_i2s(); + return err; + } + + s_i2s_running = true; + s_i2s_rate_hz = sample_rate; + ESP_LOGI(TAG, "I2S speaker ready: rate=%lu BCLK=G%d LRCK=G%d DOUT=G%d", + (unsigned long)sample_rate, ADV_I2S_BCLK, ADV_I2S_WS, ADV_I2S_DOUT); + return ESP_OK; +} + +static esp_err_t write_i2s_all(const void *data, size_t len) +{ + const uint8_t *cursor = (const uint8_t *)data; + while (len > 0) { + size_t written = 0; + esp_err_t err = i2s_channel_write(s_i2s_tx, cursor, len, &written, + pdMS_TO_TICKS(ADV_IO_TIMEOUT_MS)); + if (err != ESP_OK) { + return err; + } + if (written == 0) { + return ESP_ERR_TIMEOUT; + } + cursor += written; + len -= written; + } + return ESP_OK; +} + +static esp_err_t write_silence(uint32_t sample_rate, uint32_t ms) +{ + memset(s_pcm_buf, 0, sizeof(s_pcm_buf)); + uint32_t frames = (sample_rate * ms) / 1000; + while (frames > 0) { + uint32_t chunk_frames = frames; + if (chunk_frames > sizeof(s_pcm_buf) / (sizeof(int16_t) * 2)) { + chunk_frames = sizeof(s_pcm_buf) / (sizeof(int16_t) * 2); + } + esp_err_t err = write_i2s_all(s_pcm_buf, chunk_frames * 2 * sizeof(int16_t)); + if (err != ESP_OK) { + return err; + } + frames -= chunk_frames; + } + return ESP_OK; +} + +#if defined(CONFIG_CARDPUTER_ADV_TRAUTONIUM_ENABLE) +static int32_t clamp_i32(int32_t value, int32_t lo, int32_t hi) +{ + if (value < lo) { + return lo; + } + if (value > hi) { + return hi; + } + return value; +} + +static const trautonium_key_note_t *trautonium_find_key_note(uint8_t key_code) +{ + for (size_t i = 0; i < sizeof(s_key_notes) / sizeof(s_key_notes[0]); i++) { + if (s_key_notes[i].key_code == key_code) { + return &s_key_notes[i]; + } + } + return NULL; +} + +static uint32_t trautonium_note_freq_mhz(int8_t semitone) +{ + static const uint16_t semitone_ratio_permille[12] = { + 1000, 1059, 1122, 1189, 1259, 1335, + 1414, 1498, 1587, 1682, 1782, 1888, + }; + + if (semitone < 0) { + semitone = 0; + } + + uint32_t freq_mhz = (uint32_t)CONFIG_CARDPUTER_ADV_TRAUTONIUM_BASE_FREQ_HZ * 1000U; + for (int8_t octave = 0; octave < semitone / 12; octave++) { + if (freq_mhz > 2500000U) { + return 5000000U; + } + freq_mhz *= 2U; + } + + freq_mhz = (uint32_t)(((uint64_t)freq_mhz * + semitone_ratio_permille[semitone % 12] + 500ULL) / 1000ULL); + return freq_mhz > 5000000U ? 5000000U : freq_mhz; +} + +static uint32_t trautonium_key_freq_mhz(uint8_t key_code) +{ + const trautonium_key_note_t *note = trautonium_find_key_note(key_code); + return note != NULL ? trautonium_note_freq_mhz(note->semitone) : 0; +} + +static int8_t trautonium_raw_fallback_semitone(uint8_t key_code) +{ + if (key_code == 0) { + return 0; + } + return (int8_t)((key_code - 1) % 36); +} + +static uint8_t trautonium_key_pressure(uint8_t key_code) +{ + if (key_code == 0) { + return 62; + } + uint8_t column = (uint8_t)((key_code - 1) % 10); + uint8_t pressure = (uint8_t)(54 + column * 4); + return pressure > 92 ? 92 : pressure; +} + +static uint8_t trautonium_big_knob_for_key(uint8_t key_code, uint8_t fallback) +{ + static const uint8_t knob_keys[] = { + 5, 11, 15, 21, 25, 31, 35, 41, 45, 51, 55, 61, + }; + + for (size_t i = 0; i < sizeof(knob_keys) / sizeof(knob_keys[0]); i++) { + if (knob_keys[i] == key_code) { + return (uint8_t)(8U + (uint8_t)i * 8U); + } + } + return fallback; +} + +static bool trautonium_handle_macro_key(uint8_t key_code, bool pressed) +{ + int8_t delta = 0; + uint8_t set_value = 0xFF; + + /* + * Likely ADV punctuation raw keys. If a board revision reports these + * differently, note keys still play; the number row remains the reliable + * big-knob rail. + */ + switch (key_code) { + case 54: /* comma: less edge/S&H/spring */ + delta = -10; + break; + case 58: /* period: more edge/S&H/spring */ + delta = 10; + break; + case 64: /* slash: center macro */ + set_value = ADV_SYNTH_BIG_KNOB_DEFAULT; + break; + default: + return false; + } + + if (!pressed) { + return true; + } + + portENTER_CRITICAL(&s_synth_lock); + if (set_value != 0xFF) { + s_synth_big_knob_percent = set_value; + } else { + int32_t next = (int32_t)s_synth_big_knob_percent + delta; + s_synth_big_knob_percent = (uint8_t)clamp_i32(next, 0, 100); + } + s_synth_sh_trigger++; + uint8_t knob = s_synth_big_knob_percent; + portEXIT_CRITICAL(&s_synth_lock); + + ESP_LOGI(TAG, "Trautonium big knob=%u raw=%u", (unsigned)knob, (unsigned)key_code); + return true; +} + +static int32_t trapezoid_wave(uint32_t phase, uint8_t edge_units) +{ + const int32_t amp = 28000; + uint32_t edge = 1024U + (uint32_t)edge_units * 512U; + if (edge > 15000U) { + edge = 15000U; + } + + uint32_t p = phase >> 16; + uint32_t rise_end = edge; + uint32_t high_end = 32768U > edge ? 32768U - edge : 1U; + uint32_t fall_end = 32768U + edge; + uint32_t low_end = 65536U > edge ? 65536U - edge : 65535U; + + if (p < rise_end) { + return -amp + (int32_t)(((uint64_t)p * 2ULL * (uint64_t)amp) / edge); + } + if (p < high_end) { + return amp; + } + if (p < fall_end) { + uint32_t width = fall_end - high_end; + return amp - (int32_t)(((uint64_t)(p - high_end) * 2ULL * (uint64_t)amp) / width); + } + if (p < low_end) { + return -amp; + } + return -amp + (int32_t)(((uint64_t)(p - low_end) * 2ULL * (uint64_t)amp) / edge); +} + +static int32_t svf_bandpass(int32_t input, uint16_t freq, uint16_t damp, + int32_t *low, int32_t *band) +{ + int32_t high = input - *low - (int32_t)(((int64_t)*band * damp) >> 15); + *band += (int32_t)(((int64_t)freq * high) >> 15); + *low += (int32_t)(((int64_t)freq * *band) >> 15); + *band = clamp_i32(*band, -160000, 160000); + *low = clamp_i32(*low, -160000, 160000); + return *band; +} + +static uint32_t approach_u32(uint32_t current, uint32_t target, uint32_t step) +{ + if (current < target) { + uint32_t next = current + step; + return next < current || next > target ? target : next; + } + if (current > target) { + return current - target <= step ? target : current - step; + } + return current; +} + +static void spring_reverb_process(int32_t input, uint8_t big_knob, + int32_t *left, int32_t *right) +{ + int32_t a = s_spring_a[s_spring_a_pos]; + int32_t b = s_spring_b[s_spring_b_pos]; + int32_t c = s_spring_c[s_spring_c_pos]; + int32_t feedback = 116 + ((int32_t)big_knob * 58) / 100; + int32_t cross = 18 + ((int32_t)big_knob * 34) / 100; + int32_t tank_in = input + (((b - c) * cross) >> 8); + + s_spring_a[s_spring_a_pos] = clamp_i32(tank_in + ((a * feedback) >> 8), + -130000, 130000); + s_spring_b[s_spring_b_pos] = clamp_i32((tank_in >> 1) + (a >> 3) + + ((b * (feedback - 12)) >> 8), + -130000, 130000); + s_spring_c[s_spring_c_pos] = clamp_i32((tank_in >> 2) - (b >> 3) + + ((c * (feedback - 24)) >> 8), + -130000, 130000); + + if (++s_spring_a_pos >= ADV_SYNTH_SPRING_A) { + s_spring_a_pos = 0; + } + if (++s_spring_b_pos >= ADV_SYNTH_SPRING_B) { + s_spring_b_pos = 0; + } + if (++s_spring_c_pos >= ADV_SYNTH_SPRING_C) { + s_spring_c_pos = 0; + } + + int32_t wet_l = (a + b - c) / 3; + int32_t wet_r = (c + b - a) / 3; + int32_t mix = 18 + ((int32_t)big_knob * 42) / 100; + *left = ((input * (100 - mix)) + (wet_l * mix)) / 100; + *right = ((input * (100 - mix)) + (wet_r * mix)) / 100; +} + +static void trautonium_synth_task(void *arg) +{ + (void)arg; + uint32_t phase = 0; + uint32_t sub1_phase = 0; + uint32_t sub2_phase = 0; + uint32_t current_mhz = trautonium_key_freq_mhz(18); + uint32_t env = 0; + int32_t f1_low = 0; + int32_t f1_band = 0; + int32_t f2_low = 0; + int32_t f2_band = 0; + uint8_t last_formant = 0xFF; + uint8_t last_sh_trigger = 0; + uint32_t sh_rng = 0x51F15EEDU; + uint32_t sh_counter = 0; + int32_t sh_value = 0; + uint32_t spring_tail = 0; + uint32_t err_count = 0; + + for (;;) { + uint32_t target_mhz; + bool gate; + uint8_t formant_index; + uint8_t pressure; + uint8_t big_knob; + uint8_t sh_trigger; + + portENTER_CRITICAL(&s_synth_lock); + target_mhz = s_synth_target_mhz != 0 ? s_synth_target_mhz : trautonium_key_freq_mhz(18); + gate = s_synth_gate; + formant_index = s_synth_formant_index; + pressure = s_synth_pressure_percent != 0 ? s_synth_pressure_percent : 62; + big_knob = s_synth_big_knob_percent; + sh_trigger = s_synth_sh_trigger; + portEXIT_CRITICAL(&s_synth_lock); + + if (big_knob > 100) { + big_knob = ADV_SYNTH_BIG_KNOB_DEFAULT; + } + if (!gate && env == 0 && spring_tail == 0) { + vTaskDelay(pdMS_TO_TICKS(10)); + continue; + } + if (sh_trigger != last_sh_trigger) { + last_sh_trigger = sh_trigger; + sh_counter = 0; + } + + if (formant_index >= sizeof(s_formants) / sizeof(s_formants[0])) { + formant_index = 0; + } + const trautonium_formant_t *formant = &s_formants[formant_index]; + if (last_formant != formant_index) { + f1_low = 0; + f1_band = 0; + f2_low = 0; + f2_band = 0; + last_formant = formant_index; + } + + uint32_t glide_chunks = ((uint32_t)CONFIG_CARDPUTER_ADV_TRAUTONIUM_PORTAMENTO_MS * + ADV_SYNTH_RATE_HZ) / (1000U * ADV_SYNTH_FRAMES); + if (glide_chunks == 0) { + glide_chunks = 1; + } + uint32_t diff = current_mhz > target_mhz ? current_mhz - target_mhz : target_mhz - current_mhz; + uint32_t step = diff / glide_chunks; + if (step == 0 && diff != 0) { + step = 1; + } + current_mhz = approach_u32(current_mhz, target_mhz, step); + + uint32_t phase_inc = (uint32_t)(((uint64_t)current_mhz << 32) / + ((uint64_t)ADV_SYNTH_RATE_HZ * 1000ULL)); + + for (uint32_t i = 0; i < ADV_SYNTH_FRAMES; i++) { + if (gate) { + if (sh_counter == 0) { + uint32_t sh_rate = 4U + ((uint32_t)big_knob * 18U) / 100U; + sh_rng = sh_rng * 1664525U + 1013904223U + phase_inc + pressure; + sh_value = (int32_t)((sh_rng >> 24) & 0xFF) - 128; + sh_counter = ADV_SYNTH_RATE_HZ / sh_rate; + } else { + sh_counter--; + } + } + + phase += phase_inc; + sub1_phase += phase_inc >> 1; + sub2_phase += phase_inc >> 2; + + if (gate) { + env += (65535U - env) >> 5; + if (env < 65535U - 96U) { + env += 96U; + } else { + env = 65535U; + } + } else if (env > 0) { + uint32_t decay = (env >> 8) + 48U; + env = env > decay ? env - decay : 0; + } + + int32_t sh_depth = ((int32_t)big_knob * 52) / 100; + int32_t edge_units = (int32_t)formant->trap_edge + + ((int32_t)big_knob / 9) + + ((sh_value * sh_depth) >> 8); + uint8_t trap_edge = (uint8_t)clamp_i32(edge_units, 1, 30); + int32_t trap = trapezoid_wave(phase, trap_edge); + int32_t sub1 = (sub1_phase & 0x80000000U) ? 15000 : -15000; + int32_t sub2 = trapezoid_wave(sub2_phase, + (uint8_t)clamp_i32((int32_t)trap_edge + 3, 1, 31)) / 2; + int32_t raw = (trap * 6 + sub1 * 2 + sub2) / 9; + + int32_t driven = (raw * (int32_t)(96U + pressure)) / 128; + driven = clamp_i32(driven, -32000, 32000); + + int32_t form_shift = (sh_value * (int32_t)(420U + big_knob * 7U)) / 128; + uint16_t f1 = (uint16_t)clamp_i32((int32_t)formant->f1 + form_shift, 700, 12000); + uint16_t f2 = (uint16_t)clamp_i32((int32_t)formant->f2 + form_shift * 2, 1800, 15500); + int32_t bp1 = svf_bandpass(driven, f1, formant->damp1, &f1_low, &f1_band); + int32_t bp2 = svf_bandpass(driven, f2, formant->damp2, &f2_low, &f2_band); + int32_t voiced_formant = (bp1 * formant->gain1 + bp2 * formant->gain2) >> 8; + int32_t mixed = (driven * (int32_t)(256U - formant->formant_mix) + + voiced_formant * formant->formant_mix) >> 8; + + int32_t sample = (int32_t)(((int64_t)mixed * env) >> 16); + sample = (sample * CONFIG_CARDPUTER_ADV_TRAUTONIUM_LEVEL_PERCENT) / 100; + sample = clamp_i32(sample, -30000, 30000); + if (gate || env > 0) { + spring_tail = (ADV_SYNTH_RATE_HZ / 5U) + + (((uint32_t)big_knob * ADV_SYNTH_RATE_HZ) / 180U); + } else if (spring_tail > 0) { + spring_tail--; + } + int32_t left = sample; + int32_t right = sample; + spring_reverb_process(sample, big_knob, &left, &right); + s_synth_buf[i * 2] = (int16_t)clamp_i32(left, -30000, 30000); + s_synth_buf[i * 2 + 1] = (int16_t)clamp_i32(right, -30000, 30000); + } + + esp_err_t err = audio_lock(pdMS_TO_TICKS(100)); + if (err == ESP_OK) { + err = init_codec(); + if (err == ESP_OK) { + err = ensure_i2s(ADV_SYNTH_RATE_HZ); + } + if (err == ESP_OK) { + err = write_i2s_all(s_synth_buf, sizeof(s_synth_buf)); + } + audio_unlock(); + } + if (err != ESP_OK) { + if ((err_count++ & 0x3F) == 0) { + ESP_LOGW(TAG, "Trautonium synth write failed: %s", esp_err_to_name(err)); + } + vTaskDelay(pdMS_TO_TICKS(5)); + } + } +} + +static void start_trautonium_synth_once(void) +{ + if (s_synth_task != NULL) { + return; + } + + s_synth_target_mhz = trautonium_key_freq_mhz(18); + s_synth_pressure_percent = 62; + s_synth_big_knob_percent = ADV_SYNTH_BIG_KNOB_DEFAULT; + + BaseType_t ok = xTaskCreate(trautonium_synth_task, "adv_trap_synth", + 4096, NULL, 5, &s_synth_task); + if (ok != pdPASS) { + s_synth_task = NULL; + ESP_LOGW(TAG, "failed to start Trautonium synth task"); + } else { + ESP_LOGI(TAG, "Trautonium key synth ready: trapezoid + S/H + spring + %ums glide", + (unsigned)CONFIG_CARDPUTER_ADV_TRAUTONIUM_PORTAMENTO_MS); + } +} + +static esp_err_t get_keyboard_dev(void) +{ + if (s_kb_dev != NULL) { + return ESP_OK; + } + + esp_err_t err = get_i2c_bus(); + if (err != ESP_OK) { + return err; + } + + i2c_device_config_t dev_cfg = { + .dev_addr_length = I2C_ADDR_BIT_LEN_7, + .device_address = ADV_KB_ADDR, + .scl_speed_hz = 400000, + }; + return i2c_master_bus_add_device(s_i2c_bus, &dev_cfg, &s_kb_dev); +} + +static esp_err_t kb_write_reg(uint8_t reg, uint8_t value) +{ + uint8_t data[2] = {reg, value}; + return i2c_master_transmit(s_kb_dev, data, sizeof(data), 20); +} + +static esp_err_t kb_read_reg(uint8_t reg, uint8_t *value) +{ + return i2c_master_transmit_receive(s_kb_dev, ®, 1, value, 1, 20); +} + +static esp_err_t init_keyboard_scanner(void) +{ + gpio_config_t int_cfg = { + .pin_bit_mask = 1ULL << ADV_KB_INT, + .mode = GPIO_MODE_INPUT, + .pull_up_en = GPIO_PULLUP_ENABLE, + .pull_down_en = GPIO_PULLDOWN_DISABLE, + .intr_type = GPIO_INTR_DISABLE, + }; + gpio_config(&int_cfg); + + esp_err_t err = get_keyboard_dev(); + if (err != ESP_OK) { + return err; + } + + err = kb_write_reg(ADV_KB_REG_KP_GPIO1, 0xFF); + if (err == ESP_OK) { + err = kb_write_reg(ADV_KB_REG_KP_GPIO2, 0xFF); + } + if (err == ESP_OK) { + err = kb_write_reg(ADV_KB_REG_KP_GPIO3, 0x00); + } + if (err == ESP_OK) { + err = kb_write_reg(ADV_KB_REG_GPI_EM1, 0x00); + } + if (err == ESP_OK) { + err = kb_write_reg(ADV_KB_REG_INT_STAT, 0xFF); + } + if (err == ESP_OK) { + err = kb_write_reg(ADV_KB_REG_CFG, 0x3E); + } + if (err != ESP_OK) { + return err; + } + + uint8_t ec = 0; + err = kb_read_reg(ADV_KB_REG_KEY_LCK_EC, &ec); + if (err == ESP_OK) { + ESP_LOGI(TAG, "ADV keyboard audio scanner ready: TCA8418 addr=0x%02x INT=G%d", + ADV_KB_ADDR, ADV_KB_INT); + } + return err; +} + +static void trautonium_keyscan_task(void *arg) +{ + (void)arg; + esp_err_t err = init_keyboard_scanner(); + if (err != ESP_OK) { + ESP_LOGW(TAG, "ADV keyboard audio scanner failed: %s", esp_err_to_name(err)); + s_keyscan_task = NULL; + vTaskDelete(NULL); + return; + } + + for (;;) { + uint8_t ec = 0; + if (kb_read_reg(ADV_KB_REG_KEY_LCK_EC, &ec) == ESP_OK) { + uint8_t count = ec & 0x0F; + if (count == 0 && gpio_get_level(ADV_KB_INT) == 0) { + count = 10; + } + if (count > 10) { + count = 10; + } + + for (uint8_t i = 0; i < count; i++) { + uint8_t event = 0; + if (kb_read_reg(ADV_KB_REG_KEY_EVENT, &event) != ESP_OK || event == 0) { + break; + } + bool pressed = (event & 0x80) != 0; + uint8_t code = event & 0x7F; + if (code != 0) { + cardputer_adv_audio_key_event(code, pressed); + } + } + kb_write_reg(ADV_KB_REG_INT_STAT, 0xFF); + } + + vTaskDelay(pdMS_TO_TICKS(8)); + } +} + +static void start_keyboard_scan_once(void) +{ + if (s_keyscan_task != NULL) { + return; + } + BaseType_t ok = xTaskCreate(trautonium_keyscan_task, "adv_keyscan", + 3072, NULL, 6, &s_keyscan_task); + if (ok != pdPASS) { + s_keyscan_task = NULL; + ESP_LOGW(TAG, "failed to start ADV keyboard audio scanner"); + } +} +#endif + +static esp_err_t play_boot_chime(void) +{ + static const int16_t wave32[] = { + 0, 6393, 12540, 18204, 23170, 27245, 30273, 32138, + 32767, 32138, 30273, 27245, 23170, 18204, 12540, 6393, + 0, -6393, -12540, -18204, -23170, -27245, -30273, -32138, + -32767, -32138, -30273, -27245, -23170, -18204, -12540, -6393, + }; + + esp_err_t err = audio_lock(pdMS_TO_TICKS(ADV_IO_TIMEOUT_MS)); + if (err != ESP_OK) { + return err; + } + + err = init_codec(); + if (err == ESP_OK) { + err = ensure_i2s(ADV_CHIME_RATE_HZ); + } + + uint32_t frames = (ADV_CHIME_RATE_HZ * ADV_CHIME_MS) / 1000; + uint32_t phase = 0; + while (err == ESP_OK && frames > 0) { + uint32_t chunk_frames = frames; + if (chunk_frames > sizeof(s_pcm_buf) / (sizeof(int16_t) * 2)) { + chunk_frames = sizeof(s_pcm_buf) / (sizeof(int16_t) * 2); + } + + for (uint32_t i = 0; i < chunk_frames; i++) { + uint32_t envelope = (frames < ADV_CHIME_RATE_HZ / 40) ? frames : (ADV_CHIME_RATE_HZ / 40); + int16_t sample = wave32[(phase >> 4) & 31]; + sample = (int16_t)(((int32_t)sample * (int32_t)envelope) / (ADV_CHIME_RATE_HZ / 40)); + sample = scale_sample(sample); + s_pcm_buf[i * 2] = sample; + s_pcm_buf[i * 2 + 1] = sample; + phase += ADV_CHIME_STEP; + } + + err = write_i2s_all(s_pcm_buf, chunk_frames * 2 * sizeof(int16_t)); + frames -= chunk_frames; + } + + if (err == ESP_OK) { + ESP_LOGI(TAG, "ADV speaker boot chime played"); + err = write_silence(ADV_CHIME_RATE_HZ, 80); + } + audio_unlock(); + return err; +} + +static esp_err_t mount_sd_once(void) +{ + if (s_sd_mounted) { + return ESP_OK; + } + if (s_sd_mount_attempted) { + return s_sd_mount_err; + } + s_sd_mount_attempted = true; + + spi_bus_config_t bus_cfg = { + .mosi_io_num = ADV_SD_MOSI, + .miso_io_num = ADV_SD_MISO, + .sclk_io_num = ADV_SD_SCLK, + .quadwp_io_num = -1, + .quadhd_io_num = -1, + .max_transfer_sz = 4096, + }; + + esp_err_t err = spi_bus_initialize(ADV_SD_HOST, &bus_cfg, SDSPI_DEFAULT_DMA); + if (err != ESP_OK && err != ESP_ERR_INVALID_STATE) { + s_sd_mount_err = err; + return err; + } + + esp_vfs_fat_sdmmc_mount_config_t mount_cfg = { + .format_if_mount_failed = false, + .max_files = 2, + .allocation_unit_size = 16 * 1024, + }; + sdmmc_host_t host = SDSPI_HOST_DEFAULT(); + host.slot = ADV_SD_HOST; + host.max_freq_khz = SDMMC_FREQ_DEFAULT; + + sdspi_device_config_t slot_cfg = SDSPI_DEVICE_CONFIG_DEFAULT(); + slot_cfg.host_id = ADV_SD_HOST; + slot_cfg.gpio_cs = ADV_SD_CS; + + err = esp_vfs_fat_sdspi_mount(CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT, + &host, &slot_cfg, &mount_cfg, &s_sd_card); + if (err != ESP_OK) { + s_sd_mount_err = err; + return err; + } + + s_sd_mounted = true; + s_sd_mount_err = ESP_OK; + ESP_LOGI(TAG, "microSD mounted at %s CS=G%d MOSI=G%d MISO=G%d CLK=G%d", + CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT, + ADV_SD_CS, ADV_SD_MOSI, ADV_SD_MISO, ADV_SD_SCLK); + return ESP_OK; +} + +static esp_err_t parse_wav(FILE *fp, wav_info_t *info) +{ + uint8_t hdr[12]; + if (fread(hdr, 1, sizeof(hdr), fp) != sizeof(hdr)) { + return ESP_ERR_INVALID_SIZE; + } + if (memcmp(hdr, "RIFF", 4) != 0 || memcmp(hdr + 8, "WAVE", 4) != 0) { + return ESP_ERR_INVALID_RESPONSE; + } + + bool have_fmt = false; + bool have_data = false; + memset(info, 0, sizeof(*info)); + + while (!have_data) { + uint8_t chunk[8]; + if (fread(chunk, 1, sizeof(chunk), fp) != sizeof(chunk)) { + return ESP_ERR_NOT_FOUND; + } + uint32_t chunk_size = read_le32(chunk + 4); + + if (memcmp(chunk, "fmt ", 4) == 0) { + uint8_t fmt[16]; + if (chunk_size < sizeof(fmt) || fread(fmt, 1, sizeof(fmt), fp) != sizeof(fmt)) { + return ESP_ERR_INVALID_SIZE; + } + info->audio_format = read_le16(fmt); + info->channels = read_le16(fmt + 2); + info->sample_rate = read_le32(fmt + 4); + info->bits_per_sample = read_le16(fmt + 14); + have_fmt = true; + + long extra = (long)chunk_size - (long)sizeof(fmt); + if (extra > 0 && fseek(fp, extra, SEEK_CUR) != 0) { + return ESP_ERR_INVALID_SIZE; + } + } else if (memcmp(chunk, "data", 4) == 0) { + if (!have_fmt) { + return ESP_ERR_INVALID_STATE; + } + info->data_bytes = chunk_size; + have_data = true; + } else { + if (fseek(fp, (long)chunk_size, SEEK_CUR) != 0) { + return ESP_ERR_INVALID_SIZE; + } + } + + if ((chunk_size & 1) && !have_data) { + if (fseek(fp, 1, SEEK_CUR) != 0) { + return ESP_ERR_INVALID_SIZE; + } + } + } + + if (info->audio_format != 1 || + (info->channels != 1 && info->channels != 2) || + (info->bits_per_sample != 8 && info->bits_per_sample != 16) || + info->sample_rate == 0 || info->data_bytes == 0) { + return ESP_ERR_NOT_SUPPORTED; + } + + return ESP_OK; +} + +static size_t convert_pcm_to_stereo16(const uint8_t *input, size_t input_bytes, + const wav_info_t *info) +{ + size_t source_samples = input_bytes / (info->bits_per_sample / 8); + size_t frames = source_samples / info->channels; + size_t max_frames = sizeof(s_pcm_buf) / (sizeof(int16_t) * 2); + if (frames > max_frames) { + frames = max_frames; + } + + for (size_t frame = 0; frame < frames; frame++) { + int16_t left; + int16_t right; + if (info->bits_per_sample == 16) { + const uint8_t *p = input + frame * info->channels * 2; + left = (int16_t)read_le16(p); + if (info->channels == 2) { + right = (int16_t)read_le16(p + 2); + } else { + right = left; + } + } else { + const uint8_t *p = input + frame * info->channels; + left = (int16_t)(((int)p[0] - 128) << 8); + if (info->channels == 2) { + right = (int16_t)(((int)p[1] - 128) << 8); + } else { + right = left; + } + } + s_pcm_buf[frame * 2] = scale_sample(left); + s_pcm_buf[frame * 2 + 1] = scale_sample(right); + } + + return frames * 2 * sizeof(int16_t); +} + +static esp_err_t play_wav_file(const char *path) +{ + FILE *fp = fopen(path, "rb"); + if (fp == NULL) { + return ESP_ERR_NOT_FOUND; + } + + wav_info_t info; + esp_err_t err = parse_wav(fp, &info); + if (err != ESP_OK) { + fclose(fp); + return err; + } + + err = ensure_i2s(info.sample_rate); + if (err != ESP_OK) { + fclose(fp); + return err; + } + + ESP_LOGI(TAG, "playing WAV %s: %lu Hz, %u ch, %u bit, %lu bytes", + path, (unsigned long)info.sample_rate, info.channels, + info.bits_per_sample, (unsigned long)info.data_bytes); + + uint32_t remaining = info.data_bytes; + const uint32_t block_align = (info.bits_per_sample / 8) * info.channels; + while (remaining > 0) { + size_t to_read = remaining < sizeof(s_raw_buf) ? remaining : sizeof(s_raw_buf); + to_read -= to_read % block_align; + if (to_read == 0) { + break; + } + + size_t got = fread(s_raw_buf, 1, to_read, fp); + if (got == 0) { + break; + } + remaining -= got; + + size_t out_bytes = convert_pcm_to_stereo16(s_raw_buf, got, &info); + err = write_i2s_all(s_pcm_buf, out_bytes); + if (err != ESP_OK) { + fclose(fp); + return err; + } + } + + fclose(fp); + return write_silence(info.sample_rate, 80); +} + +esp_err_t cardputer_adv_audio_play_wav_from_sd(const char *path) +{ + if (path == NULL || path[0] == '\0') { + return ESP_ERR_INVALID_ARG; + } + + esp_err_t err = audio_lock(pdMS_TO_TICKS(ADV_IO_TIMEOUT_MS)); + if (err != ESP_OK) { + return err; + } + + err = init_codec(); + if (err == ESP_OK) { + err = mount_sd_once(); + } + if (err == ESP_OK) { + err = play_wav_file(path); + } + + audio_unlock(); + return err; +} + +void cardputer_adv_audio_key_event(uint8_t key_code, bool pressed) +{ +#if defined(CONFIG_CARDPUTER_ADV_TRAUTONIUM_ENABLE) + if (key_code == 0) { + return; + } + start_trautonium_synth_once(); + + if (trautonium_handle_macro_key(key_code, pressed)) { + return; + } + + const trautonium_key_note_t *note = trautonium_find_key_note(key_code); + int8_t semitone = note != NULL ? note->semitone : trautonium_raw_fallback_semitone(key_code); + uint8_t formant_index = note != NULL ? note->formant_index : + (uint8_t)(((key_code - 1) / 10) % + (sizeof(s_formants) / sizeof(s_formants[0]))); + + uint32_t target_mhz = trautonium_note_freq_mhz(semitone); + uint8_t pressure = trautonium_key_pressure(key_code); + uint8_t big_knob = trautonium_big_knob_for_key(key_code, s_synth_big_knob_percent); + bool gate_changed = false; + + portENTER_CRITICAL(&s_synth_lock); + if (pressed) { + s_synth_active_key = key_code; + s_synth_target_mhz = target_mhz; + s_synth_formant_index = formant_index; + s_synth_pressure_percent = pressure; + s_synth_big_knob_percent = big_knob; + s_synth_sh_trigger++; + s_synth_gate = true; + gate_changed = true; + } else if (s_synth_active_key == key_code) { + s_synth_active_key = 0; + s_synth_gate = false; + gate_changed = true; + } + portEXIT_CRITICAL(&s_synth_lock); + + if (gate_changed && pressed) { + const trautonium_formant_t *formant = &s_formants[formant_index]; + ESP_LOGI(TAG, "Trautonium key=%s raw=%u freq=%lu.%03luHz formant=%s pressure=%u knob=%u", + note != NULL ? note->label : "raw", + (unsigned)key_code, + (unsigned long)(target_mhz / 1000U), + (unsigned long)(target_mhz % 1000U), + formant->name, + (unsigned)pressure, + (unsigned)big_knob); + } +#else + (void)key_code; + (void)pressed; +#endif +} + +esp_err_t cardputer_adv_audio_startup_probe(void) +{ + esp_err_t audio_error = ESP_OK; + +#if defined(CONFIG_CARDPUTER_ADV_AUDIO_BOOT_CHIME) + esp_err_t err = play_boot_chime(); + if (err != ESP_OK) { + audio_error = err; + ESP_LOGW(TAG, "ADV speaker chime failed: %s", esp_err_to_name(err)); + } +#endif + +#if defined(CONFIG_CARDPUTER_ADV_TRAUTONIUM_ENABLE) + start_keyboard_scan_once(); +#endif + +#if defined(CONFIG_CARDPUTER_ADV_AUDIO_TRY_SD_WAV) + static const char *fallbacks[] = { + CONFIG_CARDPUTER_ADV_AUDIO_SD_WAV_PATH, + CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT "/ruview.wav", + CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT "/file1.wav", + CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT "/file2.wav", + CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT "/file3.wav", + }; + + for (size_t i = 0; i < sizeof(fallbacks) / sizeof(fallbacks[0]); i++) { + if (fallbacks[i][0] == '\0') { + continue; + } + esp_err_t err = cardputer_adv_audio_play_wav_from_sd(fallbacks[i]); + if (err == ESP_OK) { + ESP_LOGI(TAG, "ADV SD WAV proof played: %s", fallbacks[i]); + return ESP_OK; + } + ESP_LOGI(TAG, "ADV SD WAV skip %s: %s", fallbacks[i], esp_err_to_name(err)); + } +#endif + + return audio_error; +} + +#else + +esp_err_t cardputer_adv_audio_play_wav_from_sd(const char *path) +{ + (void)path; + return ESP_OK; +} + +void cardputer_adv_audio_key_event(uint8_t key_code, bool pressed) +{ + (void)key_code; + (void)pressed; +} + +esp_err_t cardputer_adv_audio_startup_probe(void) +{ + return ESP_OK; +} + +#endif diff --git a/firmware/esp32-csi-node/main/cardputer_adv_audio.h b/firmware/esp32-csi-node/main/cardputer_adv_audio.h new file mode 100644 index 0000000000..e275da620e --- /dev/null +++ b/firmware/esp32-csi-node/main/cardputer_adv_audio.h @@ -0,0 +1,47 @@ +/** + * @file cardputer_adv_audio.h + * @brief Cardputer-Adv ES8311 speaker and SD WAV playback support. + */ + +#ifndef CARDPUTER_ADV_AUDIO_H +#define CARDPUTER_ADV_AUDIO_H + +#include +#include +#include "esp_err.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Run the configured boot-time audio proof on Cardputer-Adv. + * + * On non-ADV targets or when disabled by Kconfig this returns ESP_OK without + * touching GPIO, I2C, I2S, or SD hardware. + */ +esp_err_t cardputer_adv_audio_startup_probe(void); + +/** + * Play a PCM WAV file from the Cardputer-Adv microSD slot. + * + * The path must include the ESP-IDF mount point, for example + * "/sdcard/ruview.wav". Supported input is unsigned 8-bit or signed 16-bit + * PCM, mono or stereo. + */ +esp_err_t cardputer_adv_audio_play_wav_from_sd(const char *path); + +/** + * Feed a Cardputer-Adv keyboard event to the embedded Trautonium voice. + * + * The key code is the raw TCA8418 event code without the press bit. When the + * synth is disabled this is a no-op, so display code can call it unconditionally + * behind the audio feature flag. + */ +void cardputer_adv_audio_key_event(uint8_t key_code, bool pressed); + +#ifdef __cplusplus +} +#endif + +#endif /* CARDPUTER_ADV_AUDIO_H */ diff --git a/firmware/esp32-csi-node/main/display_hal.c b/firmware/esp32-csi-node/main/display_hal.c index dbbf63e81b..b0601a7b6b 100644 --- a/firmware/esp32-csi-node/main/display_hal.c +++ b/firmware/esp32-csi-node/main/display_hal.c @@ -1,16 +1,11 @@ /** * @file display_hal.c - * @brief ADR-045: SH8601 QSPI AMOLED HAL for Waveshare ESP32-S3-Touch-AMOLED-1.8. + * @brief Target-specific ST7789 LCD HAL. * - * Uses ESP-IDF esp_lcd_panel_io_spi in QSPI mode (quad_mode=true, lcd_cmd_bits=32). - * The panel_io layer handles the 0x02/0x32 QSPI command encoding. - * - * Hardware: SH8601 368x448, FT3168 touch, TCA9554 I/O expander for power/reset. - * - * Pin assignments (Waveshare ESP32-S3-Touch-AMOLED-1.8): - * QSPI: CS=12, CLK=11, D0=4, D1=5, D2=6, D3=7 - * I2C: SDA=15, SCL=14 (shared: touch FT3168 + TCA9554 expander) - * Touch INT=21 + * Cardputer-Adv (ESP32-S3): 240x135, BL=G38, RST=G33, DC=G34, MOSI=G35, + * SCK=G36, CS=G37. + * StickC Plus2 (ESP32): 135x240, BL=G27, RST=G12, DC=G14, MOSI=G15, + * SCK=G13, CS=G5. */ #include "display_hal.h" @@ -21,362 +16,202 @@ #include #include "freertos/FreeRTOS.h" #include "freertos/task.h" -#include "esp_log.h" -#include "esp_lcd_panel_io.h" -#include "driver/spi_master.h" #include "driver/gpio.h" -#include "driver/i2c.h" +#include "driver/spi_master.h" #include "esp_heap_caps.h" +#include "esp_lcd_panel_io.h" +#include "esp_lcd_panel_ops.h" +#include "esp_lcd_panel_vendor.h" +#include "esp_log.h" static const char *TAG = "disp_hal"; -/* ---- QSPI Pin Definitions (Waveshare board) ---- */ -#define DISP_QSPI_CS 12 -#define DISP_QSPI_CLK 11 -#define DISP_QSPI_D0 4 -#define DISP_QSPI_D1 5 -#define DISP_QSPI_D2 6 -#define DISP_QSPI_D3 7 - -/* ---- I2C (shared: touch + TCA9554 expander) ---- */ -#define I2C_SDA 15 -#define I2C_SCL 14 -#define TOUCH_INT_PIN 21 -#define I2C_MASTER_NUM I2C_NUM_0 -#define I2C_MASTER_FREQ_HZ 400000 - -/* ---- TCA9554 I/O expander ---- */ -#define TCA9554_ADDR 0x20 -#define TCA9554_REG_OUTPUT 0x01 -#define TCA9554_REG_CONFIG 0x03 - -/* ---- FT3168 touch controller ---- */ -#define FT3168_ADDR 0x38 - -/* ---- Display dimensions ---- */ -#define DISP_H_RES 368 -#define DISP_V_RES 448 +#define LCD_SPI_HOST SPI2_HOST +#define LCD_PIXEL_CLOCK_HZ (20 * 1000 * 1000) +#define LCD_CMD_BITS 8 +#define LCD_PARAM_BITS 8 -/* ---- QSPI opcodes (packed into lcd_cmd bits [31:24]) ---- */ -#define LCD_OPCODE_WRITE_CMD 0x02 -#define LCD_OPCODE_WRITE_COLOR 0x32 - -/* ---- State ---- */ static esp_lcd_panel_io_handle_t s_io_handle = NULL; -static bool s_i2c_initialized = false; -static bool s_touch_initialized = false; - -/* ---- I2C helpers ---- */ - -static esp_err_t i2c_write_reg(uint8_t dev_addr, uint8_t reg, const uint8_t *data, size_t len) -{ - i2c_cmd_handle_t cmd = i2c_cmd_link_create(); - i2c_master_start(cmd); - i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_WRITE, true); - i2c_master_write_byte(cmd, reg, true); - if (data && len > 0) { - i2c_master_write(cmd, data, len, true); - } - i2c_master_stop(cmd); - esp_err_t ret = i2c_master_cmd_begin(I2C_MASTER_NUM, cmd, pdMS_TO_TICKS(100)); - i2c_cmd_link_delete(cmd); - return ret; -} +static esp_lcd_panel_handle_t s_panel_handle = NULL; +static uint16_t *s_framebuffer = NULL; +static bool s_frame_dirty = false; -static esp_err_t i2c_read_reg(uint8_t dev_addr, uint8_t reg, uint8_t *data, size_t len) +static uint16_t bw565(uint16_t rgb565) { - i2c_cmd_handle_t cmd = i2c_cmd_link_create(); - i2c_master_start(cmd); - i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_WRITE, true); - i2c_master_write_byte(cmd, reg, true); - i2c_master_start(cmd); - i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_READ, true); - i2c_master_read(cmd, data, len, I2C_MASTER_LAST_NACK); - i2c_master_stop(cmd); - esp_err_t ret = i2c_master_cmd_begin(I2C_MASTER_NUM, cmd, pdMS_TO_TICKS(100)); - i2c_cmd_link_delete(cmd); - return ret; + uint32_t r = (rgb565 >> 11) & 0x1F; + uint32_t g = (rgb565 >> 5) & 0x3F; + uint32_t b = rgb565 & 0x1F; + uint32_t lum = r * 54 + g * 183 + b * 19; + return (lum >= 4096) ? 0xFFFF : 0x0000; } -static esp_err_t init_i2c_bus(void) +static void set_backlight(bool on) { - if (s_i2c_initialized) return ESP_OK; - - i2c_config_t i2c_cfg = { - .mode = I2C_MODE_MASTER, - .sda_io_num = I2C_SDA, - .scl_io_num = I2C_SCL, - .sda_pullup_en = GPIO_PULLUP_ENABLE, - .scl_pullup_en = GPIO_PULLUP_ENABLE, - .master.clk_speed = I2C_MASTER_FREQ_HZ, + gpio_config_t bl_cfg = { + .pin_bit_mask = 1ULL << DISPLAY_PANEL_BL_PIN, + .mode = GPIO_MODE_OUTPUT, + .pull_up_en = GPIO_PULLUP_DISABLE, + .pull_down_en = GPIO_PULLDOWN_DISABLE, + .intr_type = GPIO_INTR_DISABLE, }; - - esp_err_t ret = i2c_param_config(I2C_MASTER_NUM, &i2c_cfg); - if (ret != ESP_OK) return ret; - - ret = i2c_driver_install(I2C_MASTER_NUM, I2C_MODE_MASTER, 0, 0, 0); - if (ret != ESP_OK) return ret; - - s_i2c_initialized = true; - ESP_LOGI(TAG, "I2C bus init OK (SDA=%d, SCL=%d)", I2C_SDA, I2C_SCL); - return ESP_OK; + gpio_config(&bl_cfg); + gpio_set_level(DISPLAY_PANEL_BL_PIN, on ? 1 : 0); } -/* ---- TCA9554 I/O expander: toggle pins for display power/reset ---- */ - -static esp_err_t tca9554_init_display_power(void) +static esp_err_t display_panel_init_common(void) { - /* Set pins 0, 1, 2 as outputs */ - uint8_t cfg = 0xF8; - esp_err_t ret = i2c_write_reg(TCA9554_ADDR, TCA9554_REG_CONFIG, &cfg, 1); - if (ret != ESP_OK) { - ESP_LOGW(TAG, "TCA9554 not found at 0x%02X: %s", TCA9554_ADDR, esp_err_to_name(ret)); - return ret; - } - - /* Set pins 0,1,2 LOW (reset state) */ - uint8_t out = 0x00; - i2c_write_reg(TCA9554_ADDR, TCA9554_REG_OUTPUT, &out, 1); - vTaskDelay(pdMS_TO_TICKS(200)); - - /* Set pins 0,1,2 HIGH (power on + release reset) */ - out = 0x07; - i2c_write_reg(TCA9554_ADDR, TCA9554_REG_OUTPUT, &out, 1); - vTaskDelay(pdMS_TO_TICKS(200)); - - ESP_LOGI(TAG, "TCA9554 display power/reset toggled"); - return ESP_OK; -} - -/* ---- Panel IO helpers: send commands via esp_lcd QSPI panel IO ---- */ - -static esp_err_t panel_write_cmd(uint8_t dcs_cmd, const void *data, size_t data_len) -{ - /* Pack as 32-bit lcd_cmd: [31:24]=opcode, [23:8]=dcs_cmd, [7:0]=0 */ - uint32_t lcd_cmd = ((uint32_t)LCD_OPCODE_WRITE_CMD << 24) | ((uint32_t)dcs_cmd << 8); - return esp_lcd_panel_io_tx_param(s_io_handle, (int)lcd_cmd, data, data_len); -} + ESP_LOGI(TAG, "Initializing %s ST7789 LCD (%dx%d)...", + DISPLAY_PANEL_NAME, DISPLAY_PANEL_H_RES, DISPLAY_PANEL_V_RES); -static esp_err_t panel_write_color(const void *color_data, size_t data_len) -{ - /* RAMWR (0x2C) packed as 32-bit lcd_cmd with quad opcode */ - uint32_t lcd_cmd = ((uint32_t)LCD_OPCODE_WRITE_COLOR << 24) | (0x2C << 8); - return esp_lcd_panel_io_tx_color(s_io_handle, (int)lcd_cmd, color_data, data_len); -} - -/* ---- SH8601 init sequence (from Waveshare reference) ---- */ - -typedef struct { - uint8_t cmd; - uint8_t data[4]; - uint8_t data_len; - uint16_t delay_ms; -} sh8601_init_cmd_t; - -static const sh8601_init_cmd_t sh8601_init_cmds[] = { - {0x11, {0x00}, 0, 120}, /* Sleep Out + 120ms */ - {0x44, {0x01, 0xD1}, 2, 0}, /* Partial area */ - {0x35, {0x00}, 1, 0}, /* Tearing Effect ON */ - {0x53, {0x20}, 1, 10}, /* Write CTRL Display */ - {0x2A, {0x00, 0x00, 0x01, 0x6F}, 4, 0}, /* CASET: 0-367 */ - {0x2B, {0x00, 0x00, 0x01, 0xBF}, 4, 0}, /* RASET: 0-447 */ - {0x51, {0x00}, 1, 10}, /* Brightness: 0 */ - {0x29, {0x00}, 0, 10}, /* Display ON */ - {0x51, {0xFF}, 1, 0}, /* Brightness: max */ - {0x00, {0x00}, 0xFF, 0}, /* End sentinel */ -}; - -static esp_err_t send_init_sequence(void) -{ - for (int i = 0; sh8601_init_cmds[i].data_len != 0xFF; i++) { - const sh8601_init_cmd_t *cmd = &sh8601_init_cmds[i]; - esp_err_t ret = panel_write_cmd( - cmd->cmd, - cmd->data_len > 0 ? cmd->data : NULL, - cmd->data_len); - if (ret != ESP_OK) { - ESP_LOGE(TAG, "CMD 0x%02X failed: %s", cmd->cmd, esp_err_to_name(ret)); - return ret; - } - if (cmd->delay_ms > 0) { - vTaskDelay(pdMS_TO_TICKS(cmd->delay_ms)); - } - } - return ESP_OK; -} + set_backlight(false); -/* ---- Public API ---- */ - -esp_err_t display_hal_init_panel(void) -{ - ESP_LOGI(TAG, "Initializing Waveshare AMOLED 1.8\" (SH8601 368x448)..."); - - /* Step 1: Init I2C bus */ - esp_err_t ret = init_i2c_bus(); - if (ret != ESP_OK) { - ESP_LOGW(TAG, "I2C bus init failed"); - return ESP_ERR_NOT_FOUND; - } - - /* Step 2: TCA9554 display power/reset (optional — only present on Waveshare board) */ - ret = tca9554_init_display_power(); - if (ret != ESP_OK) { - ESP_LOGW(TAG, "TCA9554 not found — assuming display power is always-on (direct wiring)"); - /* Continue without TCA9554 — the display may be powered directly */ - } - - /* Step 3: Initialize SPI bus */ spi_bus_config_t bus_cfg = { - .sclk_io_num = DISP_QSPI_CLK, - .data0_io_num = DISP_QSPI_D0, - .data1_io_num = DISP_QSPI_D1, - .data2_io_num = DISP_QSPI_D2, - .data3_io_num = DISP_QSPI_D3, - .max_transfer_sz = DISP_H_RES * DISP_V_RES * 2, + .sclk_io_num = DISPLAY_PANEL_SCLK_PIN, + .mosi_io_num = DISPLAY_PANEL_MOSI_PIN, + .miso_io_num = -1, + .quadwp_io_num = -1, + .quadhd_io_num = -1, + .max_transfer_sz = DISPLAY_PANEL_H_RES * 40 * sizeof(uint16_t), }; - ret = spi_bus_initialize(SPI2_HOST, &bus_cfg, SPI_DMA_CH_AUTO); - if (ret != ESP_OK) { - ESP_LOGW(TAG, "SPI bus init failed: %s", esp_err_to_name(ret)); + esp_err_t ret = spi_bus_initialize(LCD_SPI_HOST, &bus_cfg, SPI_DMA_CH_AUTO); + if (ret != ESP_OK && ret != ESP_ERR_INVALID_STATE) { + ESP_LOGE(TAG, "SPI bus init failed: %s", esp_err_to_name(ret)); return ESP_ERR_NOT_FOUND; } - /* Step 4: Create panel IO with QSPI mode */ - esp_lcd_panel_io_spi_config_t io_config = { - .dc_gpio_num = -1, /* No DC pin in QSPI mode */ - .cs_gpio_num = DISP_QSPI_CS, - .pclk_hz = 40 * 1000 * 1000, - .lcd_cmd_bits = 32, /* 32-bit command: [opcode|dcs_cmd|0x00] */ - .lcd_param_bits = 8, - .spi_mode = 0, + esp_lcd_panel_io_spi_config_t io_cfg = { + .dc_gpio_num = DISPLAY_PANEL_DC_PIN, + .cs_gpio_num = DISPLAY_PANEL_CS_PIN, + .pclk_hz = LCD_PIXEL_CLOCK_HZ, + .lcd_cmd_bits = LCD_CMD_BITS, + .lcd_param_bits = LCD_PARAM_BITS, + .spi_mode = 0, .trans_queue_depth = 10, - .flags = { - .quad_mode = true, - }, }; - ret = esp_lcd_new_panel_io_spi((esp_lcd_spi_bus_handle_t)SPI2_HOST, &io_config, &s_io_handle); + ret = esp_lcd_new_panel_io_spi((esp_lcd_spi_bus_handle_t)LCD_SPI_HOST, &io_cfg, &s_io_handle); if (ret != ESP_OK) { ESP_LOGE(TAG, "Panel IO init failed: %s", esp_err_to_name(ret)); - spi_bus_free(SPI2_HOST); return ESP_ERR_NOT_FOUND; } - ESP_LOGI(TAG, "QSPI panel IO created (40MHz, quad mode)"); - /* Step 5: Send SH8601 init sequence */ - ret = send_init_sequence(); + esp_lcd_panel_dev_config_t panel_cfg = { + .reset_gpio_num = DISPLAY_PANEL_RST_PIN, + .rgb_ele_order = LCD_RGB_ELEMENT_ORDER_BGR, + .bits_per_pixel = 16, + }; + + ret = esp_lcd_new_panel_st7789(s_io_handle, &panel_cfg, &s_panel_handle); if (ret != ESP_OK) { - ESP_LOGW(TAG, "SH8601 init sequence failed"); + ESP_LOGE(TAG, "ST7789 panel create failed: %s", esp_err_to_name(ret)); esp_lcd_panel_io_del(s_io_handle); - spi_bus_free(SPI2_HOST); s_io_handle = NULL; return ESP_ERR_NOT_FOUND; } - /* Step 6: Draw test pattern — cyan bar at top */ - ESP_LOGI(TAG, "Drawing test pattern..."); - uint16_t *line_buf = heap_caps_malloc(DISP_H_RES * 2, MALLOC_CAP_DMA); - if (line_buf) { - uint8_t caset[4] = {0, 0, (DISP_H_RES - 1) >> 8, (DISP_H_RES - 1) & 0xFF}; - uint8_t raset[4] = {0, 0, (DISP_V_RES - 1) >> 8, (DISP_V_RES - 1) & 0xFF}; - panel_write_cmd(0x2A, caset, 4); - panel_write_cmd(0x2B, raset, 4); - - for (int y = 0; y < DISP_V_RES; y++) { - uint16_t color = (y < 30) ? 0x07FF : 0x0841; - for (int x = 0; x < DISP_H_RES; x++) { - line_buf[x] = color; - } - panel_write_color(line_buf, DISP_H_RES * 2); - } - free(line_buf); - ESP_LOGI(TAG, "Test pattern drawn"); + ESP_ERROR_CHECK(esp_lcd_panel_reset(s_panel_handle)); + ESP_ERROR_CHECK(esp_lcd_panel_init(s_panel_handle)); +#if defined(CONFIG_IDF_TARGET_ESP32S3) + ESP_ERROR_CHECK(esp_lcd_panel_swap_xy(s_panel_handle, true)); + ESP_ERROR_CHECK(esp_lcd_panel_mirror(s_panel_handle, true, false)); +#else + ESP_ERROR_CHECK(esp_lcd_panel_swap_xy(s_panel_handle, false)); + ESP_ERROR_CHECK(esp_lcd_panel_mirror(s_panel_handle, false, false)); +#endif + ESP_ERROR_CHECK(esp_lcd_panel_set_gap(s_panel_handle, DISPLAY_PANEL_GAP_X, DISPLAY_PANEL_GAP_Y)); + ESP_ERROR_CHECK(esp_lcd_panel_invert_color(s_panel_handle, true)); + ESP_ERROR_CHECK(esp_lcd_panel_disp_on_off(s_panel_handle, true)); + + s_framebuffer = heap_caps_malloc(DISPLAY_PANEL_H_RES * DISPLAY_PANEL_V_RES * sizeof(uint16_t), + MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL); + if (!s_framebuffer) { + ESP_LOGE(TAG, "Framebuffer allocation failed"); + return ESP_ERR_NO_MEM; } - - ESP_LOGI(TAG, "SH8601 panel init OK (%dx%d)", DISP_H_RES, DISP_V_RES); + memset(s_framebuffer, 0, DISPLAY_PANEL_H_RES * DISPLAY_PANEL_V_RES * sizeof(uint16_t)); + s_frame_dirty = true; + + set_backlight(true); + display_hal_present(); + ESP_LOGI(TAG, "%s ST7789 panel init OK: pclk=%dHz gap=(%d,%d) framebuffer=%u bytes bw=1", + DISPLAY_PANEL_NAME, LCD_PIXEL_CLOCK_HZ, DISPLAY_PANEL_GAP_X, DISPLAY_PANEL_GAP_Y, + (unsigned)(DISPLAY_PANEL_H_RES * DISPLAY_PANEL_V_RES * sizeof(uint16_t))); return ESP_OK; } -void display_hal_draw(int x_start, int y_start, int x_end, int y_end, - const void *color_data) +#if BOARD_CARDPUTER_ADV +static esp_err_t cardputer_adv_display_init(void) { - if (!s_io_handle) return; - - /* SH8601 requires coordinates divisible by 2 */ - x_start &= ~1; - y_start &= ~1; - if (x_end & 1) x_end++; - if (y_end & 1) y_end++; - if (x_end > DISP_H_RES) x_end = DISP_H_RES; - if (y_end > DISP_V_RES) y_end = DISP_V_RES; - - uint8_t caset[4] = { - (x_start >> 8) & 0xFF, x_start & 0xFF, - ((x_end - 1) >> 8) & 0xFF, (x_end - 1) & 0xFF, - }; - panel_write_cmd(0x2A, caset, 4); - - uint8_t raset[4] = { - (y_start >> 8) & 0xFF, y_start & 0xFF, - ((y_end - 1) >> 8) & 0xFF, (y_end - 1) & 0xFF, - }; - panel_write_cmd(0x2B, raset, 4); + return display_panel_init_common(); +} +#endif - size_t len = (x_end - x_start) * (y_end - y_start) * 2; - panel_write_color(color_data, len); +#if BOARD_M5STICKC_PLUS +static esp_err_t stickc_plus_display_init(void) +{ + return display_panel_init_common(); } +#endif -esp_err_t display_hal_init_touch(void) +esp_err_t display_hal_init_panel(void) { - ESP_LOGI(TAG, "Probing FT3168 touch controller..."); +#if BOARD_CARDPUTER_ADV + return cardputer_adv_display_init(); +#elif BOARD_M5STICKC_PLUS + return stickc_plus_display_init(); +#else +#error "DISPLAY_ENABLE is only supported on ESP32-S3 Cardputer-Adv and ESP32 StickC Plus2 targets" +#endif +} - if (!s_i2c_initialized) { - esp_err_t ret = init_i2c_bus(); - if (ret != ESP_OK) return ESP_ERR_NOT_FOUND; +void display_hal_draw(int x_start, int y_start, int x_end, int y_end, + const void *color_data) +{ + if (!s_panel_handle || !s_framebuffer || !color_data) return; + if (x_start < 0) x_start = 0; + if (y_start < 0) y_start = 0; + if (x_end > DISPLAY_PANEL_H_RES) x_end = DISPLAY_PANEL_H_RES; + if (y_end > DISPLAY_PANEL_V_RES) y_end = DISPLAY_PANEL_V_RES; + if (x_start >= x_end || y_start >= y_end) return; + + const uint16_t *src = (const uint16_t *)color_data; + const int w = x_end - x_start; + const int h = y_end - y_start; + for (int y = 0; y < h; y++) { + uint16_t *dst = &s_framebuffer[(y_start + y) * DISPLAY_PANEL_H_RES + x_start]; + const uint16_t *row = &src[y * w]; + for (int x = 0; x < w; x++) { + dst[x] = bw565(row[x]); + } } + s_frame_dirty = true; +} - gpio_config_t int_cfg = { - .pin_bit_mask = (1ULL << TOUCH_INT_PIN), - .mode = GPIO_MODE_INPUT, - .pull_up_en = GPIO_PULLUP_ENABLE, - .intr_type = GPIO_INTR_DISABLE, - }; - gpio_config(&int_cfg); - - uint8_t chip_id = 0; - esp_err_t ret = i2c_read_reg(FT3168_ADDR, 0xA8, &chip_id, 1); - if (ret != ESP_OK || chip_id == 0x00 || chip_id == 0xFF) { - ESP_LOGW(TAG, "FT3168 not found (ret=%s, id=0x%02X)", esp_err_to_name(ret), chip_id); - return ESP_ERR_NOT_FOUND; - } +void display_hal_present(void) +{ + if (!s_panel_handle || !s_framebuffer || !s_frame_dirty) return; + esp_lcd_panel_draw_bitmap(s_panel_handle, 0, 0, + DISPLAY_PANEL_H_RES, DISPLAY_PANEL_V_RES, + s_framebuffer); + s_frame_dirty = false; +} - s_touch_initialized = true; - ESP_LOGI(TAG, "FT3168 touch init OK (chip_id=0x%02X)", chip_id); - return ESP_OK; +esp_err_t display_hal_init_touch(void) +{ + return ESP_ERR_NOT_FOUND; } bool display_hal_touch_read(uint16_t *x, uint16_t *y) { - if (!s_touch_initialized) return false; - - uint8_t buf[7] = {0}; - esp_err_t ret = i2c_read_reg(FT3168_ADDR, 0x01, buf, 7); - if (ret != ESP_OK) return false; - - uint8_t num_points = buf[1]; - if (num_points == 0 || num_points > 2) return false; - - *x = ((buf[2] & 0x0F) << 8) | buf[3]; - *y = ((buf[4] & 0x0F) << 8) | buf[5]; - return true; + (void)x; + (void)y; + return false; } void display_hal_set_brightness(uint8_t percent) { - if (!s_io_handle) return; - if (percent > 100) percent = 100; - uint8_t val = (uint8_t)((uint32_t)percent * 255 / 100); - panel_write_cmd(0x51, &val, 1); + set_backlight(percent > 0); } #endif /* CONFIG_DISPLAY_ENABLE */ diff --git a/firmware/esp32-csi-node/main/display_hal.h b/firmware/esp32-csi-node/main/display_hal.h index de48f50ed4..7bac49d18b 100644 --- a/firmware/esp32-csi-node/main/display_hal.h +++ b/firmware/esp32-csi-node/main/display_hal.h @@ -1,8 +1,8 @@ /** * @file display_hal.h - * @brief ADR-045: RM67162 QSPI AMOLED + CST816S touch HAL. + * @brief Board-specific ST7789 LCD HAL for Cardputer-Adv and StickC Plus2. * - * Hardware abstraction for the LilyGO T-Display-S3 AMOLED panel. + * Hardware abstraction for the active board display panel. * Probes hardware at boot; returns ESP_ERR_NOT_FOUND if absent. */ @@ -13,14 +13,44 @@ #include #include "esp_err.h" +#if defined(CONFIG_IDF_TARGET_ESP32S3) +#define BOARD_CARDPUTER_ADV 1 +#define DISPLAY_PANEL_NAME "Cardputer-Adv" +#define DISPLAY_PANEL_H_RES 240 +#define DISPLAY_PANEL_V_RES 135 +#define DISPLAY_PANEL_GAP_X 40 +#define DISPLAY_PANEL_GAP_Y 53 +#define DISPLAY_PANEL_BL_PIN 38 +#define DISPLAY_PANEL_RST_PIN 33 +#define DISPLAY_PANEL_DC_PIN 34 +#define DISPLAY_PANEL_MOSI_PIN 35 +#define DISPLAY_PANEL_SCLK_PIN 36 +#define DISPLAY_PANEL_CS_PIN 37 +#elif defined(CONFIG_IDF_TARGET_ESP32) +#define BOARD_M5STICKC_PLUS 1 +#define DISPLAY_PANEL_NAME "StickC Plus2" +#define DISPLAY_PANEL_H_RES 135 +#define DISPLAY_PANEL_V_RES 240 +#define DISPLAY_PANEL_GAP_X 52 +#define DISPLAY_PANEL_GAP_Y 40 +#define DISPLAY_PANEL_BL_PIN 27 +#define DISPLAY_PANEL_RST_PIN 12 +#define DISPLAY_PANEL_DC_PIN 14 +#define DISPLAY_PANEL_MOSI_PIN 15 +#define DISPLAY_PANEL_SCLK_PIN 13 +#define DISPLAY_PANEL_CS_PIN 5 +#else +#error "DISPLAY_ENABLE is only supported on ESP32-S3 Cardputer-Adv and ESP32 StickC Plus2 targets" +#endif + #ifdef __cplusplus extern "C" { #endif /** - * Probe and initialize the RM67162 QSPI AMOLED panel. + * Probe and initialize the active target's ST7789 LCD panel. * - * Configures QSPI bus, sends panel init sequence, and fills + * Configures SPI bus, sends panel init sequence, and fills * the screen with dark background to confirm it works. * Returns ESP_ERR_NOT_FOUND if the panel does not respond. * @@ -29,8 +59,7 @@ extern "C" { esp_err_t display_hal_init_panel(void); /** - * Draw a rectangle of pixels to the AMOLED. - * Sends CASET + RASET + RAMWR directly via QSPI. + * Copy a rectangle of pixels into the RAM framebuffer. * * @param x_start Left column (inclusive). * @param y_start Top row (inclusive). @@ -42,7 +71,15 @@ void display_hal_draw(int x_start, int y_start, int x_end, int y_end, const void *color_data); /** - * Probe and initialize the CST816S capacitive touch controller. + * Push the complete RAM framebuffer to the LCD. + * + * The active target screen is small enough for full-frame RGB565 updates. + * Presenting whole frames avoids partial-window artifacts on the ST7789. + */ +void display_hal_present(void); + +/** + * Probe and initialize touch controller when present. * * @return ESP_OK on success, ESP_ERR_NOT_FOUND if no touch IC detected. */ @@ -58,7 +95,7 @@ esp_err_t display_hal_init_touch(void); bool display_hal_touch_read(uint16_t *x, uint16_t *y); /** - * Set AMOLED brightness via MIPI DCS command. + * Set LCD backlight state. * * @param percent Brightness 0-100. */ diff --git a/firmware/esp32-csi-node/main/display_task.c b/firmware/esp32-csi-node/main/display_task.c index 9d834edc5a..7e705ba2e5 100644 --- a/firmware/esp32-csi-node/main/display_task.c +++ b/firmware/esp32-csi-node/main/display_task.c @@ -1,9 +1,8 @@ /** * @file display_task.c - * @brief ADR-045: FreeRTOS display task — LVGL pump on Core 0, priority 1. + * @brief ADR-045: FreeRTOS display heartbeat task - live graph loop. * - * Gracefully skips if RM67162 panel or SPIRAM is absent. - * Reads from edge_get_vitals() / edge_get_multi_person() (thread-safe). + * Gracefully skips if the active target's LCD hardware is absent. */ #include "display_task.h" @@ -11,68 +10,163 @@ #if CONFIG_DISPLAY_ENABLE +#include +#include #include #include "freertos/FreeRTOS.h" #include "freertos/task.h" #include "esp_log.h" -#include "esp_heap_caps.h" -#include "lvgl.h" +#include "battery_monitor.h" +#include "csi_collector.h" +#include "edge_processing.h" #include "display_hal.h" -#include "display_ui.h" - -#define DISP_H_RES 368 -#define DISP_V_RES 448 static const char *TAG = "disp_task"; /* ---- Config ---- */ -#ifdef CONFIG_DISPLAY_FPS_LIMIT -#define DISP_FPS_LIMIT CONFIG_DISPLAY_FPS_LIMIT -#else -#define DISP_FPS_LIMIT 30 -#endif +#define DISP_HEARTBEAT_PERIOD_MS 250 + +#define DISP_TASK_STACK (12 * 1024) +#define DISP_TASK_PRIORITY 6 +#define DISP_TASK_CORE 1 -#define DISP_TASK_STACK (8 * 1024) -#define DISP_TASK_PRIORITY 1 -#define DISP_TASK_CORE 0 +/* A live graph keeps the display useful without falling back to the full UI. */ +#define DISP_GRAPH_TOP_BAND_H 8 +#define DISP_GRAPH_BOTTOM_BAND_H 8 +#define DISP_GRAPH_MIN_VISIBLE 2 -#define DISP_BUF_LINES 40 +static uint16_t s_row[DISPLAY_PANEL_H_RES]; +static uint8_t s_motion_history[DISPLAY_PANEL_H_RES]; +static uint16_t s_history_head; -/* ---- LVGL flush callback — calls display_hal_draw directly ---- */ -static void lvgl_flush_cb(lv_disp_drv_t *drv, const lv_area_t *area, lv_color_t *color_p) +static int clamp_int(int value, int lo, int hi) { - display_hal_draw(area->x1, area->y1, area->x2 + 1, area->y2 + 1, color_p); - lv_disp_flush_ready(drv); + if (value < lo) return lo; + if (value > hi) return hi; + return value; } -/* ---- LVGL touch input callback ---- */ -static void lvgl_touch_cb(lv_indev_drv_t *drv, lv_indev_data_t *data) +static int motion_sample_from_vitals(const edge_vitals_pkt_t *vitals, bool has_vitals) { - uint16_t x, y; - if (display_hal_touch_read(&x, &y)) { - data->point.x = x; - data->point.y = y; - data->state = LV_INDEV_STATE_PRESSED; - } else { - data->state = LV_INDEV_STATE_RELEASED; + if (!has_vitals) { + return 0; + } + + int motion = (int)(vitals->motion_energy * 18.0f); + if (motion < 0) motion = 0; + if (motion > 100) motion = 100; + + /* Presence gives a little more lift when motion is weak. */ + if (motion < DISP_GRAPH_MIN_VISIBLE && vitals->presence_score > 0.0f) { + motion = (int)(vitals->presence_score * 6.0f); + } + + return clamp_int(motion, 0, 100); +} + +static int rssi_fill_from_vitals(const edge_vitals_pkt_t *vitals, bool has_vitals) +{ + if (!has_vitals) { + return 0; + } + + /* Map -100..-40 dBm to 0..100% so the bar stays readable. */ + int fill = ((int)vitals->rssi + 100) * 100 / 60; + return clamp_int(fill, 0, 100); +} + +static void render_graph_frame(int graph_sample, int battery_fill, int rssi_fill) +{ + const int width = DISPLAY_PANEL_H_RES; + const int height = DISPLAY_PANEL_V_RES; + const int plot_top = DISP_GRAPH_TOP_BAND_H; + const int plot_bottom = height - DISP_GRAPH_BOTTOM_BAND_H - 1; + const int plot_height = plot_bottom - plot_top + 1; + s_motion_history[s_history_head] = (uint8_t)graph_sample; + s_history_head = (uint16_t)((s_history_head + 1U) % DISPLAY_PANEL_H_RES); + const uint16_t history_base = s_history_head; + + for (int y = 0; y < height; y++) { + memset(s_row, 0, sizeof(s_row)); + + if (y < DISP_GRAPH_TOP_BAND_H) { + int fill = (battery_fill * width) / 100; + fill = clamp_int(fill, 0, width); + for (int x = 0; x < fill; x++) { + s_row[x] = 0xFFFF; + } + } else if (y >= height - DISP_GRAPH_BOTTOM_BAND_H) { + int fill = (rssi_fill * width) / 100; + fill = clamp_int(fill, 0, width); + for (int x = 0; x < fill; x++) { + s_row[x] = 0xFFFF; + } + } else { + const int threshold = plot_bottom - y + 1; + for (int x = 0; x < width; x++) { + int idx = (history_base + x) % width; + int sample_height = (s_motion_history[idx] * plot_height) / 100; + if (sample_height >= threshold) { + s_row[x] = 0xFFFF; + } + } + } + + display_hal_draw(0, y, width, y + 1, s_row); } } /* ---- Display task ---- */ static void display_task(void *arg) { - const TickType_t frame_period = pdMS_TO_TICKS(1000 / DISP_FPS_LIMIT); + (void)arg; - ESP_LOGI(TAG, "Display task running on Core %d, %d fps limit", - xPortGetCoreID(), DISP_FPS_LIMIT); + const TickType_t frame_period = pdMS_TO_TICKS(DISP_HEARTBEAT_PERIOD_MS); + TickType_t last_wake = xTaskGetTickCount(); + uint32_t frame = 0; - display_ui_create(lv_scr_act()); + ESP_LOGI(TAG, "Display graph task running on Core %d, %u ms period", + xPortGetCoreID(), (unsigned)DISP_HEARTBEAT_PERIOD_MS); - TickType_t last_wake = xTaskGetTickCount(); while (1) { - display_ui_update(); - lv_timer_handler(); + frame++; + + edge_vitals_pkt_t vitals; + bool has_vitals = edge_get_vitals(&vitals); + + battery_status_t battery; + bool has_battery = (battery_monitor_read(&battery) == ESP_OK && battery.valid); + + int motion_sample = motion_sample_from_vitals(&vitals, has_vitals); + int battery_fill = has_battery ? (int)battery.percent : 0; + int rssi_fill = has_vitals ? rssi_fill_from_vitals(&vitals, true) : 0; + int graph_sample = motion_sample; + if (graph_sample < DISP_GRAPH_MIN_VISIBLE) { + graph_sample = rssi_fill / 2; + } + + render_graph_frame(graph_sample, battery_fill, rssi_fill); + + ESP_LOGI(TAG, "Display frame %lu before present (graph=%d motion=%d battery=%u%% rssi=%d node=%u)", + (unsigned long)frame, + graph_sample, + motion_sample, + has_battery ? (unsigned)battery.percent : 0U, + has_vitals ? (int)vitals.rssi : 0, + (unsigned)csi_collector_get_node_id()); + display_hal_present(); + ESP_LOGI(TAG, "Display frame %lu after present", (unsigned long)frame); + + if (frame == 1) { + ESP_LOGI(TAG, "Display first graph frame rendered"); + } else if ((frame % 100U) == 0U) { + ESP_LOGI(TAG, "Display graph heartbeat frame %lu graph=%d motion=%d battery=%u%% rssi=%d", + (unsigned long)frame, graph_sample, motion_sample, + has_battery ? (unsigned)battery.percent : 0U, + has_vitals ? (int)vitals.rssi : 0); + } + vTaskDelayUntil(&last_wake, frame_period); } } @@ -83,73 +177,14 @@ esp_err_t display_task_start(void) { ESP_LOGI(TAG, "Initializing display subsystem..."); - bool use_psram = false; -#if CONFIG_SPIRAM - size_t psram_free = heap_caps_get_free_size(MALLOC_CAP_SPIRAM); - if (psram_free >= 64 * 1024) { - use_psram = true; - ESP_LOGI(TAG, "PSRAM available: %u KB — using PSRAM buffers", (unsigned)(psram_free / 1024)); - } else { - ESP_LOGW(TAG, "PSRAM too small (%u bytes) — falling back to internal DMA memory", (unsigned)psram_free); - } -#else - ESP_LOGW(TAG, "SPIRAM not enabled — using internal DMA memory (smaller buffers)"); -#endif - /* Probe display hardware */ esp_err_t ret = display_hal_init_panel(); if (ret != ESP_OK) { - ESP_LOGW(TAG, "Display not available — running headless"); + ESP_LOGW(TAG, "Display not available - running headless"); return ESP_OK; } - /* Init touch (optional) */ - esp_err_t touch_ret = display_hal_init_touch(); - - /* Initialize LVGL */ - lv_init(); - - /* Double-buffered draw buffers — prefer PSRAM, fall back to internal DMA */ - size_t buf_lines = use_psram ? DISP_BUF_LINES : 10; /* Smaller buffers without PSRAM */ - size_t buf_size = DISP_H_RES * buf_lines * sizeof(lv_color_t); - uint32_t alloc_caps = use_psram ? MALLOC_CAP_SPIRAM : (MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL); - lv_color_t *buf1 = heap_caps_malloc(buf_size, alloc_caps); - lv_color_t *buf2 = heap_caps_malloc(buf_size, alloc_caps); - if (!buf1 || !buf2) { - ESP_LOGE(TAG, "Failed to allocate LVGL buffers (%u bytes, caps=0x%lx)", - (unsigned)buf_size, (unsigned long)alloc_caps); - if (buf1) { - free(buf1); - buf1 = NULL; - } - if (buf2) { - free(buf2); - buf2 = NULL; - } - return ESP_OK; - } - ESP_LOGI(TAG, "LVGL buffers: 2x %u bytes (%u lines, %s)", - (unsigned)buf_size, (unsigned)buf_lines, use_psram ? "PSRAM" : "internal DMA"); - - static lv_disp_draw_buf_t draw_buf; - lv_disp_draw_buf_init(&draw_buf, buf1, buf2, DISP_H_RES * buf_lines); - - static lv_disp_drv_t disp_drv; - lv_disp_drv_init(&disp_drv); - disp_drv.hor_res = DISP_H_RES; - disp_drv.ver_res = DISP_V_RES; - disp_drv.flush_cb = lvgl_flush_cb; - disp_drv.draw_buf = &draw_buf; - lv_disp_drv_register(&disp_drv); - - if (touch_ret == ESP_OK) { - static lv_indev_drv_t indev_drv; - lv_indev_drv_init(&indev_drv); - indev_drv.type = LV_INDEV_TYPE_POINTER; - indev_drv.read_cb = lvgl_touch_cb; - lv_indev_drv_register(&indev_drv); - ESP_LOGI(TAG, "Touch input registered"); - } + ESP_LOGI(TAG, "Display panel initialized, starting graph loop"); BaseType_t xret = xTaskCreatePinnedToCore( display_task, "display", DISP_TASK_STACK, @@ -157,11 +192,11 @@ esp_err_t display_task_start(void) if (xret != pdPASS) { ESP_LOGE(TAG, "Failed to create display task"); - return ESP_OK; + return ESP_ERR_NO_MEM; } - ESP_LOGI(TAG, "Display task started (Core %d, priority %d, %d fps)", - DISP_TASK_CORE, DISP_TASK_PRIORITY, DISP_FPS_LIMIT); + ESP_LOGI(TAG, "Display task started (Core %d, priority %d, %u ms heartbeat)", + DISP_TASK_CORE, DISP_TASK_PRIORITY, (unsigned)DISP_HEARTBEAT_PERIOD_MS); return ESP_OK; } diff --git a/firmware/esp32-csi-node/main/display_task.h b/firmware/esp32-csi-node/main/display_task.h index b5af706005..98b1ee15ef 100644 --- a/firmware/esp32-csi-node/main/display_task.h +++ b/firmware/esp32-csi-node/main/display_task.h @@ -1,6 +1,6 @@ /** * @file display_task.h - * @brief ADR-045: FreeRTOS display task — LVGL pump on Core 0. + * @brief ADR-045: FreeRTOS display task — live graph loop. */ #ifndef DISPLAY_TASK_H @@ -13,12 +13,14 @@ extern "C" { #endif /** - * Start the display task on Core 0, priority 1. + * Start the live graph display task on Core 1, priority 6. * - * Probes for RM67162 panel and SPIRAM. If either is absent, - * logs a warning and returns ESP_OK (graceful skip). + * Probes for the active target's LCD hardware. If the LCD is absent, logs a + * warning and returns ESP_OK (graceful skip). If display init succeeds but + * the raw heartbeat task cannot be created, returns an error so the caller + * can log the real fault. * - * @return ESP_OK always (display is optional). + * @return ESP_OK on skip or success; error on display init/task failure. */ esp_err_t display_task_start(void); diff --git a/firmware/esp32-csi-node/main/display_ui.c b/firmware/esp32-csi-node/main/display_ui.c index 901867fbde..fa9463d1c1 100644 --- a/firmware/esp32-csi-node/main/display_ui.c +++ b/firmware/esp32-csi-node/main/display_ui.c @@ -2,13 +2,14 @@ * @file display_ui.c * @brief ADR-045: LVGL 4-view swipeable UI — Dashboard | Vitals | Presence | System. * - * Dark theme (#0a0a0f background) with cyan (#00d4ff) accent. - * Glowing line effects via layered semi-transparent chart series. + * High-contrast black/white feature dashboard for the Cardputer-Adv LCD. */ #include "display_ui.h" #include "nvs_config.h" #include "csi_collector.h" /* csi_collector_get_node_id() - defensive #390 */ +#include "c6_sync_espnow.h" +#include "cardputer_adv_audio.h" #include "sdkconfig.h" extern nvs_config_t g_nvs_config; @@ -21,25 +22,62 @@ extern nvs_config_t g_nvs_config; #include "esp_system.h" #include "esp_timer.h" #include "esp_heap_caps.h" +#include "driver/gpio.h" +#include "driver/i2c_master.h" #include "edge_processing.h" +#include "battery_monitor.h" static const char *TAG = "disp_ui"; /* ---- Theme colors ---- */ -#define COLOR_BG lv_color_make(0x0A, 0x0A, 0x0F) -#define COLOR_CYAN lv_color_make(0x00, 0xD4, 0xFF) -#define COLOR_AMBER lv_color_make(0xFF, 0xB0, 0x00) -#define COLOR_GREEN lv_color_make(0x00, 0xFF, 0x80) -#define COLOR_RED lv_color_make(0xFF, 0x40, 0x40) -#define COLOR_DIM lv_color_make(0x30, 0x30, 0x40) -#define COLOR_TEXT lv_color_make(0xCC, 0xCC, 0xDD) -#define COLOR_TEXT_DIM lv_color_make(0x66, 0x66, 0x77) +#define COLOR_BG lv_color_black() +#define COLOR_WHITE lv_color_white() +#define COLOR_DIM lv_color_make(0x55, 0x55, 0x55) +#define COLOR_TEXT lv_color_white() +#define COLOR_TEXT_DIM lv_color_make(0x80, 0x80, 0x80) /* ---- Chart data points ---- */ -#define CHART_POINTS 60 +#define CHART_POINTS 96 +#define SCREEN_W 240 +#define SCREEN_H 135 +#define GRID_GAP 0 +#define VIEW_COUNT 4 +#define VIEW_HOLD_MS 2000 +#define VIEW_KEY_HOLD_MS 15000 +#define UI_CONTENT_REFRESH_MS 125 + +/* Cardputer-Adv keyboard: TCA8418 on G8/G9/G11 per M5Stack pin map. */ +#define KB_I2C_PORT I2C_NUM_0 +#define KB_I2C_SDA 8 +#define KB_I2C_SCL 9 +#define KB_I2C_INT 11 +#define KB_I2C_ADDR 0x34 +#define TCA_REG_CFG 0x01 +#define TCA_REG_INT_STAT 0x02 +#define TCA_REG_KEY_LCK_EC 0x03 +#define TCA_REG_KEY_EVENT 0x04 +#define TCA_REG_GPI_EM1 0x09 +#define TCA_REG_KP_GPIO1 0x1D +#define TCA_REG_KP_GPIO2 0x1E +#define TCA_REG_KP_GPIO3 0x1F /* ---- View handles ---- */ static lv_obj_t *s_tileview = NULL; +static uint8_t s_active_view = 0; +static uint32_t s_last_view_switch_ms = 0; +static uint32_t s_last_key_ms = 0; +static uint32_t s_last_view_key_ms = 0; +static lv_obj_t *s_view_tabs[VIEW_COUNT]; +static const char *s_view_names[VIEW_COUNT] = {"DASH", "VITAL", "PRES", "SYS"}; +static lv_obj_t *s_scan_bar = NULL; +static lv_obj_t *s_scan_dot = NULL; +static lv_obj_t *s_debug_frame = NULL; +static uint32_t s_debug_total_frames = 0; + +static bool s_kb_init_attempted = false; +static bool s_kb_available = false; +static i2c_master_bus_handle_t s_kb_bus = NULL; +static i2c_master_dev_handle_t s_kb_dev = NULL; /* Dashboard */ static lv_obj_t *s_dash_chart = NULL; @@ -47,6 +85,11 @@ static lv_chart_series_t *s_csi_series = NULL; static lv_obj_t *s_dash_persons = NULL; static lv_obj_t *s_dash_rssi = NULL; static lv_obj_t *s_dash_motion = NULL; +static lv_obj_t *s_dash_adv = NULL; +static lv_obj_t *s_dash_stick = NULL; +static lv_obj_t *s_dash_server = NULL; +static lv_obj_t *s_dash_zoom = NULL; +static uint8_t s_dash_zoom_level = 0; /* Vitals */ static lv_obj_t *s_vital_chart = NULL; @@ -56,8 +99,8 @@ static lv_obj_t *s_vital_bpm_br = NULL; static lv_obj_t *s_vital_bpm_hr = NULL; /* Presence */ -#define GRID_COLS 4 -#define GRID_ROWS 4 +#define GRID_COLS 8 +#define GRID_ROWS 5 static lv_obj_t *s_grid_cells[GRID_COLS * GRID_ROWS]; static lv_obj_t *s_presence_label = NULL; @@ -69,6 +112,9 @@ static lv_obj_t *s_sys_rssi = NULL; static lv_obj_t *s_sys_uptime = NULL; static lv_obj_t *s_sys_fps = NULL; static lv_obj_t *s_sys_node = NULL; +static lv_obj_t *s_sys_battery = NULL; +static lv_obj_t *s_sys_power = NULL; +static lv_obj_t *s_sys_peer = NULL; /* ---- Style helpers ---- */ static lv_style_t s_style_bg; @@ -76,6 +122,14 @@ static lv_style_t s_style_label; static lv_style_t s_style_label_big; static bool s_styles_inited = false; +/* + * The esp-idf LVGL component is configured from sdkconfig in this build. Keep + * the UI on the default 14px font so display builds survive sdkconfig changes + * that disable the smaller Montserrat variants. + */ +#define UI_FONT_SMALL LV_FONT_DEFAULT +#define UI_FONT_BIG LV_FONT_DEFAULT + static void init_styles(void) { if (s_styles_inited) return; @@ -85,15 +139,15 @@ static void init_styles(void) lv_style_set_bg_color(&s_style_bg, COLOR_BG); lv_style_set_bg_opa(&s_style_bg, LV_OPA_COVER); lv_style_set_border_width(&s_style_bg, 0); - lv_style_set_pad_all(&s_style_bg, 4); + lv_style_set_pad_all(&s_style_bg, 0); lv_style_init(&s_style_label); lv_style_set_text_color(&s_style_label, COLOR_TEXT); - lv_style_set_text_font(&s_style_label, &lv_font_montserrat_14); + lv_style_set_text_font(&s_style_label, UI_FONT_SMALL); lv_style_init(&s_style_label_big); - lv_style_set_text_color(&s_style_label_big, COLOR_CYAN); - lv_style_set_text_font(&s_style_label_big, &lv_font_montserrat_14); + lv_style_set_text_color(&s_style_label_big, COLOR_WHITE); + lv_style_set_text_font(&s_style_label_big, UI_FONT_BIG); } static lv_obj_t *make_label(lv_obj_t *parent, const char *text, const lv_style_t *style) @@ -111,40 +165,350 @@ static lv_obj_t *make_tile(lv_obj_t *tv, uint8_t col, uint8_t row) return tile; } +static void format_battery_line(char *buf, size_t len, const char *label, + uint8_t node_id, bool live, uint8_t flags, + uint8_t percent, uint16_t millivolts, + uint8_t status, uint32_t age_ms) +{ + (void)status; + (void)age_ms; + if (!live) { + snprintf(buf, len, "%s: WAIT", label); + } else if (flags & 0x01) { + snprintf(buf, len, "%s%u %u%% %umV", + label, (unsigned)node_id, (unsigned)percent, + (unsigned)millivolts); + } else { + snprintf(buf, len, "%s%u LIVE batt?", label, (unsigned)node_id); + } +} + +static void apply_dash_chart_zoom(void) +{ + if (!s_dash_chart) return; + + static const uint16_t zoom_x[] = {256, 512, 1024}; + static const int32_t y_max[] = {100, 50, 25}; + static const char *label[] = {"Z1", "Z2", "Z4"}; + + uint8_t idx = s_dash_zoom_level; + if (idx >= 3) idx = 0; + + lv_chart_set_range(s_dash_chart, LV_CHART_AXIS_PRIMARY_Y, 0, y_max[idx]); + lv_chart_set_zoom_x(s_dash_chart, zoom_x[idx]); + lv_chart_set_zoom_y(s_dash_chart, 256); + if (s_dash_zoom) { + lv_label_set_text(s_dash_zoom, label[idx]); + } +} + +static void dash_chart_event_cb(lv_event_t *e) +{ + if (lv_event_get_code(e) != LV_EVENT_CLICKED) return; + s_dash_zoom_level = (s_dash_zoom_level + 1) % 3; + apply_dash_chart_zoom(); +} + +static void update_view_tabs(void) +{ + static const char *active_names[VIEW_COUNT] = {"", "", "", ""}; + for (uint8_t i = 0; i < VIEW_COUNT; i++) { + if (!s_view_tabs[i]) continue; + bool active = i == s_active_view; + lv_obj_set_style_bg_color(s_view_tabs[i], active ? COLOR_WHITE : COLOR_BG, 0); + lv_obj_set_style_bg_opa(s_view_tabs[i], active ? LV_OPA_COVER : LV_OPA_70, 0); + lv_obj_set_style_border_color(s_view_tabs[i], COLOR_WHITE, 0); + lv_obj_set_style_border_width(s_view_tabs[i], 1, 0); + lv_obj_set_style_text_color(s_view_tabs[i], active ? COLOR_BG : COLOR_WHITE, 0); + lv_label_set_text(s_view_tabs[i], active ? active_names[i] : s_view_names[i]); + lv_obj_move_foreground(s_view_tabs[i]); + } +} + +static void update_scan_marker(uint32_t now_ms) +{ + int x = (int)((now_ms / 18) % SCREEN_W); + int y = 17 + (int)((now_ms / 31) % (SCREEN_H - 38)); + + if (s_scan_bar) { + lv_obj_set_pos(s_scan_bar, x, 0); + lv_obj_move_foreground(s_scan_bar); + } + if (s_scan_dot) { + lv_obj_set_pos(s_scan_dot, x > 5 ? x - 5 : x, y); + lv_obj_set_style_bg_color(s_scan_dot, ((now_ms / 250) & 1) ? COLOR_WHITE : COLOR_TEXT_DIM, 0); + lv_obj_move_foreground(s_scan_dot); + } +} + +static void create_view_tabs(lv_obj_t *parent) +{ + const int tab_w = SCREEN_W / VIEW_COUNT; + for (uint8_t i = 0; i < VIEW_COUNT; i++) { + lv_obj_t *tab = make_label(parent, s_view_names[i], &s_style_label); + lv_obj_set_size(tab, tab_w, 15); + lv_obj_set_pos(tab, i * tab_w, SCREEN_H - 15); + lv_obj_set_style_text_align(tab, LV_TEXT_ALIGN_CENTER, 0); + lv_obj_set_style_pad_top(tab, 1, 0); + lv_obj_set_style_pad_bottom(tab, 1, 0); + lv_obj_set_style_pad_left(tab, 0, 0); + lv_obj_set_style_pad_right(tab, 0, 0); + s_view_tabs[i] = tab; + } + update_view_tabs(); + + s_scan_bar = lv_obj_create(parent); + lv_obj_set_size(s_scan_bar, 18, SCREEN_H); + lv_obj_set_pos(s_scan_bar, 0, 0); + lv_obj_set_style_bg_color(s_scan_bar, COLOR_WHITE, 0); + lv_obj_set_style_bg_opa(s_scan_bar, LV_OPA_70, 0); + lv_obj_set_style_border_width(s_scan_bar, 0, 0); + lv_obj_set_style_pad_all(s_scan_bar, 0, 0); + lv_obj_set_style_radius(s_scan_bar, 0, 0); + + s_scan_dot = lv_obj_create(parent); + lv_obj_set_size(s_scan_dot, 32, 32); + lv_obj_set_pos(s_scan_dot, 0, 30); + lv_obj_set_style_bg_color(s_scan_dot, COLOR_WHITE, 0); + lv_obj_set_style_bg_opa(s_scan_dot, LV_OPA_COVER, 0); + lv_obj_set_style_border_width(s_scan_dot, 0, 0); + lv_obj_set_style_pad_all(s_scan_dot, 0, 0); + lv_obj_set_style_radius(s_scan_dot, 0, 0); + + s_debug_frame = make_label(parent, "FRAME 000000", &s_style_label); + lv_obj_align(s_debug_frame, LV_ALIGN_TOP_RIGHT, -4, 4); + lv_obj_set_style_text_color(s_debug_frame, COLOR_WHITE, 0); + lv_obj_set_style_bg_color(s_debug_frame, COLOR_BG, 0); + lv_obj_set_style_bg_opa(s_debug_frame, LV_OPA_COVER, 0); + lv_obj_move_foreground(s_debug_frame); +} + +static void select_view(uint8_t view, bool anim, uint32_t now_ms) +{ + if (!s_tileview) return; + if (view >= VIEW_COUNT) view = 0; + s_active_view = view; + lv_obj_set_tile_id(s_tileview, s_active_view, 0, anim ? LV_ANIM_ON : LV_ANIM_OFF); + update_view_tabs(); + s_last_view_switch_ms = now_ms; +} + +static esp_err_t kb_write_reg(uint8_t reg, uint8_t value) +{ + uint8_t data[2] = {reg, value}; + return i2c_master_transmit(s_kb_dev, data, sizeof(data), 20); +} + +static esp_err_t kb_read_reg(uint8_t reg, uint8_t *value) +{ + return i2c_master_transmit_receive(s_kb_dev, ®, 1, value, 1, 20); +} + +static void init_keyboard(void) +{ + if (s_kb_init_attempted) return; + s_kb_init_attempted = true; + + gpio_config_t int_cfg = { + .pin_bit_mask = 1ULL << KB_I2C_INT, + .mode = GPIO_MODE_INPUT, + .pull_up_en = GPIO_PULLUP_ENABLE, + .pull_down_en = GPIO_PULLDOWN_DISABLE, + .intr_type = GPIO_INTR_DISABLE, + }; + gpio_config(&int_cfg); + + i2c_master_bus_config_t bus_cfg = { + .i2c_port = KB_I2C_PORT, + .sda_io_num = KB_I2C_SDA, + .scl_io_num = KB_I2C_SCL, + .clk_source = I2C_CLK_SRC_DEFAULT, + .glitch_ignore_cnt = 7, + .flags.enable_internal_pullup = true, + }; + esp_err_t err = i2c_new_master_bus(&bus_cfg, &s_kb_bus); + if (err == ESP_ERR_INVALID_STATE) { + err = i2c_master_get_bus_handle(KB_I2C_PORT, &s_kb_bus); + if (err == ESP_OK) { + ESP_LOGI(TAG, "keyboard reusing I2C%d bus on G%d/G%d", + KB_I2C_PORT, KB_I2C_SDA, KB_I2C_SCL); + } + } + if (err != ESP_OK) { + ESP_LOGW(TAG, "keyboard I2C bus init failed: %s", esp_err_to_name(err)); + return; + } + + i2c_device_config_t dev_cfg = { + .dev_addr_length = I2C_ADDR_BIT_LEN_7, + .device_address = KB_I2C_ADDR, + .scl_speed_hz = 400000, + }; + err = i2c_master_bus_add_device(s_kb_bus, &dev_cfg, &s_kb_dev); + if (err != ESP_OK) { + ESP_LOGW(TAG, "keyboard TCA8418 add failed: %s", esp_err_to_name(err)); + return; + } + + /* + * Cardputer-ADV uses the TCA8418 in an 8x8 keypad mode. The controller + * otherwise powers up quietly and may acknowledge I2C without emitting keys. + */ + kb_write_reg(TCA_REG_KP_GPIO1, 0xFF); + kb_write_reg(TCA_REG_KP_GPIO2, 0xFF); + kb_write_reg(TCA_REG_KP_GPIO3, 0x00); + kb_write_reg(TCA_REG_GPI_EM1, 0x00); + kb_write_reg(TCA_REG_INT_STAT, 0xFF); + kb_write_reg(TCA_REG_CFG, 0x3E); + + uint8_t ec = 0; + if (kb_read_reg(TCA_REG_KEY_LCK_EC, &ec) == ESP_OK) { + s_kb_available = true; + ESP_LOGI(TAG, "keyboard ready: TCA8418 addr=0x%02x SDA=G%d SCL=G%d INT=G%d", + KB_I2C_ADDR, KB_I2C_SDA, KB_I2C_SCL, KB_I2C_INT); + } else { + ESP_LOGW(TAG, "keyboard TCA8418 not responding at 0x%02x", KB_I2C_ADDR); + } +} + +static void handle_key_event(uint8_t code, bool pressed, uint32_t now_ms) +{ + s_last_key_ms = now_ms; + +#if defined(CONFIG_CARDPUTER_ADV_AUDIO_ENABLE) && defined(CONFIG_IDF_TARGET_ESP32S3) + cardputer_adv_audio_key_event(code, pressed); +#endif + + if (!pressed || code < 1 || code > VIEW_COUNT) { + return; + } + if ((uint32_t)(now_ms - s_last_view_key_ms) <= 180U) { + return; + } + + uint8_t view = code - 1; + s_last_view_key_ms = now_ms; + ESP_LOGI(TAG, "keyboard key=%u -> view=%s", (unsigned)code, s_view_names[view]); + select_view(view, true, now_ms); +} + +static void __attribute__((unused)) update_keyboard(uint32_t now_ms) +{ + init_keyboard(); + if (!s_kb_available) return; + + uint8_t ec = 0; + if (kb_read_reg(TCA_REG_KEY_LCK_EC, &ec) != ESP_OK) return; + uint8_t count = ec & 0x0F; + if (count == 0 && gpio_get_level(KB_I2C_INT) == 0) { + count = 10; + } + if (count > 10) count = 10; + + for (uint8_t i = 0; i < count; i++) { + uint8_t event = 0; + if (kb_read_reg(TCA_REG_KEY_EVENT, &event) != ESP_OK) return; + if (event == 0) { + break; + } + bool pressed = (event & 0x80) != 0; + uint8_t code = event & 0x7F; + if (code != 0) { + handle_key_event(code, pressed, now_ms); + } + } + kb_write_reg(TCA_REG_INT_STAT, 0xFF); +} + +static int moving_trace_value(uint32_t now_ms, bool has_vitals, const edge_vitals_pkt_t *vitals) +{ + uint32_t phase = (now_ms / 90) % 48; + int sweep = (phase < 24) ? (int)phase : (int)(47 - phase); + int val = 10 + sweep * 3; + + if (has_vitals && vitals) { + int motion = (int)(vitals->motion_energy * 18.0f); + if (motion > val) { + val = motion; + } + if (vitals->rssi < 0) { + int rssi_motion = 100 + vitals->rssi; + if (rssi_motion > val) { + val = rssi_motion; + } + } + } + + if (val > 100) val = 100; + if (val < 0) val = 0; + return val; +} + +static void update_auto_view(uint32_t now_ms) +{ + if (!s_tileview) return; + if (s_last_key_ms != 0 && now_ms - s_last_key_ms < VIEW_KEY_HOLD_MS) return; + if (s_last_view_switch_ms == 0) { + s_last_view_switch_ms = now_ms; + return; + } + if (now_ms - s_last_view_switch_ms < VIEW_HOLD_MS) return; + + s_active_view = (s_active_view + 1) % VIEW_COUNT; + select_view(s_active_view, true, now_ms); +} + /* ---- View 0: Dashboard ---- */ static void create_dashboard(lv_obj_t *tile) { - make_label(tile, "CSI Dashboard", &s_style_label); - - /* CSI amplitude chart */ s_dash_chart = lv_chart_create(tile); - lv_obj_set_size(s_dash_chart, 400, 130); - lv_obj_align(s_dash_chart, LV_ALIGN_TOP_LEFT, 0, 24); + lv_obj_set_size(s_dash_chart, SCREEN_W, SCREEN_H); + lv_obj_align(s_dash_chart, LV_ALIGN_TOP_LEFT, 0, 0); lv_chart_set_type(s_dash_chart, LV_CHART_TYPE_LINE); lv_chart_set_point_count(s_dash_chart, CHART_POINTS); lv_chart_set_range(s_dash_chart, LV_CHART_AXIS_PRIMARY_Y, 0, 100); + lv_chart_set_div_line_count(s_dash_chart, 7, 13); + + /* Scope-style field: graph lines only, no outer box. */ lv_obj_set_style_bg_color(s_dash_chart, COLOR_BG, 0); - lv_obj_set_style_border_color(s_dash_chart, COLOR_DIM, 0); + lv_obj_set_style_border_width(s_dash_chart, 0, 0); + lv_obj_set_style_pad_all(s_dash_chart, 0, 0); + lv_obj_set_style_line_color(s_dash_chart, COLOR_DIM, LV_PART_MAIN); + lv_obj_set_style_line_opa(s_dash_chart, LV_OPA_60, LV_PART_MAIN); lv_obj_set_style_line_width(s_dash_chart, 0, LV_PART_TICKS); + lv_obj_add_flag(s_dash_chart, LV_OBJ_FLAG_CLICKABLE); + lv_obj_add_event_cb(s_dash_chart, dash_chart_event_cb, LV_EVENT_CLICKED, NULL); - s_csi_series = lv_chart_add_series(s_dash_chart, COLOR_CYAN, LV_CHART_AXIS_PRIMARY_Y); + s_csi_series = lv_chart_add_series(s_dash_chart, COLOR_WHITE, LV_CHART_AXIS_PRIMARY_Y); + lv_obj_set_style_size(s_dash_chart, 0, LV_PART_INDICATOR); + lv_obj_set_style_line_width(s_dash_chart, 2, LV_PART_ITEMS); - /* Stats panel on the right */ - lv_obj_t *panel = lv_obj_create(tile); - lv_obj_set_size(panel, 120, 130); - lv_obj_align(panel, LV_ALIGN_TOP_RIGHT, 0, 24); - lv_obj_set_style_bg_color(panel, lv_color_make(0x12, 0x12, 0x1A), 0); - lv_obj_set_style_border_width(panel, 1, 0); - lv_obj_set_style_border_color(panel, COLOR_DIM, 0); - lv_obj_set_style_pad_all(panel, 8, 0); - lv_obj_set_flex_flow(panel, LV_FLEX_FLOW_COLUMN); - lv_obj_set_flex_align(panel, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START); + /* Edge telemetry. Labels are drawn directly on black; no panels/boxes. */ + lv_obj_t *title = make_label(tile, "ADV UI 0602", &s_style_label); + lv_obj_align(title, LV_ALIGN_TOP_LEFT, 0, 0); - make_label(panel, "Persons", &s_style_label); - s_dash_persons = make_label(panel, "0", &s_style_label_big); + s_dash_zoom = make_label(tile, "Z1", &s_style_label); + lv_obj_align(s_dash_zoom, LV_ALIGN_TOP_MID, 0, 0); - s_dash_rssi = make_label(panel, "RSSI: --", &s_style_label); - s_dash_motion = make_label(panel, "Motion: 0.0", &s_style_label); + s_dash_adv = make_label(tile, "ADV --", &s_style_label); + lv_obj_align(s_dash_adv, LV_ALIGN_TOP_RIGHT, 0, 0); + + s_dash_persons = make_label(tile, "P0", &s_style_label_big); + lv_obj_align(s_dash_persons, LV_ALIGN_RIGHT_MID, 0, -20); + + s_dash_rssi = make_label(tile, "R--", &s_style_label); + lv_obj_align(s_dash_rssi, LV_ALIGN_LEFT_MID, 0, 0); + + s_dash_motion = make_label(tile, "M0.0", &s_style_label); + lv_obj_align(s_dash_motion, LV_ALIGN_BOTTOM_MID, 0, 0); + + s_dash_server = make_label(tile, "SRC WAIT", &s_style_label); + lv_obj_align(s_dash_server, LV_ALIGN_BOTTOM_LEFT, 0, 0); + + s_dash_stick = make_label(tile, "STK WAIT", &s_style_label); + lv_obj_align(s_dash_stick, LV_ALIGN_BOTTOM_RIGHT, 0, 0); + + apply_dash_chart_zoom(); } /* ---- View 1: Vitals ---- */ @@ -153,80 +517,90 @@ static void create_vitals(lv_obj_t *tile) make_label(tile, "Vital Signs", &s_style_label); s_vital_chart = lv_chart_create(tile); - lv_obj_set_size(s_vital_chart, 480, 150); - lv_obj_align(s_vital_chart, LV_ALIGN_TOP_LEFT, 0, 24); + lv_obj_set_size(s_vital_chart, SCREEN_W, SCREEN_H - 15); + lv_obj_align(s_vital_chart, LV_ALIGN_TOP_LEFT, 0, 0); lv_chart_set_type(s_vital_chart, LV_CHART_TYPE_LINE); lv_chart_set_point_count(s_vital_chart, CHART_POINTS); lv_chart_set_range(s_vital_chart, LV_CHART_AXIS_PRIMARY_Y, 0, 120); + lv_chart_set_div_line_count(s_vital_chart, 7, 13); lv_obj_set_style_bg_color(s_vital_chart, COLOR_BG, 0); - lv_obj_set_style_border_color(s_vital_chart, COLOR_DIM, 0); + lv_obj_set_style_border_width(s_vital_chart, 0, 0); + lv_obj_set_style_pad_all(s_vital_chart, 0, 0); + lv_obj_set_style_line_color(s_vital_chart, COLOR_DIM, LV_PART_MAIN); + lv_obj_set_style_line_opa(s_vital_chart, LV_OPA_60, LV_PART_MAIN); lv_obj_set_style_line_width(s_vital_chart, 0, LV_PART_TICKS); - /* Breathing series (cyan) */ - s_breath_series = lv_chart_add_series(s_vital_chart, COLOR_CYAN, LV_CHART_AXIS_PRIMARY_Y); - /* Heart rate series (amber) */ - s_hr_series = lv_chart_add_series(s_vital_chart, COLOR_AMBER, LV_CHART_AXIS_PRIMARY_Y); + s_breath_series = lv_chart_add_series(s_vital_chart, COLOR_WHITE, LV_CHART_AXIS_PRIMARY_Y); + s_hr_series = lv_chart_add_series(s_vital_chart, COLOR_TEXT_DIM, LV_CHART_AXIS_PRIMARY_Y); + lv_obj_set_style_size(s_vital_chart, 0, LV_PART_INDICATOR); + lv_obj_set_style_line_width(s_vital_chart, 2, LV_PART_ITEMS); /* BPM readouts */ s_vital_bpm_br = make_label(tile, "Breathing: -- BPM", &s_style_label); - lv_obj_align(s_vital_bpm_br, LV_ALIGN_BOTTOM_LEFT, 4, -8); - lv_obj_set_style_text_color(s_vital_bpm_br, COLOR_CYAN, 0); + lv_obj_align(s_vital_bpm_br, LV_ALIGN_BOTTOM_LEFT, 4, -4); + lv_obj_set_style_text_color(s_vital_bpm_br, COLOR_WHITE, 0); s_vital_bpm_hr = make_label(tile, "Heart Rate: -- BPM", &s_style_label); - lv_obj_align(s_vital_bpm_hr, LV_ALIGN_BOTTOM_RIGHT, -4, -8); - lv_obj_set_style_text_color(s_vital_bpm_hr, COLOR_AMBER, 0); + lv_obj_align(s_vital_bpm_hr, LV_ALIGN_BOTTOM_RIGHT, -4, -4); + lv_obj_set_style_text_color(s_vital_bpm_hr, COLOR_TEXT_DIM, 0); } /* ---- View 2: Presence Grid ---- */ static void create_presence(lv_obj_t *tile) { - make_label(tile, "Occupancy Map", &s_style_label); + make_label(tile, "Occupancy", &s_style_label); + + s_presence_label = make_label(tile, "Persons: 0", &s_style_label); + lv_obj_align(s_presence_label, LV_ALIGN_TOP_RIGHT, -2, 0); - int cell_w = 50; - int cell_h = 45; - int x_off = (368 - GRID_COLS * (cell_w + 4)) / 2; - int y_off = 30; + int cell_w = (SCREEN_W - ((GRID_COLS - 1) * GRID_GAP)) / GRID_COLS; + int cell_h = (SCREEN_H - 13 - ((GRID_ROWS - 1) * GRID_GAP)) / GRID_ROWS; + int grid_w = GRID_COLS * cell_w + (GRID_COLS - 1) * GRID_GAP; + int x_off = (SCREEN_W - grid_w) / 2; + int y_off = 13; for (int r = 0; r < GRID_ROWS; r++) { for (int c = 0; c < GRID_COLS; c++) { lv_obj_t *cell = lv_obj_create(tile); lv_obj_set_size(cell, cell_w, cell_h); - lv_obj_set_pos(cell, x_off + c * (cell_w + 4), y_off + r * (cell_h + 4)); + lv_obj_set_pos(cell, x_off + c * (cell_w + GRID_GAP), y_off + r * (cell_h + GRID_GAP)); lv_obj_set_style_bg_color(cell, COLOR_DIM, 0); lv_obj_set_style_bg_opa(cell, LV_OPA_COVER, 0); - lv_obj_set_style_border_color(cell, COLOR_DIM, 0); - lv_obj_set_style_border_width(cell, 1, 0); - lv_obj_set_style_radius(cell, 4, 0); + lv_obj_set_style_border_width(cell, 0, 0); + lv_obj_set_style_pad_all(cell, 0, 0); + lv_obj_set_style_radius(cell, 0, 0); s_grid_cells[r * GRID_COLS + c] = cell; } } - - s_presence_label = make_label(tile, "Persons: 0", &s_style_label); - lv_obj_align(s_presence_label, LV_ALIGN_BOTTOM_MID, 0, -8); } /* ---- View 3: System ---- */ static void create_system(lv_obj_t *tile) { - make_label(tile, "System Info", &s_style_label); - - lv_obj_t *panel = lv_obj_create(tile); - lv_obj_set_size(panel, 500, 180); - lv_obj_align(panel, LV_ALIGN_TOP_LEFT, 0, 24); - lv_obj_set_style_bg_color(panel, lv_color_make(0x12, 0x12, 0x1A), 0); - lv_obj_set_style_border_width(panel, 1, 0); - lv_obj_set_style_border_color(panel, COLOR_DIM, 0); - lv_obj_set_style_pad_all(panel, 10, 0); - lv_obj_set_flex_flow(panel, LV_FLEX_FLOW_COLUMN); - lv_obj_set_flex_align(panel, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START); - - s_sys_node = make_label(panel, "Node: --", &s_style_label); - s_sys_cpu = make_label(panel, "CPU: --%", &s_style_label); - s_sys_heap = make_label(panel, "Heap: -- KB free", &s_style_label); - s_sys_psram = make_label(panel, "PSRAM: -- KB free",&s_style_label); - s_sys_rssi = make_label(panel, "WiFi RSSI: --", &s_style_label); - s_sys_uptime = make_label(panel, "Uptime: --", &s_style_label); - s_sys_fps = make_label(panel, "FPS: --", &s_style_label); + /* Two-column raw text, no containing panel. */ + s_sys_node = make_label(tile, "Node: --", &s_style_label); + s_sys_cpu = make_label(tile, "CPU: --%", &s_style_label); + s_sys_heap = make_label(tile, "Heap: -- KB free", &s_style_label); + s_sys_psram = make_label(tile, "PSRAM: -- KB free", &s_style_label); + s_sys_rssi = make_label(tile, "WiFi RSSI: --", &s_style_label); + + s_sys_battery = make_label(tile, "Battery: UNKNOWN", &s_style_label); + s_sys_power = make_label(tile, "Power: UNKNOWN", &s_style_label); + s_sys_peer = make_label(tile, "Peer: WAITING", &s_style_label); + s_sys_uptime = make_label(tile, "Uptime: --", &s_style_label); + s_sys_fps = make_label(tile, "FPS: --", &s_style_label); + + lv_obj_set_pos(s_sys_node, 0, 0); + lv_obj_set_pos(s_sys_cpu, 0, 13); + lv_obj_set_pos(s_sys_heap, 0, 26); + lv_obj_set_pos(s_sys_psram, 0, 39); + lv_obj_set_pos(s_sys_rssi, 0, 52); + + lv_obj_set_pos(s_sys_battery, 112, 0); + lv_obj_set_pos(s_sys_power, 112, 13); + lv_obj_set_pos(s_sys_peer, 112, 26); + lv_obj_set_pos(s_sys_uptime, 112, 39); + lv_obj_set_pos(s_sys_fps, 112, 52); } /* ---- Public API ---- */ @@ -248,65 +622,113 @@ void display_ui_create(lv_obj_t *parent) create_vitals(t1); create_presence(t2); create_system(t3); + create_view_tabs(parent); - ESP_LOGI(TAG, "UI created: 4 views (Dashboard|Vitals|Presence|System)"); + ESP_LOGI(TAG, "UI created: 4 views with tab buttons (DASH|VITAL|PRES|SYS)"); } /* ---- FPS tracking ---- */ static uint32_t s_frame_count = 0; static uint32_t s_last_fps_time = 0; static uint32_t s_current_fps = 0; +static uint32_t s_last_content_refresh_ms = 0; void display_ui_update(void) { /* FPS counter */ s_frame_count++; + s_debug_total_frames++; + if (s_debug_frame) { + char dbg[32]; + snprintf(dbg, sizeof(dbg), "FRAME %06lu", (unsigned long)s_debug_total_frames); + lv_label_set_text(s_debug_frame, dbg); + lv_obj_move_foreground(s_debug_frame); + } uint32_t now_ms = (uint32_t)(esp_timer_get_time() / 1000); +#if !defined(CONFIG_CARDPUTER_ADV_TRAUTONIUM_ENABLE) + update_keyboard(now_ms); +#endif + update_auto_view(now_ms); + update_scan_marker(now_ms); if (now_ms - s_last_fps_time >= 1000) { s_current_fps = s_frame_count; s_frame_count = 0; s_last_fps_time = now_ms; } + if (s_last_content_refresh_ms != 0 && + (uint32_t)(now_ms - s_last_content_refresh_ms) < UI_CONTENT_REFRESH_MS) { + return; + } + s_last_content_refresh_ms = now_ms; + /* Read edge data (thread-safe) */ edge_vitals_pkt_t vitals; bool has_vitals = edge_get_vitals(&vitals); - edge_person_vitals_t persons[EDGE_MAX_PERSONS]; - uint8_t n_active = 0; - edge_get_multi_person(persons, &n_active); - /* ---- Dashboard update ---- */ - if (s_dash_chart && has_vitals) { - /* Push motion energy as amplitude proxy (scaled 0-100) */ - int val = (int)(vitals.motion_energy * 10.0f); - if (val > 100) val = 100; - if (val < 0) val = 0; + if (s_dash_chart) { + int val = moving_trace_value(now_ms, has_vitals, &vitals); lv_chart_set_next_value(s_dash_chart, s_csi_series, val); } if (s_dash_persons) { char buf[8]; - snprintf(buf, sizeof(buf), "%u", has_vitals ? vitals.n_persons : 0); + snprintf(buf, sizeof(buf), "P%u", has_vitals ? vitals.n_persons : 0); lv_label_set_text(s_dash_persons, buf); } if (s_dash_rssi && has_vitals) { char buf[16]; - snprintf(buf, sizeof(buf), "RSSI: %d", vitals.rssi); + snprintf(buf, sizeof(buf), "R%d", vitals.rssi); lv_label_set_text(s_dash_rssi, buf); } - if (s_dash_motion && has_vitals) { + if (s_dash_motion) { char buf[24]; - snprintf(buf, sizeof(buf), "Motion: %.1f", (double)vitals.motion_energy); + if (has_vitals) { + snprintf(buf, sizeof(buf), "M%.1f", (double)vitals.motion_energy); + } else { + snprintf(buf, sizeof(buf), "M scan"); + } lv_label_set_text(s_dash_motion, buf); } + if (s_dash_server) { + lv_label_set_text(s_dash_server, has_vitals ? "SRC LIVE" : "SRC WAIT"); + } + + { + char buf[48]; + battery_status_t battery; + if (s_dash_adv && battery_monitor_read(&battery) == ESP_OK && battery.valid) { + format_battery_line(buf, sizeof(buf), "ADV", csi_collector_get_node_id(), true, + 0x01 | (battery.charging ? 0x02 : 0x00), + battery.percent, battery.millivolts, + (uint8_t)battery.status, 0); + lv_label_set_text(s_dash_adv, buf); + } else if (s_dash_adv) { + format_battery_line(buf, sizeof(buf), "ADV", csi_collector_get_node_id(), false, + 0, 255, 0, BATTERY_POWER_UNKNOWN, 0); + lv_label_set_text(s_dash_adv, buf); + } + + c6_espnow_peer_status_t peer; + bool peer_live = c6_sync_espnow_get_peer_status(&peer); + if (s_dash_stick) { + format_battery_line(buf, sizeof(buf), "STK", peer.node_id, peer_live, + peer.flags, peer.percent, peer.millivolts, + peer.status, peer.age_ms); + lv_label_set_text(s_dash_stick, buf); + } + } + /* ---- Vitals update ---- */ - if (s_vital_chart && has_vitals) { - int br = (int)(vitals.breathing_rate / 100); /* Fixed-point to int BPM */ - int hr = (int)(vitals.heartrate / 10000); + if (s_vital_chart) { + int br = has_vitals ? (int)(vitals.breathing_rate / 100) : 0; /* Fixed-point to int BPM */ + int hr = has_vitals ? (int)(vitals.heartrate / 10000) : 0; + if (br <= 0) br = 12 + (int)((now_ms / 500) % 8); + if (hr <= 0) hr = 58 + (int)((now_ms / 180) % 22); if (br > 120) br = 120; if (hr > 120) hr = 120; lv_chart_set_next_value(s_vital_chart, s_breath_series, br); @@ -321,29 +743,31 @@ void display_ui_update(void) } /* ---- Presence grid update ---- */ - if (has_vitals) { - /* Simple visualization: color cells based on motion energy distribution */ - float energy = vitals.motion_energy; - uint8_t active_cells = (uint8_t)(energy * 2); /* Scale for visibility */ - if (active_cells > GRID_COLS * GRID_ROWS) active_cells = GRID_COLS * GRID_ROWS; + { + uint8_t active_cells = 0; + uint8_t sweep_cell = (uint8_t)((now_ms / 85) % (GRID_COLS * GRID_ROWS)); + + if (has_vitals) { + float energy = vitals.motion_energy; + active_cells = (uint8_t)(energy * 2); /* Scale for visibility */ + if (active_cells > GRID_COLS * GRID_ROWS) active_cells = GRID_COLS * GRID_ROWS; + } else { + active_cells = (uint8_t)(4 + ((now_ms / 250) % 18)); + } for (int i = 0; i < GRID_COLS * GRID_ROWS; i++) { - if (i < active_cells) { - /* Color gradient: green → amber → red based on intensity */ - if (energy > 5.0f) { - lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_RED, 0); - } else if (energy > 2.0f) { - lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_AMBER, 0); - } else { - lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_GREEN, 0); - } + if (i == sweep_cell) { + lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_WHITE, 0); + } else if (i < active_cells) { + lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_TEXT_DIM, 0); } else { lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_DIM, 0); } } char buf[20]; - snprintf(buf, sizeof(buf), "Persons: %u", vitals.n_persons); + snprintf(buf, sizeof(buf), has_vitals ? "Persons: %u" : "SCAN %02u", + has_vitals ? vitals.n_persons : sweep_cell); lv_label_set_text(s_presence_label, buf); } @@ -371,6 +795,33 @@ void display_ui_update(void) lv_label_set_text(s_sys_rssi, buf); } + battery_status_t battery; + if (battery_monitor_read(&battery) == ESP_OK && battery.valid) { + snprintf(buf, sizeof(buf), "Battery: %u%% (%umV)", + (unsigned)battery.percent, (unsigned)battery.millivolts); + lv_label_set_text(s_sys_battery, buf); + snprintf(buf, sizeof(buf), "Power: %s", + battery_monitor_status_name(battery.status)); + lv_label_set_text(s_sys_power, buf); + } else { + lv_label_set_text(s_sys_battery, "Battery: UNKNOWN"); + lv_label_set_text(s_sys_power, "Power: UNKNOWN"); + } + + c6_espnow_peer_status_t peer; + if (c6_sync_espnow_get_peer_status(&peer)) { + if (peer.flags & 0x01) { + snprintf(buf, sizeof(buf), "Peer n%u: %u%% %umV", + (unsigned)peer.node_id, (unsigned)peer.percent, + (unsigned)peer.millivolts); + } else { + snprintf(buf, sizeof(buf), "Peer n%u: LIVE batt?", (unsigned)peer.node_id); + } + lv_label_set_text(s_sys_peer, buf); + } else { + lv_label_set_text(s_sys_peer, "Peer: WAITING"); + } + uint32_t uptime_s = (uint32_t)(esp_timer_get_time() / 1000000); uint32_t h = uptime_s / 3600; uint32_t m = (uptime_s % 3600) / 60; diff --git a/firmware/esp32-csi-node/main/edge_processing.c b/firmware/esp32-csi-node/main/edge_processing.c index ea67738c4f..67061539c6 100644 --- a/firmware/esp32-csi-node/main/edge_processing.c +++ b/firmware/esp32-csi-node/main/edge_processing.c @@ -22,6 +22,7 @@ #include "nvs_config.h" #include "csi_collector.h" /* csi_collector_get_node_id() - defensive #390 */ #include "mmwave_sensor.h" +#include "battery_monitor.h" /* Runtime config — declared in main.c, loaded from NVS at boot. */ extern nvs_config_t g_nvs_config; @@ -273,6 +274,9 @@ static float s_adaptive_threshold; /** Last vitals send timestamp. */ static int64_t s_last_vitals_send_us; +/** Last battery telemetry send timestamp. */ +static int64_t s_last_battery_send_us; + /** Delta compression state. */ static uint8_t s_prev_iq[EDGE_MAX_IQ_BYTES]; static uint16_t s_prev_iq_len; @@ -703,6 +707,29 @@ static void send_feature_vector(void) stream_sender_send((const uint8_t *)&pkt, sizeof(pkt)); } +static void send_battery_packet(void) +{ + edge_battery_pkt_t pkt; + battery_status_t battery; + memset(&pkt, 0, sizeof(pkt)); + + pkt.magic = EDGE_BATTERY_MAGIC; + pkt.node_id = csi_collector_get_node_id(); + pkt.percent = 255; + pkt.status = BATTERY_POWER_UNKNOWN; + pkt.timestamp_ms = (uint32_t)(esp_timer_get_time() / 1000); + + if (battery_monitor_read(&battery) == ESP_OK && battery.valid) { + pkt.percent = battery.percent; + pkt.millivolts = battery.millivolts; + pkt.status = (uint8_t)battery.status; + pkt.flags |= 0x01; + if (battery.charging) pkt.flags |= 0x02; + } + + stream_sender_send((const uint8_t *)&pkt, sizeof(pkt)); +} + /* ====================================================================== * Main DSP Pipeline (runs on Core 1) * ====================================================================== */ @@ -877,6 +904,14 @@ static void process_frame(const edge_ring_slot_t *slot) } } +#if CONFIG_BATTERY_MONITOR_ENABLE + int64_t battery_interval_us = (int64_t)CONFIG_BATTERY_SEND_INTERVAL_MS * 1000; + if ((now_us - s_last_battery_send_us) >= battery_interval_us) { + send_battery_packet(); + s_last_battery_send_us = now_us; + } +#endif + /* --- Step 14 (ADR-040): Dispatch to WASM modules --- */ if (s_cfg.tier >= 2 && s_pkt_valid) { /* Extract amplitudes from I/Q for WASM host API. */ diff --git a/firmware/esp32-csi-node/main/edge_processing.h b/firmware/esp32-csi-node/main/edge_processing.h index 6af25685c1..5c6998172b 100644 --- a/firmware/esp32-csi-node/main/edge_processing.h +++ b/firmware/esp32-csi-node/main/edge_processing.h @@ -27,6 +27,7 @@ /* ---- Magic numbers ---- */ #define EDGE_VITALS_MAGIC 0xC5110002 /**< Vitals packet magic. */ #define EDGE_COMPRESSED_MAGIC 0xC5110005 /**< Compressed frame magic (was 0xC5110003, reassigned for ADR-069). */ +#define EDGE_BATTERY_MAGIC 0xC5110008 /**< Battery status packet magic. */ /* ---- Buffer sizes ---- */ #define EDGE_RING_SLOTS 16 /**< SPSC ring buffer slots (power of 2). */ @@ -152,6 +153,20 @@ typedef struct __attribute__((packed)) { _Static_assert(sizeof(edge_fused_vitals_pkt_t) == 48, "fused vitals must be 48 bytes"); +/* ---- Cardputer battery status packet (16 bytes, wire format) ---- */ +typedef struct __attribute__((packed)) { + uint32_t magic; /**< EDGE_BATTERY_MAGIC = 0xC5110008. */ + uint8_t node_id; + uint8_t percent; /**< 0-100 when valid, 255 when unknown. */ + uint8_t flags; /**< Bit0=valid, Bit1=charging. */ + uint8_t status; /**< battery_power_status_t value. */ + uint16_t millivolts; /**< Pack voltage in mV when valid. */ + uint16_t reserved; + uint32_t timestamp_ms; +} edge_battery_pkt_t; + +_Static_assert(sizeof(edge_battery_pkt_t) == 16, "battery packet must be 16 bytes"); + /* ---- Edge configuration (from NVS) ---- */ typedef struct { uint8_t tier; /**< Processing tier: 0=raw, 1=basic, 2=full. */ diff --git a/firmware/esp32-csi-node/main/esp32cam_dual_stream.c b/firmware/esp32-csi-node/main/esp32cam_dual_stream.c new file mode 100644 index 0000000000..abf3b79a7c --- /dev/null +++ b/firmware/esp32-csi-node/main/esp32cam_dual_stream.c @@ -0,0 +1,250 @@ +#include "esp32cam_dual_stream.h" + +#include +#include + +#include "freertos/FreeRTOS.h" +#include "freertos/task.h" +#include "esp_camera.h" +#include "esp_heap_caps.h" +#include "esp_log.h" +#include "esp_timer.h" +#include "sdkconfig.h" + +static const char *TAG = "esp32cam_dual"; + +/* AI Thinker ESP32-CAM / OV2640 pin map. */ +#define CAM_PIN_D0 5 +#define CAM_PIN_D1 18 +#define CAM_PIN_D2 19 +#define CAM_PIN_D3 21 +#define CAM_PIN_D4 36 +#define CAM_PIN_D5 39 +#define CAM_PIN_D6 34 +#define CAM_PIN_D7 35 +#define CAM_PIN_VSYNC 25 +#define CAM_PIN_HREF 23 +#define CAM_PIN_PCLK 22 + +#define STREAM_BOUNDARY "ruviewframe" + +static bool s_camera_ready; + +static esp_err_t send_unavailable(httpd_req_t *req, const char *message) +{ + httpd_resp_set_status(req, "503 Service Unavailable"); + httpd_resp_set_type(req, "text/plain"); + return httpd_resp_sendstr(req, message); +} + +static framesize_t configured_frame_size(void) +{ +#if CONFIG_ESP32CAM_FRAME_QQVGA + return FRAMESIZE_QQVGA; +#elif CONFIG_ESP32CAM_FRAME_VGA + return FRAMESIZE_VGA; +#else + return FRAMESIZE_QVGA; +#endif +} + +static const char *configured_frame_size_name(void) +{ +#if CONFIG_ESP32CAM_FRAME_QQVGA + return "QQVGA"; +#elif CONFIG_ESP32CAM_FRAME_VGA + return "VGA"; +#else + return "QVGA"; +#endif +} + +static esp_err_t camera_init_once(void) +{ + if (s_camera_ready) { + return ESP_OK; + } + + const bool psram_ready = heap_caps_get_total_size(MALLOC_CAP_SPIRAM) > 0; + camera_config_t config = { + .pin_pwdn = CONFIG_ESP32CAM_PIN_PWDN, + .pin_reset = CONFIG_ESP32CAM_PIN_RESET, + .pin_xclk = CONFIG_ESP32CAM_PIN_XCLK, + .pin_sccb_sda = CONFIG_ESP32CAM_PIN_SIOD, + .pin_sccb_scl = CONFIG_ESP32CAM_PIN_SIOC, + .pin_d7 = CAM_PIN_D7, + .pin_d6 = CAM_PIN_D6, + .pin_d5 = CAM_PIN_D5, + .pin_d4 = CAM_PIN_D4, + .pin_d3 = CAM_PIN_D3, + .pin_d2 = CAM_PIN_D2, + .pin_d1 = CAM_PIN_D1, + .pin_d0 = CAM_PIN_D0, + .pin_vsync = CAM_PIN_VSYNC, + .pin_href = CAM_PIN_HREF, + .pin_pclk = CAM_PIN_PCLK, + .xclk_freq_hz = CONFIG_ESP32CAM_XCLK_FREQ_HZ, + .ledc_timer = LEDC_TIMER_0, + .ledc_channel = LEDC_CHANNEL_0, + .pixel_format = PIXFORMAT_JPEG, + .frame_size = configured_frame_size(), + .jpeg_quality = CONFIG_ESP32CAM_JPEG_QUALITY, + .fb_count = psram_ready ? 2 : 1, + .fb_location = psram_ready ? CAMERA_FB_IN_PSRAM : CAMERA_FB_IN_DRAM, + .grab_mode = CAMERA_GRAB_LATEST, + }; + + esp_err_t ret = esp_camera_init(&config); + if (ret != ESP_OK) { + ESP_LOGE(TAG, "esp_camera_init failed: %s", esp_err_to_name(ret)); + return ret; + } + + sensor_t *sensor = esp_camera_sensor_get(); + if (sensor != NULL) { + sensor->set_quality(sensor, CONFIG_ESP32CAM_JPEG_QUALITY); + sensor->set_framesize(sensor, configured_frame_size()); + } + + s_camera_ready = true; + ESP_LOGI(TAG, + "ESP32-CAM dual stream ready: frame=%s quality=%d fps_cap=%d psram=%s", + configured_frame_size_name(), + CONFIG_ESP32CAM_JPEG_QUALITY, + CONFIG_ESP32CAM_STREAM_FPS, + psram_ready ? "yes" : "no"); + return ESP_OK; +} + +static esp_err_t status_handler(httpd_req_t *req) +{ + char payload[192]; + int len = snprintf(payload, sizeof(payload), + "{\"camera_ready\":%s,\"mode\":\"dual-csi-mjpeg\"," + "\"frame_size\":\"%s\",\"jpeg_quality\":%d," + "\"stream_fps\":%d,\"path\":\"/stream\"}", + s_camera_ready ? "true" : "false", + configured_frame_size_name(), + CONFIG_ESP32CAM_JPEG_QUALITY, + CONFIG_ESP32CAM_STREAM_FPS); + httpd_resp_set_type(req, "application/json"); + return httpd_resp_send(req, payload, len); +} + +static esp_err_t page_handler(httpd_req_t *req) +{ + static const char html[] = + "" + "RuView ESP32-CAM" + "" + "

RuView ESP32-CAM Dual CSI + MJPEG

" + "

CSI UDP stays active. Snapshot: /cam.jpg. " + "Status: /cam/status.

"; + httpd_resp_set_type(req, "text/html; charset=utf-8"); + return httpd_resp_send(req, html, HTTPD_RESP_USE_STRLEN); +} + +static esp_err_t jpg_handler(httpd_req_t *req) +{ + if (!s_camera_ready) { + return send_unavailable(req, "camera not ready"); + } + + camera_fb_t *fb = esp_camera_fb_get(); + if (fb == NULL) { + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "capture failed"); + return ESP_FAIL; + } + + httpd_resp_set_type(req, "image/jpeg"); + httpd_resp_set_hdr(req, "Cache-Control", "no-store"); + esp_err_t ret = httpd_resp_send(req, (const char *)fb->buf, fb->len); + esp_camera_fb_return(fb); + return ret; +} + +static esp_err_t stream_handler(httpd_req_t *req) +{ + if (!s_camera_ready) { + return send_unavailable(req, "camera not ready"); + } + + httpd_resp_set_type(req, "multipart/x-mixed-replace;boundary=" STREAM_BOUNDARY); + httpd_resp_set_hdr(req, "Cache-Control", "no-store"); + + const TickType_t delay_ticks = pdMS_TO_TICKS(1000 / CONFIG_ESP32CAM_STREAM_FPS); + char header[96]; + while (true) { + camera_fb_t *fb = esp_camera_fb_get(); + if (fb == NULL) { + ESP_LOGW(TAG, "stream capture failed"); + vTaskDelay(pdMS_TO_TICKS(250)); + continue; + } + + int header_len = snprintf(header, sizeof(header), + "\r\n--" STREAM_BOUNDARY + "\r\nContent-Type: image/jpeg" + "\r\nContent-Length: %u\r\n\r\n", + (unsigned)fb->len); + esp_err_t ret = httpd_resp_send_chunk(req, header, header_len); + if (ret == ESP_OK) { + ret = httpd_resp_send_chunk(req, (const char *)fb->buf, fb->len); + } + esp_camera_fb_return(fb); + if (ret != ESP_OK) { + break; + } + vTaskDelay(delay_ticks); + } + + httpd_resp_send_chunk(req, NULL, 0); + return ESP_OK; +} + +esp_err_t esp32cam_dual_stream_register(httpd_handle_t server) +{ + if (server == NULL) { + return ESP_ERR_INVALID_ARG; + } + + esp_err_t ret = camera_init_once(); + if (ret != ESP_OK) { + return ret; + } + + const httpd_uri_t page_uri = { + .uri = "/cam", + .method = HTTP_GET, + .handler = page_handler, + .user_ctx = NULL, + }; + const httpd_uri_t jpg_uri = { + .uri = "/cam.jpg", + .method = HTTP_GET, + .handler = jpg_handler, + .user_ctx = NULL, + }; + const httpd_uri_t stream_uri = { + .uri = "/stream", + .method = HTTP_GET, + .handler = stream_handler, + .user_ctx = NULL, + }; + const httpd_uri_t status_uri = { + .uri = "/cam/status", + .method = HTTP_GET, + .handler = status_handler, + .user_ctx = NULL, + }; + + ESP_ERROR_CHECK(httpd_register_uri_handler(server, &page_uri)); + ESP_ERROR_CHECK(httpd_register_uri_handler(server, &jpg_uri)); + ESP_ERROR_CHECK(httpd_register_uri_handler(server, &stream_uri)); + ESP_ERROR_CHECK(httpd_register_uri_handler(server, &status_uri)); + + ESP_LOGI(TAG, "Camera endpoints registered: /cam /cam.jpg /stream /cam/status"); + return ESP_OK; +} diff --git a/firmware/esp32-csi-node/main/esp32cam_dual_stream.h b/firmware/esp32-csi-node/main/esp32cam_dual_stream.h new file mode 100644 index 0000000000..29509be3bb --- /dev/null +++ b/firmware/esp32-csi-node/main/esp32cam_dual_stream.h @@ -0,0 +1,9 @@ +#ifndef ESP32CAM_DUAL_STREAM_H +#define ESP32CAM_DUAL_STREAM_H + +#include "esp_err.h" +#include "esp_http_server.h" + +esp_err_t esp32cam_dual_stream_register(httpd_handle_t server); + +#endif /* ESP32CAM_DUAL_STREAM_H */ diff --git a/firmware/esp32-csi-node/main/idf_component.yml b/firmware/esp32-csi-node/main/idf_component.yml index 4ec1d552ae..12987a0a04 100644 --- a/firmware/esp32-csi-node/main/idf_component.yml +++ b/firmware/esp32-csi-node/main/idf_component.yml @@ -11,3 +11,6 @@ dependencies: ## Onboard WS2812 LED Disabling espressif/led_strip: "^3.0.0" + + ## ESP32-CAM OV2640 capture for opt-in dual CSI+MJPEG firmware + espressif/esp32-camera: "^2.0.15" diff --git a/firmware/esp32-csi-node/main/lv_conf.h b/firmware/esp32-csi-node/main/lv_conf.h index d839c38c0a..ca6f1db213 100644 --- a/firmware/esp32-csi-node/main/lv_conf.h +++ b/firmware/esp32-csi-node/main/lv_conf.h @@ -2,8 +2,8 @@ * @file lv_conf.h * @brief LVGL compile-time configuration for ESP32-S3 AMOLED display (ADR-045). * - * Tuned for RM67162 536x240 QSPI AMOLED with 8MB PSRAM. - * Color depth: RGB565 (16-bit) for QSPI bandwidth. + * Tuned for Cardputer-Adv ST7789V2 240x135 LCD. + * Color depth: RGB565 (16-bit). * Double-buffered in SPIRAM, 30fps target. */ @@ -14,7 +14,7 @@ /* ---- Core ---- */ #define LV_COLOR_DEPTH 16 -#define LV_COLOR_16_SWAP 1 /* Byte-swap for SPI/QSPI displays */ +#define LV_COLOR_16_SWAP 0 /* ESP-IDF ST7789 SPI panel consumes native RGB565 words */ #define LV_MEM_CUSTOM 1 /* Use ESP-IDF heap instead of LVGL's internal allocator */ #define LV_MEM_CUSTOM_INCLUDE #define LV_MEM_CUSTOM_ALLOC malloc @@ -22,8 +22,8 @@ #define LV_MEM_CUSTOM_REALLOC realloc /* ---- Display ---- */ -#define LV_HOR_RES_MAX 368 -#define LV_VER_RES_MAX 448 +#define LV_HOR_RES_MAX 240 +#define LV_VER_RES_MAX 135 #define LV_DPI_DEF 200 /* ---- Tick (provided by esp_timer in display_task.c) ---- */ @@ -38,6 +38,8 @@ #define LV_IMG_CACHE_DEF_SIZE 0 /* ---- Fonts ---- */ +#define LV_FONT_MONTSERRAT_10 1 +#define LV_FONT_MONTSERRAT_12 1 #define LV_FONT_MONTSERRAT_14 1 #define LV_FONT_MONTSERRAT_20 1 #define LV_FONT_DEFAULT &lv_font_montserrat_14 @@ -79,7 +81,7 @@ #define LV_USE_ASSERT_MALLOC 1 /* ---- GPU / render ---- */ -#define LV_USE_GPU_ESP32_S3 0 /* No parallel LCD interface — we use QSPI */ +#define LV_USE_GPU_ESP32_S3 0 /* ---- Animation ---- */ #define LV_USE_ANIM 1 diff --git a/firmware/esp32-csi-node/main/main.c b/firmware/esp32-csi-node/main/main.c index ef57689087..745e9b2e84 100644 --- a/firmware/esp32-csi-node/main/main.c +++ b/firmware/esp32-csi-node/main/main.c @@ -29,8 +29,10 @@ #include "wasm_runtime.h" #include "wasm_upload.h" #include "display_task.h" +#include "cardputer_adv_audio.h" #include "mmwave_sensor.h" #include "swarm_bridge.h" +#include "esp32cam_dual_stream.h" #include "rv_radio_ops.h" /* ADR-081 Layer 1 — Radio Abstraction Layer. */ #include "adaptive_controller.h" /* ADR-081 Layer 2 — Adaptive controller. */ #include "c6_twt.h" /* ADR-110: TWT (no-op stub on S3) */ @@ -47,7 +49,9 @@ static const char *TAG = "main"; /* ADR-040: WASM timer handle (calls on_timer at configurable interval). */ +#if defined(CONFIG_WASM_ENABLE) static esp_timer_handle_t s_wasm_timer; +#endif /* CONFIG_WASM_ENABLE */ /* Runtime configuration (loaded from NVS or Kconfig defaults). * Global so other modules (wasm_upload.c) can access pubkey, etc. */ @@ -173,14 +177,12 @@ void app_main(void) ESP_LOGI(TAG, "%s CSI Node (ADR-018 / ADR-110) — v%s — Node ID: %d", target_name, app_desc->version, g_nvs_config.node_id); +#if defined(CONFIG_IDF_TARGET_ESP32S3) /* Turn off onboard WS2812 LED. - * S3 dev boards put the LED on GPIO 38; C6 dev boards on GPIO 8. - * On C6, GPIO 38 doesn't exist (only 0-30) — gate the init by target. */ -#if defined(CONFIG_IDF_TARGET_ESP32C6) - const int led_gpio = 8; -#else + * S3 dev boards put the LED on GPIO 38. + * ESP32-PICO must not touch GPIO38/RMT here. */ const int led_gpio = 38; -#endif + led_strip_handle_t led_strip; led_strip_config_t strip_config = { .strip_gpio_num = led_gpio, @@ -196,6 +198,26 @@ void app_main(void) if (led_strip_new_rmt_device(&strip_config, &rmt_config, &led_strip) == ESP_OK) { led_strip_clear(led_strip); } +#endif /* CONFIG_IDF_TARGET_ESP32S3 */ + + /* + * Start the display heartbeat before network bring-up. wifi_init_sta() + * can wait for a connection for a long time; the LCD should still boot, + * animate, and show a heartbeat while WiFi/UDP/edge processing come online. + */ +#ifdef CONFIG_DISPLAY_ENABLE + esp_err_t disp_ret = display_task_start(); + if (disp_ret != ESP_OK) { + ESP_LOGW(TAG, "Display init returned: %s", esp_err_to_name(disp_ret)); + } +#endif + +#if defined(CONFIG_CARDPUTER_ADV_AUDIO_ENABLE) + esp_err_t audio_ret = cardputer_adv_audio_startup_probe(); + if (audio_ret != ESP_OK) { + ESP_LOGW(TAG, "Cardputer-Adv audio proof returned: %s", esp_err_to_name(audio_ret)); + } +#endif /* ADR-110 P4: 802.15.4 mesh time-sync (C6 only). * Initialized BEFORE WiFi so it's available even when WiFi STA can't @@ -272,11 +294,17 @@ void app_main(void) * both S3 and C6 — replaces the broken 802.15.4 RX path in c6_timesync. * Skip on QEMU mock (no real WiFi → no ESP-NOW). */ #ifndef CONFIG_CSI_MOCK_SKIP_WIFI_CONNECT +#if defined(CONFIG_IDF_TARGET_ESP32) + /* ESP-NOW sync disabled on classic ESP32-PICO baseline. + * This isolates TGx WDT resets seen immediately after c6_espnow leader step-down. */ + ESP_LOGW(TAG, "c6_sync_espnow disabled on ESP32-PICO baseline"); +#else esp_err_t espnow_ret = c6_sync_espnow_init(); if (espnow_ret != ESP_OK) { ESP_LOGW(TAG, "c6_sync_espnow_init failed: %s (continuing without ESP-NOW sync)", esp_err_to_name(espnow_ret)); } +#endif /* CONFIG_IDF_TARGET_ESP32 */ #endif /* ADR-039: Initialize edge processing pipeline. */ @@ -302,21 +330,38 @@ void app_main(void) if (ota_ret != ESP_OK) { ESP_LOGW(TAG, "OTA server init failed: %s", esp_err_to_name(ota_ret)); } +#if defined(CONFIG_ESP32CAM_DUAL_FIRMWARE) + esp_err_t cam_ret = ESP_ERR_INVALID_STATE; + if (ota_server != NULL) { + cam_ret = esp32cam_dual_stream_register(ota_server); + if (cam_ret != ESP_OK) { + ESP_LOGW(TAG, "ESP32-CAM dual stream init failed: %s", esp_err_to_name(cam_ret)); + } + } +#endif #else esp_err_t ota_ret = ESP_ERR_NOT_SUPPORTED; ESP_LOGI(TAG, "Mock CSI mode: skipping OTA server (no network)"); #endif + const char *wasm_status = "off"; + /* ADR-040: Initialize WASM programmable sensing runtime. */ + /* WASM init/upload/timer only exists when WASM is compiled in. */ +#if defined(CONFIG_WASM_ENABLE) esp_err_t wasm_ret = wasm_runtime_init(); if (wasm_ret != ESP_OK) { ESP_LOGW(TAG, "WASM runtime init failed: %s", esp_err_to_name(wasm_ret)); + wasm_status = "off"; } else { + wasm_status = "ready"; /* Register WASM upload endpoints on the OTA HTTP server. */ if (ota_server != NULL) { wasm_upload_register(ota_server); } + /* WASM timer only exists when WASM is compiled in. */ +#if defined(CONFIG_WASM_ENABLE) /* Start periodic timer for wasm_runtime_on_timer(). */ esp_timer_create_args_t timer_args = { .callback = (void (*)(void *))wasm_runtime_on_timer, @@ -337,8 +382,18 @@ void app_main(void) } else { ESP_LOGW(TAG, "WASM timer create failed: %s", esp_err_to_name(timer_ret)); } +#endif /* CONFIG_WASM_ENABLE */ } - +#endif /* CONFIG_WASM_ENABLE */ + +#if defined(CONFIG_IDF_TARGET_ESP32) + /* Classic ESP32 StickC/PICO nodes are CSI-only in this build. Do not probe + * the mmWave UART here: on the StickC Plus recovery path the watchdog reset + * happens immediately after network bring-up, before a clean no-sensor + * startup can complete. S3/ADV keeps the full auto-detect path below. */ + esp_err_t mmwave_ret = ESP_ERR_NOT_SUPPORTED; + ESP_LOGI(TAG, "mmWave probe disabled on ESP32-PICO baseline (CSI-only mode)"); +#else /* ADR-063: Initialize mmWave sensor (auto-detect on UART). */ esp_err_t mmwave_ret = mmwave_sensor_init(-1, -1); /* -1 = use default GPIO pins */ if (mmwave_ret == ESP_OK) { @@ -350,6 +405,7 @@ void app_main(void) } else { ESP_LOGI(TAG, "No mmWave sensor detected (CSI-only mode)"); } +#endif /* ADR-066: Initialize swarm bridge to Cognitum Seed (if configured). */ esp_err_t swarm_ret = ESP_ERR_INVALID_ARG; @@ -402,19 +458,11 @@ void app_main(void) /* Initialize power management. */ power_mgmt_init(g_nvs_config.power_duty); - /* ADR-045: Start AMOLED display task (gracefully skips if no display). */ -#ifdef CONFIG_DISPLAY_ENABLE - esp_err_t disp_ret = display_task_start(); - if (disp_ret != ESP_OK) { - ESP_LOGW(TAG, "Display init returned: %s", esp_err_to_name(disp_ret)); - } -#endif - ESP_LOGI(TAG, "CSI streaming active → %s:%d (edge_tier=%u, OTA=%s, WASM=%s, mmWave=%s, swarm=%s, adapt=%s)", g_nvs_config.target_ip, g_nvs_config.target_port, g_nvs_config.edge_tier, (ota_ret == ESP_OK) ? "ready" : "off", - (wasm_ret == ESP_OK) ? "ready" : "off", + wasm_status, (mmwave_ret == ESP_OK) ? "active" : "off", (swarm_ret == ESP_OK) ? g_nvs_config.seed_url : "off", (adapt_ret == ESP_OK) ? "on" : "off"); diff --git a/firmware/esp32-csi-node/main/ota_update.c b/firmware/esp32-csi-node/main/ota_update.c index eed5c667f4..97d00715b1 100644 --- a/firmware/esp32-csi-node/main/ota_update.c +++ b/firmware/esp32-csi-node/main/ota_update.c @@ -10,6 +10,8 @@ #include "ota_update.h" +#include +#include #include #include "esp_log.h" #include "esp_ota_ops.h" @@ -17,14 +19,17 @@ #include "esp_app_desc.h" #include "nvs_flash.h" #include "nvs.h" +#include "battery_monitor.h" +#include "csi_collector.h" +#include "edge_processing.h" static const char *TAG = "ota_update"; /** OTA HTTP server port. */ #define OTA_PORT 8032 -/** Maximum firmware size (900 KB — matches CI binary size gate). */ -#define OTA_MAX_SIZE (900 * 1024) +/** Number of samples kept on the web graph. */ +#define GRAPH_HISTORY_SAMPLES 96 /** NVS namespace and key for the OTA pre-shared key. */ #define OTA_NVS_NAMESPACE "security" @@ -36,6 +41,273 @@ static const char *TAG = "ota_update"; /** Cached PSK loaded from NVS at init time. Empty = auth disabled. */ static char s_ota_psk[OTA_PSK_MAX_LEN] = {0}; +static const char GRAPH_HTML[] = R"rawliteral( + + + + + +RuView Live Graph + + + +
+
+
+
RuView Live Graph
+
Browser view of the same live edge vitals that drive the S3 display graph.
+
+
WAITING
+
+ +
+
Motion
--%
+
Presence
--%
+
RSSI
-- dBm
+
People
--
+
Battery
--
+
+ +
+ +
+ +
+ Breathing: -- BPM + Heart: -- BPM + Power: -- + Updated: -- +
+
+ + + + +)rawliteral"; + +static int clamp_int(int value, int lo, int hi) +{ + if (value < lo) return lo; + if (value > hi) return hi; + return value; +} + +static int scaled_motion_percent(const edge_vitals_pkt_t *vitals) +{ + if (!vitals) return 0; + return clamp_int((int)(vitals->motion_energy * 18.0f), 0, 100); +} + +static int scaled_presence_percent(const edge_vitals_pkt_t *vitals) +{ + if (!vitals) return 0; + return clamp_int((int)(vitals->presence_score * 18.0f), 0, 100); +} + +static void build_graph_snapshot(char *response, size_t response_len) +{ + edge_vitals_pkt_t vitals; + bool has_vitals = edge_get_vitals(&vitals); + + battery_status_t battery = {0}; + esp_err_t battery_ret = battery_monitor_read(&battery); + bool battery_live = (battery_ret == ESP_OK && battery.valid); + + int motion = has_vitals ? scaled_motion_percent(&vitals) : 0; + int presence = has_vitals ? scaled_presence_percent(&vitals) : 0; + int breathing = has_vitals ? (int)(vitals.breathing_rate / 100U) : 0; + int heartrate = has_vitals ? (int)(vitals.heartrate / 10000U) : 0; + int rssi = has_vitals ? (int)vitals.rssi : 0; + int persons = has_vitals ? (int)vitals.n_persons : 0; + int battery_percent = battery_live ? (int)battery.percent : -1; + int battery_mv = battery_live ? (int)battery.millivolts : -1; + const char *battery_status = battery_live ? battery_monitor_status_name(battery.status) : "UNKNOWN"; + uint32_t timestamp_ms = has_vitals ? vitals.timestamp_ms : 0; + uint8_t node_id = has_vitals ? vitals.node_id : csi_collector_get_node_id(); + + snprintf(response, response_len, + "{\"live\":%s,\"node_id\":%u,\"motion\":%d,\"presence\":%d," + "\"breathing_bpm\":%d,\"heartrate_bpm\":%d,\"rssi\":%d,\"persons\":%d," + "\"battery_percent\":%d,\"battery_mv\":%d,\"battery_status\":\"%s\"," + "\"timestamp_ms\":%lu}", + has_vitals ? "true" : "false", + (unsigned)node_id, + motion, presence, + breathing, heartrate, rssi, persons, + battery_percent, battery_mv, battery_status, + (unsigned long)timestamp_ms); +} + +static esp_err_t graph_page_handler(httpd_req_t *req) +{ + httpd_resp_set_type(req, "text/html; charset=utf-8"); + httpd_resp_send(req, GRAPH_HTML, HTTPD_RESP_USE_STRLEN); + return ESP_OK; +} + +static esp_err_t graph_data_handler(httpd_req_t *req) +{ + char response[384]; + build_graph_snapshot(response, sizeof(response)); + httpd_resp_set_type(req, "application/json"); + httpd_resp_send(req, response, HTTPD_RESP_USE_STRLEN); + return ESP_OK; +} + /** * ADR-050: Verify the Authorization header contains the correct PSK. * Returns true only when a PSK is provisioned AND the Bearer token @@ -95,11 +367,11 @@ static esp_err_t ota_status_handler(httpd_req_t *req) int len = snprintf(response, sizeof(response), "{\"version\":\"%s\",\"date\":\"%s\",\"time\":\"%s\"," "\"running_partition\":\"%s\",\"next_partition\":\"%s\"," - "\"max_size\":%d}", + "\"max_size\":%lu}", app->version, app->date, app->time, running ? running->label : "unknown", update ? update->label : "none", - OTA_MAX_SIZE); + (unsigned long)(update ? update->size : 0)); httpd_resp_set_type(req, "application/json"); httpd_resp_send(req, response, len); @@ -121,12 +393,6 @@ static esp_err_t ota_upload_handler(httpd_req_t *req) ESP_LOGI(TAG, "OTA update started, content_length=%d", req->content_len); - if (req->content_len <= 0 || req->content_len > OTA_MAX_SIZE) { - httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, - "Invalid firmware size (must be 1B - 900KB)"); - return ESP_FAIL; - } - const esp_partition_t *update_partition = esp_ota_get_next_update_partition(NULL); if (update_partition == NULL) { httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, @@ -134,6 +400,15 @@ static esp_err_t ota_upload_handler(httpd_req_t *req) return ESP_FAIL; } + if (req->content_len <= 0 || req->content_len > (int)update_partition->size) { + char err_msg[96]; + snprintf(err_msg, sizeof(err_msg), + "Invalid firmware size (must be 1B - %luB)", + (unsigned long)update_partition->size); + httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, err_msg); + return ESP_FAIL; + } + esp_ota_handle_t ota_handle; esp_err_t err = esp_ota_begin(update_partition, OTA_WITH_SEQUENTIAL_WRITES, &ota_handle); if (err != ESP_OK) { @@ -214,9 +489,17 @@ static esp_err_t ota_start_server(httpd_handle_t *out_handle) { httpd_config_t config = HTTPD_DEFAULT_CONFIG(); config.server_port = OTA_PORT; - config.max_uri_handlers = 12; /* Extra slots for WASM endpoints (ADR-040). */ - /* Increase receive timeout for large uploads. */ + config.max_uri_handlers = 16; /* Extra slots for WASM + ESP32-CAM dual endpoints. */ + /* + * OTA commits validate a >1 MB app image after the request body has been + * received. The HTTPD default task stack is tight for that path on S3 and + * can reset the connection before esp_ota_set_boot_partition() runs. Give + * the handler enough stack and keep the socket alive while validation and + * the final JSON response complete. + */ + config.stack_size = 8192; config.recv_wait_timeout = 30; + config.send_wait_timeout = 30; httpd_handle_t server = NULL; esp_err_t err = httpd_start(&server, &config); @@ -235,6 +518,30 @@ static esp_err_t ota_start_server(httpd_handle_t *out_handle) }; httpd_register_uri_handler(server, &status_uri); + httpd_uri_t graph_page_uri = { + .uri = "/", + .method = HTTP_GET, + .handler = graph_page_handler, + .user_ctx = NULL, + }; + httpd_register_uri_handler(server, &graph_page_uri); + + httpd_uri_t graph_alias_uri = { + .uri = "/graph", + .method = HTTP_GET, + .handler = graph_page_handler, + .user_ctx = NULL, + }; + httpd_register_uri_handler(server, &graph_alias_uri); + + httpd_uri_t graph_data_uri = { + .uri = "/graph/data", + .method = HTTP_GET, + .handler = graph_data_handler, + .user_ctx = NULL, + }; + httpd_register_uri_handler(server, &graph_data_uri); + httpd_uri_t upload_uri = { .uri = "/ota", .method = HTTP_POST, @@ -244,6 +551,9 @@ static esp_err_t ota_start_server(httpd_handle_t *out_handle) httpd_register_uri_handler(server, &upload_uri); ESP_LOGI(TAG, "OTA HTTP server started on port %d", OTA_PORT); + ESP_LOGI(TAG, " GET / — live graph page"); + ESP_LOGI(TAG, " GET /graph — live graph page alias"); + ESP_LOGI(TAG, " GET /graph/data — graph telemetry JSON"); ESP_LOGI(TAG, " GET /ota/status — firmware version info"); ESP_LOGI(TAG, " POST /ota — upload new firmware binary"); diff --git a/firmware/esp32-csi-node/ota_flash.py b/firmware/esp32-csi-node/ota_flash.py new file mode 100755 index 0000000000..5bfa0e21b2 --- /dev/null +++ b/firmware/esp32-csi-node/ota_flash.py @@ -0,0 +1,141 @@ +#!/usr/bin/env python3 +"""Upload a RuView ESP32 app image through the node's HTTP OTA endpoint.""" + +from __future__ import annotations + +import argparse +import json +import os +import subprocess +import sys +from pathlib import Path + + +ROOT = Path(__file__).resolve().parent +DEFAULT_IMAGE = ROOT / "build" / "esp32-csi-node.bin" + + +def run_curl(args: list[str], timeout: int) -> subprocess.CompletedProcess[str]: + return subprocess.run( + ["curl", *args], + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + timeout=timeout, + check=False, + ) + + +def curl_common(host: str, interface: str | None, timeout: int) -> list[str]: + args = ["--fail-with-body", "--show-error", "--silent", "--max-time", str(timeout)] + if interface: + args.extend(["--interface", interface]) + return args + + +def get_status(host: str, interface: str | None, timeout: int) -> dict[str, object]: + url = f"http://{host}:8032/ota/status" + result = run_curl([*curl_common(host, interface, timeout), url], timeout + 2) + if result.returncode != 0: + raise RuntimeError( + f"status failed for {url}: {result.stderr.strip() or result.stdout.strip()}" + ) + try: + return json.loads(result.stdout) + except json.JSONDecodeError as exc: + raise RuntimeError(f"status returned non-JSON: {result.stdout[:200]!r}") from exc + + +def upload(host: str, image: Path, token: str, interface: str | None, timeout: int) -> str: + url = f"http://{host}:8032/ota" + result = run_curl( + [ + *curl_common(host, interface, timeout), + "--request", + "POST", + "--header", + f"Authorization: Bearer {token}", + "--header", + "Content-Type: application/octet-stream", + "--data-binary", + f"@{image}", + url, + ], + timeout + 5, + ) + if result.returncode != 0: + raise RuntimeError( + f"upload failed for {url}: {result.stderr.strip() or result.stdout.strip()}" + ) + return result.stdout.strip() + + +def parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser( + description="Flash a running RuView node over HTTP OTA on port 8032." + ) + parser.add_argument("host", help="Node IP address or hostname, for example 192.168.1.161") + parser.add_argument( + "--image", + type=Path, + default=DEFAULT_IMAGE, + help=f"RuView app image to upload (default: {DEFAULT_IMAGE})", + ) + parser.add_argument( + "--psk", + default=os.environ.get("RUVIEW_OTA_PSK"), + help="OTA bearer token. Defaults to RUVIEW_OTA_PSK.", + ) + parser.add_argument( + "--interface", + default=os.environ.get("RUVIEW_OTA_IFACE", "wlan0"), + help="Network interface for curl, or empty string to let routing choose (default: wlan0).", + ) + parser.add_argument("--timeout", type=int, default=30, help="curl timeout in seconds") + parser.add_argument( + "--status-only", + action="store_true", + help="Only print /ota/status; do not upload firmware.", + ) + return parser.parse_args() + + +def main() -> int: + args = parse_args() + image = args.image.expanduser().resolve() + interface = args.interface or None + + try: + status = get_status(args.host, interface, args.timeout) + print(json.dumps(status, indent=2, sort_keys=True)) + + if args.status_only: + return 0 + + if not args.psk: + print("error: --psk or RUVIEW_OTA_PSK is required for upload", file=sys.stderr) + return 2 + if not image.is_file(): + print(f"error: image not found: {image}", file=sys.stderr) + return 2 + + max_size = int(status.get("max_size") or 0) + image_size = image.stat().st_size + if max_size and image_size > max_size: + print( + f"error: image is too large for next OTA partition " + f"({image_size} > {max_size} bytes)", + file=sys.stderr, + ) + return 2 + + print(f"uploading {image} ({image_size} bytes) to {args.host}:8032 ...") + print(upload(args.host, image, args.psk, interface, args.timeout)) + return 0 + except (RuntimeError, subprocess.TimeoutExpired) as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/firmware/esp32-csi-node/provision.py b/firmware/esp32-csi-node/provision.py index d87ccd0446..830b994043 100644 --- a/firmware/esp32-csi-node/provision.py +++ b/firmware/esp32-csi-node/provision.py @@ -79,6 +79,7 @@ ("zone", lambda value: value is not None), ("swarm_hb", lambda value: value is not None), ("swarm_ingest", lambda value: value is not None), + ("ota_psk", bool), ] @@ -108,6 +109,7 @@ def has_config_value(args): "channel", "filter_mac", "hop_channels", "hop_dwell", "seed_url", "seed_token", "zone", "swarm_hb", "swarm_ingest", + "ota_psk", ] @@ -234,6 +236,9 @@ def build_nvs_csv(args): writer.writerow(["swarm_hb", "data", "u16", str(args.swarm_hb)]) if args.swarm_ingest is not None: writer.writerow(["swarm_ingest", "data", "u16", str(args.swarm_ingest)]) + if args.ota_psk: + writer.writerow(["security", "namespace", "", ""]) + writer.writerow(["ota_psk", "data", "string", args.ota_psk]) return buf.getvalue() @@ -352,6 +357,8 @@ def main(): parser.add_argument("--zone", type=str, help="Zone name for this node (e.g. lobby, hallway)") parser.add_argument("--swarm-hb", type=int, help="Swarm heartbeat interval in seconds (default 30)") parser.add_argument("--swarm-ingest", type=int, help="Swarm vector ingest interval in seconds (default 5)") + parser.add_argument("--ota-psk", type=str, + help="Provision OTA upload bearer token in NVS security/ota_psk") parser.add_argument("--dry-run", action="store_true", help="Generate NVS binary but don't flash") parser.add_argument("--force-partial", action="store_true", help="[deprecated since #391/#574] Suppress the missing-WiFi-trio " @@ -476,6 +483,8 @@ def main(): print(f" Swarm HB: {args.swarm_hb}s") if args.swarm_ingest is not None: print(f" Swarm Ingest: {args.swarm_ingest}s") + if args.ota_psk: + print(" OTA PSK: (set)") csv_content = build_nvs_csv(args) diff --git a/firmware/esp32-csi-node/sdkconfig.defaults b/firmware/esp32-csi-node/sdkconfig.defaults index 9ba4494b56..35e8761854 100644 --- a/firmware/esp32-csi-node/sdkconfig.defaults +++ b/firmware/esp32-csi-node/sdkconfig.defaults @@ -35,6 +35,18 @@ CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 # Extra WiFi IRAM placement (defense-in-depth for RuView#396 SPI cache race) CONFIG_ESP_WIFI_EXTRA_IRAM_OPT=y +# Cardputer-Adv speaker + SD WAV proof. +CONFIG_CARDPUTER_ADV_AUDIO_ENABLE=y +CONFIG_CARDPUTER_ADV_AUDIO_BOOT_CHIME=y +CONFIG_CARDPUTER_ADV_AUDIO_TRY_SD_WAV=y +CONFIG_CARDPUTER_ADV_AUDIO_SD_MOUNT_POINT="/sdcard" +CONFIG_CARDPUTER_ADV_AUDIO_SD_WAV_PATH="/sdcard/ruview.wav" +CONFIG_CARDPUTER_ADV_AUDIO_VOLUME_PERCENT=35 +CONFIG_CARDPUTER_ADV_TRAUTONIUM_ENABLE=y +CONFIG_CARDPUTER_ADV_TRAUTONIUM_PORTAMENTO_MS=220 +CONFIG_CARDPUTER_ADV_TRAUTONIUM_BASE_FREQ_HZ=131 +CONFIG_CARDPUTER_ADV_TRAUTONIUM_LEVEL_PERCENT=58 + # ADR-081: adaptive_controller runs emit_feature_state + stream_sender # network I/O inside Timer Svc callbacks, exceeding the 2 KiB default. # Without this, the device bootloops with diff --git a/firmware/esp32-csi-node/sdkconfig.defaults.esp32cam b/firmware/esp32-csi-node/sdkconfig.defaults.esp32cam new file mode 100644 index 0000000000..8b061dc747 --- /dev/null +++ b/firmware/esp32-csi-node/sdkconfig.defaults.esp32cam @@ -0,0 +1,61 @@ +# RuView ESP32-CAM CSI Node — classic ESP32 / AI Thinker style boards. +# +# Build: +# idf.py -B build-esp32cam \ +# -D SDKCONFIG=build-esp32cam/sdkconfig \ +# -D SDKCONFIG_DEFAULTS=sdkconfig.defaults.esp32cam \ +# set-target esp32 build +# +# Flash: +# python -m esptool --chip esp32 --port /dev/ttyUSB0 --baud 460800 \ +# write_flash --flash_mode dio --flash_size 4MB \ +# 0x1000 build-esp32cam/bootloader/bootloader.bin \ +# 0x8000 build-esp32cam/partition_table/partition-table.bin \ +# 0xf000 build-esp32cam/ota_data_initial.bin \ +# 0x20000 build-esp32cam/esp32-csi-node.bin + +CONFIG_IDF_TARGET="esp32" + +# Most ESP32-CAM modules ship with 4 MB flash. Keep OTA slots but use the +# smaller partition table so the same RuView OTA/provision flow still works. +CONFIG_PARTITION_TABLE_CUSTOM=y +CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_4mb.csv" +CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y +CONFIG_ESPTOOLPY_FLASHSIZE="4MB" +CONFIG_ESPTOOLPY_FLASHMODE_DIO=y +CONFIG_ESPTOOLPY_FLASHMODE="dio" + +# Real RF sensing path. +CONFIG_ESP_WIFI_CSI_ENABLED=y +# CONFIG_CSI_MOCK_ENABLED is not set + +# ESP32-CAM has no RuView display, battery monitor, Cardputer audio, mmWave +# add-on, or WASM memory budget in this profile. The camera module is left idle; +# this firmware uses the board as a WiFi CSI node. +# CONFIG_DISPLAY_ENABLE is not set +# CONFIG_BATTERY_MONITOR_ENABLE is not set +# CONFIG_WASM_ENABLE is not set + +# Defaults are only fallbacks. provision.py writes SSID, target, channel, and +# node ID to NVS after flashing. +CONFIG_CSI_NODE_ID=2 +CONFIG_CSI_TARGET_IP="192.168.1.100" +CONFIG_CSI_TARGET_PORT=5005 +CONFIG_CSI_WIFI_SSID="wifi-densepose" +CONFIG_CSI_WIFI_PASSWORD="" +CONFIG_CSI_WIFI_CHANNEL=6 + +# Keep the edge feature/vitals packets that the desktop UI already understands. +CONFIG_EDGE_TIER=2 +CONFIG_EDGE_VITAL_INTERVAL_MS=1000 +CONFIG_EDGE_TOP_K=8 +CONFIG_EDGE_FALL_THRESH=15000 +CONFIG_EDGE_POWER_DUTY=100 + +# Stack sizing needed by the shared adaptive-controller/network callback path. +CONFIG_COMPILER_OPTIMIZATION_SIZE=y +CONFIG_BOOTLOADER_LOG_LEVEL_WARN=y +CONFIG_LOG_DEFAULT_LEVEL_INFO=y +CONFIG_LWIP_SO_RCVBUF=y +CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 +CONFIG_FREERTOS_TIMER_TASK_STACK_DEPTH=8192 diff --git a/firmware/esp32-csi-node/sdkconfig.defaults.esp32cam-dual b/firmware/esp32-csi-node/sdkconfig.defaults.esp32cam-dual new file mode 100644 index 0000000000..746432d6f3 --- /dev/null +++ b/firmware/esp32-csi-node/sdkconfig.defaults.esp32cam-dual @@ -0,0 +1,67 @@ +# RuView ESP32-CAM dual firmware — classic ESP32 / AI Thinker style boards. +# Keeps RuView CSI UDP telemetry and exposes OV2640 MJPEG endpoints on port 8032: +# http://:8032/cam +# http://:8032/cam.jpg +# http://:8032/stream +# http://:8032/cam/status +# +# Build: +# idf.py -B build-esp32cam-dual \ +# -D SDKCONFIG=build-esp32cam-dual/sdkconfig \ +# -D SDKCONFIG_DEFAULTS=sdkconfig.defaults.esp32cam-dual \ +# set-target esp32 build + +CONFIG_IDF_TARGET="esp32" + +CONFIG_PARTITION_TABLE_CUSTOM=y +CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_4mb.csv" +CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y +CONFIG_ESPTOOLPY_FLASHSIZE="4MB" +CONFIG_ESPTOOLPY_FLASHMODE_DIO=y +CONFIG_ESPTOOLPY_FLASHMODE="dio" + +CONFIG_ESP_WIFI_CSI_ENABLED=y +# CONFIG_CSI_MOCK_ENABLED is not set + +# AI Thinker ESP32-CAM modules normally include PSRAM. Keep booting if a clone +# lacks PSRAM; the camera module falls back to a single DRAM frame buffer. +CONFIG_SPIRAM=y +CONFIG_SPIRAM_MODE_QUAD=y +CONFIG_SPIRAM_TYPE_AUTO=y +CONFIG_SPIRAM_SPEED_40M=y +CONFIG_SPIRAM_BOOT_INIT=y +CONFIG_SPIRAM_IGNORE_NOTFOUND=y +CONFIG_SPIRAM_USE_MALLOC=y +CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=16384 +CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768 + +# No display/audio/WASM on ESP32-CAM; preserve RAM for WiFi CSI + camera. +# CONFIG_DISPLAY_ENABLE is not set +# CONFIG_BATTERY_MONITOR_ENABLE is not set +# CONFIG_WASM_ENABLE is not set + +CONFIG_ESP32CAM_DUAL_FIRMWARE=y +CONFIG_ESP32CAM_STREAM_FPS=4 +CONFIG_ESP32CAM_FRAME_QVGA=y +CONFIG_ESP32CAM_JPEG_QUALITY=14 +CONFIG_ESP32CAM_XCLK_FREQ_HZ=10000000 + +CONFIG_CSI_NODE_ID=3 +CONFIG_CSI_TARGET_IP="192.168.1.100" +CONFIG_CSI_TARGET_PORT=5005 +CONFIG_CSI_WIFI_SSID="wifi-densepose" +CONFIG_CSI_WIFI_PASSWORD="" +CONFIG_CSI_WIFI_CHANNEL=6 + +CONFIG_EDGE_TIER=2 +CONFIG_EDGE_VITAL_INTERVAL_MS=1000 +CONFIG_EDGE_TOP_K=8 +CONFIG_EDGE_FALL_THRESH=15000 +CONFIG_EDGE_POWER_DUTY=100 + +CONFIG_COMPILER_OPTIMIZATION_SIZE=y +CONFIG_BOOTLOADER_LOG_LEVEL_WARN=y +CONFIG_LOG_DEFAULT_LEVEL_INFO=y +CONFIG_LWIP_SO_RCVBUF=y +CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 +CONFIG_FREERTOS_TIMER_TASK_STACK_DEPTH=8192 diff --git a/plugins/ruview/skills/ruview-rvagent/SKILL.md b/plugins/ruview/skills/ruview-rvagent/SKILL.md index da4f5ba02e..7feb097e56 100644 --- a/plugins/ruview/skills/ruview-rvagent/SKILL.md +++ b/plugins/ruview/skills/ruview-rvagent/SKILL.md @@ -1,6 +1,7 @@ --- name: ruview-rvagent description: Explore and prototype rvAgent + RVF integration for RuView agentic flows. Use when working on cross-cog coordination, operator-facing agents reading BFLD / pose / vitals events live, or persisting agent state alongside sensing data in the same RVF container. +allowed-tools: Read, Grep, Glob, Bash --- # RuView rvAgent + RVF integration diff --git a/tools/smoke/observatory-playwright-screenshot.sh b/tools/smoke/observatory-playwright-screenshot.sh new file mode 100755 index 0000000000..291dfcae28 --- /dev/null +++ b/tools/smoke/observatory-playwright-screenshot.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +set -euo pipefail + +BASE="${1:-http://127.0.0.1:3000/observatory.html}" +OUT="${2:-/tmp/ruview-observatory-playwright.png}" + +if [ -z "${PLAYWRIGHT_BROWSERS_PATH:-}" ] && [ -d /run/media/deck/SDCARD/agents/playwright-browsers ]; then + export PLAYWRIGHT_BROWSERS_PATH=/run/media/deck/SDCARD/agents/playwright-browsers +fi + +npx playwright screenshot \ + --browser=chromium \ + --viewport-size=1280,800 \ + "$BASE" \ + "$OUT" + +python3 - "$OUT" <<'PY' +import sys +from PIL import Image, ImageStat + +p = sys.argv[1] +im = Image.open(p).convert("RGB") +stat = ImageStat.Stat(im) +print(f"file={p}") +print(f"size={im.size[0]}x{im.size[1]}") +print("stddev=" + ",".join(f"{x:.2f}" for x in stat.stddev)) +print("nonblank=" + str(max(stat.stddev) > 5).lower()) +PY diff --git a/ui/app.js b/ui/app.js index 5c5bada66e..b8c0eb0362 100644 --- a/ui/app.js +++ b/ui/app.js @@ -3,13 +3,12 @@ import { TabManager } from './components/TabManager.js'; import { DashboardTab } from './components/DashboardTab.js'; import { HardwareTab } from './components/HardwareTab.js'; -import { LiveDemoTab } from './components/LiveDemoTab.js'; +import { LiveViewTab } from './components/LiveViewTab.js'; import { SensingTab } from './components/SensingTab.js'; import { apiService } from './services/api.service.js'; import { wsService } from './services/websocket.service.js'; import { healthService } from './services/health.service.js'; import { sensingService } from './services/sensing.service.js'; -import { backendDetector } from './utils/backend-detector.js'; import { KeyboardShortcuts } from './utils/keyboard-shortcuts.js'; import { PerfMonitor } from './utils/perf-monitor.js'; import { toastManager } from './utils/toast.js'; @@ -75,33 +74,20 @@ class WiFiDensePoseApp { return response; }); - // Detect backend availability and initialize accordingly - const useMock = await backendDetector.shouldUseMockServer(); - - if (useMock) { - console.log('🧪 Initializing with mock server for testing'); - // Import and start mock server only when needed - const { mockServer } = await import('./utils/mock-server.js'); - mockServer.start(); - - // Show notification to user - this.showBackendStatus('Mock server active - testing mode', 'warning'); - } else { - console.log('🔌 Connecting to backend...'); - - try { - const health = await healthService.checkLiveness(); - console.log('✅ Backend responding:', health); - this.showBackendStatus('Connected to Rust sensing server', 'success'); - } catch (error) { - console.warn('⚠️ Backend not available:', error.message); - this.showBackendStatus('Backend unavailable — start sensing-server', 'warning'); - } + console.log('🔌 Connecting to backend...'); - // Start the sensing WebSocket service early so the dashboard and - // live-demo tabs can show the correct data-source status immediately. - sensingService.start(); + try { + const health = await healthService.checkLiveness(); + console.log('✅ Backend responding:', health); + this.showBackendStatus('Connected to sensing server', 'success'); + } catch (error) { + console.warn('⚠️ Backend not available:', error.message); + this.showBackendStatus('Backend unavailable — no live data', 'warning'); } + + // Keep hardware status current on every tab. The optional sensing + // WebSocket is opened lazily by SensingTab so unrelated pages stay quiet. + sensingService.start({ websocket: false }); } // Initialize UI components @@ -143,11 +129,11 @@ class WiFiDensePoseApp { this.components.hardware.init(); } - // Live demo tab - const demoContainer = document.getElementById('demo'); - if (demoContainer) { - this.components.demo = new LiveDemoTab(demoContainer); - this.components.demo.init(); + // Live view tab + const liveContainer = document.getElementById('live'); + if (liveContainer) { + this.components.live = new LiveViewTab(liveContainer); + this.components.live.init(); } // Sensing tab @@ -304,11 +290,6 @@ class WiFiDensePoseApp { handleTabChange(newTab, oldTab) { console.log(`Tab changed from ${oldTab} to ${newTab}`); - // Stop demo if leaving demo tab - if (oldTab === 'demo' && this.components.demo) { - this.components.demo.stopDemo(); - } - // Update components based on active tab switch (newTab) { case 'dashboard': @@ -319,8 +300,8 @@ class WiFiDensePoseApp { // Hardware visualization is always active break; - case 'demo': - // Demo starts manually + case 'live': + // Live view starts manually break; case 'sensing': @@ -473,4 +454,4 @@ document.addEventListener('DOMContentLoaded', () => { }); // Export for testing -export { WiFiDensePoseApp }; \ No newline at end of file +export { WiFiDensePoseApp }; diff --git a/ui/components/DashboardTab.js b/ui/components/DashboardTab.js index 9ecd022621..b165deb6e8 100644 --- a/ui/components/DashboardTab.js +++ b/ui/components/DashboardTab.js @@ -4,12 +4,22 @@ import { healthService } from '../services/health.service.js'; import { poseService } from '../services/pose.service.js'; import { sensingService } from '../services/sensing.service.js'; +export function parseFocusNodeIdFromSearch(search = '') { + const params = new URLSearchParams(search); + const raw = params.get('node') || params.get('node_id') || ''; + const normalized = raw.toLowerCase().replace(/^node-?/, ''); + const nodeId = Number.parseInt(normalized, 10); + return Number.isFinite(nodeId) && nodeId > 0 ? nodeId : null; +} + export class DashboardTab { constructor(containerElement) { this.container = containerElement; this.statsElements = {}; this.healthSubscription = null; this.statsInterval = null; + this.nodeStatusInterval = null; + this.focusNodeId = parseFocusNodeIdFromSearch(window.location.search); } // Initialize component @@ -32,12 +42,8 @@ export class DashboardTab { }; } - // Status indicators - this.statusElements = { - apiStatus: this.container.querySelector('.api-status'), - streamStatus: this.container.querySelector('.stream-status'), - hardwareStatus: this.container.querySelector('.hardware-status') - }; + this.nodeSummary = this.container.querySelector('#node-summary'); + this.nodeStatusGrid = this.container.querySelector('#node-status-grid'); } // Load initial data @@ -74,11 +80,15 @@ export class DashboardTab { }); // Initial update this.updateDataSourceIndicator(); + this.updateNodeStatus(); // Start periodic stats updates this.statsInterval = setInterval(() => { this.updateLiveStats(); }, 5000); + this.nodeStatusInterval = setInterval(() => { + this.updateNodeStatus(); + }, 1000); // Start health monitoring healthService.startHealthMonitoring(30000); @@ -93,9 +103,8 @@ export class DashboardTab { const statusMsg = el.querySelector('.status-message'); const config = { 'live': { text: 'ESP32', status: 'healthy', msg: 'Real hardware connected' }, - 'server-simulated': { text: 'SIMULATED', status: 'warning', msg: 'Server running without hardware' }, + 'stale': { text: 'STALE', status: 'degraded', msg: 'Waiting for fresh feature state' }, 'reconnecting': { text: 'RECONNECTING', status: 'degraded', msg: 'Attempting to connect...' }, - 'simulated': { text: 'OFFLINE', status: 'unhealthy', msg: 'Server unreachable, local fallback' }, }; const cfg = config[ds] || config['reconnecting']; el.className = `component-status status-${cfg.status}`; @@ -103,6 +112,142 @@ export class DashboardTab { if (statusMsg) statusMsg.textContent = cfg.msg; } + async updateNodeStatus() { + if (!this.nodeStatusGrid || !this.nodeSummary) return; + + try { + const response = await fetch('/api/v1/cardputer/status', { cache: 'no-store' }); + if (!response.ok) { + throw new Error(`status ${response.status}`); + } + const status = await response.json(); + this.renderNodeStatus(status); + } catch (error) { + this.nodeSummary.textContent = 'Status unavailable'; + this.nodeStatusGrid.replaceChildren( + this.createNodeEmptyState(`Cardputer status endpoint unavailable: ${error.message}`) + ); + } + } + + renderNodeStatus(status) { + const nodes = Array.isArray(status.nodes) ? status.nodes : []; + const liveCount = Number.isFinite(status.live_node_count) + ? status.live_node_count + : nodes.filter(node => node.live).length; + const totalCount = Number.isFinite(status.node_count) ? status.node_count : nodes.length; + + const focusNode = this.focusNodeId + ? nodes.find(node => Number(node.node_id) === this.focusNodeId) + : null; + this.nodeSummary.textContent = totalCount > 0 + ? this.formatNodeSummary(liveCount, totalCount, focusNode) + : 'Waiting for packets'; + + if (nodes.length === 0) { + this.nodeStatusGrid.replaceChildren(this.createNodeEmptyState('No node packets received')); + return; + } + + const cards = [...nodes] + .sort((a, b) => this.compareNodes(a, b)) + .map(node => this.createNodeStatusCard(node)); + this.nodeStatusGrid.replaceChildren(...cards); + } + + formatNodeSummary(liveCount, totalCount, focusNode) { + if (!this.focusNodeId) return `${liveCount}/${totalCount} live`; + if (!focusNode) return `Node ${this.focusNodeId} waiting - ${liveCount}/${totalCount} live`; + return `Node ${this.focusNodeId} ${focusNode.live ? 'live' : 'stale'} - ${liveCount}/${totalCount} live`; + } + + compareNodes(a, b) { + const aId = Number(a.node_id); + const bId = Number(b.node_id); + if (this.focusNodeId) { + if (aId === this.focusNodeId && bId !== this.focusNodeId) return -1; + if (bId === this.focusNodeId && aId !== this.focusNodeId) return 1; + } + return aId - bId; + } + + createNodeStatusCard(node) { + const card = document.createElement('article'); + card.className = `node-status-card ${node.live ? 'node-live' : 'node-stale'}`; + if (Number(node.node_id) === this.focusNodeId) { + card.classList.add('node-focused'); + } + + const header = document.createElement('div'); + header.className = 'node-status-header'; + const title = document.createElement('h4'); + title.textContent = `Node ${node.node_id}`; + const pill = document.createElement('span'); + pill.className = 'node-status-pill'; + pill.textContent = node.live ? 'LIVE' : 'STALE'; + header.append(title, pill); + + const fields = document.createElement('dl'); + fields.className = 'node-status-fields'; + [ + ['Source', node.last_source || '-'], + ['Packets', this.formatNumber(node.packet_count || 0)], + ['Age', this.formatAge(node.last_packet_age_s)], + ['Type', node.last_packet_type || '-'], + ['Presence', this.formatMetric(node.feature_state?.presence_score)], + ['Motion', this.formatMetric(node.feature_state?.motion_score)], + ['RSSI', this.formatRssi(node.rssi_dbm)], + ['Battery', this.formatBattery(node.battery)], + ['Seen', this.formatPacketTypes(node.packet_types)] + ].forEach(([label, value]) => { + const term = document.createElement('dt'); + term.textContent = label; + const detail = document.createElement('dd'); + detail.textContent = value; + fields.append(term, detail); + }); + + card.append(header, fields); + return card; + } + + createNodeEmptyState(message) { + const empty = document.createElement('div'); + empty.className = 'node-status-empty'; + empty.textContent = message; + return empty; + } + + formatAge(value) { + if (!Number.isFinite(value)) return '-'; + if (value < 1) return `${Math.round(value * 1000)}ms`; + return `${value.toFixed(1)}s`; + } + + formatMetric(value, digits = 2) { + if (!Number.isFinite(value)) return '-'; + return Number(value).toFixed(digits); + } + + formatRssi(value) { + if (!Number.isFinite(value)) return 'not reported'; + return `${Math.round(value)} dBm`; + } + + formatBattery(battery) { + if (!battery?.valid) return 'not reported'; + const charge = battery.charging ? ' charging' : ''; + return `${battery.percent}%${charge}`; + } + + formatPacketTypes(packetTypes) { + if (!packetTypes || Object.keys(packetTypes).length === 0) return '-'; + return Object.entries(packetTypes) + .sort(([a], [b]) => a.localeCompare(b)) + .map(([type, count]) => `${type}:${count}`) + .join(' '); + } + // Update API info display updateApiInfo(info) { // Update version @@ -221,8 +366,7 @@ export class DashboardTab { // Update system metrics updateSystemMetrics(metrics) { - // Handle both flat and nested metric structures - // Backend returns system_metrics.cpu.percent, mock returns metrics.cpu.percent + // Handle both flat and nested metric structures. const systemMetrics = metrics.system_metrics || metrics; const cpuPercent = systemMetrics.cpu?.percent || systemMetrics.cpu_percent; const memoryPercent = systemMetrics.memory?.percent || systemMetrics.memory_percent; @@ -431,7 +575,10 @@ export class DashboardTab { if (this.statsInterval) { clearInterval(this.statsInterval); } + if (this.nodeStatusInterval) { + clearInterval(this.nodeStatusInterval); + } healthService.stopHealthMonitoring(); } -} \ No newline at end of file +} diff --git a/ui/components/HardwareTab.js b/ui/components/HardwareTab.js index 7f36113a93..48a1141296 100644 --- a/ui/components/HardwareTab.js +++ b/ui/components/HardwareTab.js @@ -11,7 +11,7 @@ export class HardwareTab { // Initialize component init() { this.setupAntennas(); - this.startCSISimulation(); + this.startCSIStatus(); } // Set up antenna interactions @@ -26,17 +26,10 @@ export class HardwareTab { }); } - // Start CSI simulation - startCSISimulation() { + // Start CSI status display. Values remain empty until live hardware supplies data. + startCSIStatus() { // Initial update this.updateCSIDisplay(); - - // Set up periodic updates - this.csiUpdateInterval = setInterval(() => { - if (this.hasActiveAntennas()) { - this.updateCSIDisplay(); - } - }, 1000); } // Check if any antennas are active @@ -64,36 +57,10 @@ export class HardwareTab { return; } - // Generate realistic CSI values based on active antennas - const txCount = activeAntennas.filter(a => a.classList.contains('tx')).length; - const rxCount = activeAntennas.filter(a => a.classList.contains('rx')).length; - - // Amplitude increases with more active antennas - const baseAmplitude = 0.3 + (txCount * 0.1) + (rxCount * 0.05); - const amplitude = Math.min(0.95, baseAmplitude + (Math.random() * 0.1 - 0.05)); - - // Phase varies more with multiple antennas - const phaseVariation = 0.5 + (activeAntennas.length * 0.1); - const phase = 0.5 + Math.random() * phaseVariation; - - // Update display - if (amplitudeFill) { - amplitudeFill.style.width = `${amplitude * 100}%`; - amplitudeFill.style.transition = 'width 0.5s ease'; - } - - if (phaseFill) { - phaseFill.style.width = `${phase * 50}%`; - phaseFill.style.transition = 'width 0.5s ease'; - } - - if (amplitudeValue) { - amplitudeValue.textContent = amplitude.toFixed(2); - } - - if (phaseValue) { - phaseValue.textContent = `${phase.toFixed(1)}π`; - } + if (amplitudeFill) amplitudeFill.style.width = '0%'; + if (phaseFill) phaseFill.style.width = '0%'; + if (amplitudeValue) amplitudeValue.textContent = '--'; + if (phaseValue) phaseValue.textContent = '--'; // Update antenna array visualization this.updateAntennaArray(activeAntennas); @@ -129,18 +96,12 @@ export class HardwareTab { arrayStatus.appendChild(createInfoDiv('Active TX:', `${txActive}/3`)); arrayStatus.appendChild(createInfoDiv('Active RX:', `${rxActive}/6`)); - arrayStatus.appendChild(createInfoDiv('Signal Quality:', `${this.calculateSignalQuality(txActive, rxActive)}%`)); + arrayStatus.appendChild(createInfoDiv('Signal Quality:', 'Live data required')); } - // Calculate signal quality based on active antennas + // Calculate signal quality from live CSI only. calculateSignalQuality(txCount, rxCount) { - if (txCount === 0 || rxCount === 0) return 0; - - const txRatio = txCount / 3; - const rxRatio = rxCount / 6; - const quality = (txRatio * 0.4 + rxRatio * 0.6) * 100; - - return Math.round(quality); + return 0; } // Toggle all antennas @@ -171,4 +132,4 @@ export class HardwareTab { antenna.removeEventListener('click', this.toggleAntenna); }); } -} \ No newline at end of file +} diff --git a/ui/components/LiveDemoTab.js b/ui/components/LiveViewTab.js similarity index 89% rename from ui/components/LiveDemoTab.js rename to ui/components/LiveViewTab.js index 4dec767d20..8020e3534e 100644 --- a/ui/components/LiveDemoTab.js +++ b/ui/components/LiveViewTab.js @@ -1,4 +1,4 @@ -// Live Demo Tab Component - Enhanced Version +// Live View Tab Component import { PoseDetectionCanvas } from './PoseDetectionCanvas.js'; import { poseService } from '../services/pose.service.js'; @@ -10,7 +10,7 @@ import { sensingService } from '../services/sensing.service.js'; let modelService = null; let trainingService = null; -export class LiveDemoTab { +export class LiveViewTab { constructor(containerElement) { this.container = containerElement; this.state = { @@ -73,17 +73,17 @@ export class LiveDemoTab { createLogger() { return { - debug: (...args) => console.debug('[LIVEDEMO-DEBUG]', new Date().toISOString(), ...args), - info: (...args) => console.info('[LIVEDEMO-INFO]', new Date().toISOString(), ...args), - warn: (...args) => console.warn('[LIVEDEMO-WARN]', new Date().toISOString(), ...args), - error: (...args) => console.error('[LIVEDEMO-ERROR]', new Date().toISOString(), ...args) + debug: (...args) => console.debug('[LIVEVIEW-DEBUG]', new Date().toISOString(), ...args), + info: (...args) => console.info('[LIVEVIEW-INFO]', new Date().toISOString(), ...args), + warn: (...args) => console.warn('[LIVEVIEW-WARN]', new Date().toISOString(), ...args), + error: (...args) => console.error('[LIVEVIEW-ERROR]', new Date().toISOString(), ...args) }; } // Initialize component async init() { try { - this.logger.info('Initializing LiveDemoTab component'); + this.logger.info('Initializing LiveViewTab component'); // Load optional services (non-blocking) try { @@ -116,25 +116,9 @@ export class LiveDemoTab { // Initialize state this.updateUI(); - // Auto-start pose detection when a backend is reachable. - // Check after a brief delay (sensing WS may still be connecting). - this._autoStartOnce = false; - const tryAutoStart = () => { - if (this._autoStartOnce || this.state.isActive) return; - const ds = sensingService.dataSource; - if (ds === 'live' || ds === 'server-simulated') { - this._autoStartOnce = true; - this.logger.info('Auto-starting pose detection (data source: ' + ds + ')'); - this.startDemo(); - } - }; - setTimeout(tryAutoStart, 2000); - // Also listen for sensing state changes in case server connects later - this._autoStartUnsub = sensingService.onStateChange(tryAutoStart); - - this.logger.info('LiveDemoTab component initialized successfully'); + this.logger.info('LiveViewTab component initialized successfully'); } catch (error) { - this.logger.error('Failed to initialize LiveDemoTab', { error: error.message }); + this.logger.error('Failed to initialize LiveViewTab', { error: error.message }); this.showError(`Initialization failed: ${error.message}`); } } @@ -145,24 +129,33 @@ export class LiveDemoTab { if (!existingCanvas) { // Create enhanced structure if it doesn't exist const enhancedHTML = ` -
- -
+
+ +
Detecting data source...
-
-
-

Live Human Pose Detection

-
- - Ready +
+
+

Live View

+
+

Live View README

+

This page renders live RuView pose and sensing output only. If hardware or backend data is missing, it shows offline or waiting state instead of local stand-in data.

+
    +
  • Start Detection opens the live pose stream.
    • +
  • Data Source shows whether packets are live, stale, reconnecting, or offline.
    • +
  • Stop Detection is manual; switching tabs does not stop live data.
    • +
  • How data is used: live pose frames drive the canvas, counters, model comparison, and source banners; missing data stays visible as waiting/offline.
    • +
+
+
+ + Ready
-
- - - +
+ + @@ -268,13 +267,13 @@ export class PoseDetectionCanvas { box-shadow: 0 4px 12px rgba(59, 130, 246, 0.2); } - .btn-demo { + .btn-local-disabled { background: rgba(139, 92, 246, 0.15); color: #a78bfa; border-color: rgba(139, 92, 246, 0.3); } - .btn-demo:hover:not(:disabled) { + .btn-local-disabled:hover:not(:disabled) { background: rgba(139, 92, 246, 0.25); border-color: rgba(139, 92, 246, 0.5); box-shadow: 0 4px 12px rgba(139, 92, 246, 0.2); @@ -437,10 +436,6 @@ export class PoseDetectionCanvas { const reconnectBtn = document.getElementById(`reconnect-btn-${this.containerId}`); reconnectBtn.addEventListener('click', () => this.reconnect()); - // Demo button - const demoBtn = document.getElementById(`demo-btn-${this.containerId}`); - demoBtn.addEventListener('click', () => this.toggleDemo()); - // Trail toggle button const trailBtn = document.getElementById(`trail-btn-${this.containerId}`); trailBtn.addEventListener('click', () => this.toggleTrail()); @@ -804,368 +799,33 @@ export class PoseDetectionCanvas { ctx.globalAlpha = 1.0; } - // Toggle demo mode - toggleDemo() { - if (this.demoState && this.demoState.isRunning) { - this.stopDemo(); - this.updateDemoButton(false); - } else { - this.runDemo(); - this.updateDemoButton(true); - } - } - - // Demo mode - renders animated test pose data - runDemo() { - this.logger.info('Running animated demo mode'); - - // Stop any existing demo animation - this.stopDemo(); - - // Force enable all visual elements for demo - this.originalConfig = { ...this.renderer.config }; - this.renderer.updateConfig({ - showKeypoints: true, - showSkeleton: true, - showBoundingBox: true, - showConfidence: true, - confidenceThreshold: 0.1, - keypointConfidenceThreshold: 0.1 - }); - - // Initialize animation state - this.demoState = { - isRunning: true, - frameCount: 0, - startTime: Date.now(), - animations: { - person1: { type: 'walking', phase: 0, centerX: 150, centerY: 250 }, - person2: { type: 'waving', phase: 0, centerX: 350, centerY: 270 }, - person3: { type: 'dancing', phase: 0, centerX: 550, centerY: 260 } - } - }; - - // Start animation loop - this.startDemoAnimation(); - - // Show demo notification - this.showDemoNotification('🎭 Animated Demo Active - Walking, Waving & Dancing'); + // Local pose synthesis is disabled; live hardware is required. + rejectLocalPoseMode() { + this.showError('Local stand-in pose data has been removed. Connect live hardware to render poses.'); } - stopDemo() { - if (this.demoState && this.demoState.isRunning) { - this.demoState.isRunning = false; - if (this.demoAnimationFrame) { - cancelAnimationFrame(this.demoAnimationFrame); - } - if (this.originalConfig) { - this.renderer.updateConfig(this.originalConfig); - } - // Clear canvas - if (this.renderer) { - this.renderer.clearCanvas(); - } - this.logger.info('Demo stopped'); - } + runLocalPoseMode() { + this.rejectLocalPoseMode(); } - updateDemoButton(isRunning) { - const demoBtn = document.getElementById(`demo-btn-${this.containerId}`); - if (demoBtn) { - demoBtn.textContent = isRunning ? 'Stop Demo' : 'Demo'; - demoBtn.style.background = isRunning ? '#dc3545' : '#6f42c1'; - demoBtn.style.borderColor = isRunning ? '#dc3545' : '#6f42c1'; + clearLocalPoseState() { + if (this.localPoseAnimationFrame) { + cancelAnimationFrame(this.localPoseAnimationFrame); + this.localPoseAnimationFrame = null; } + if (this.renderer) this.renderer.clearCanvas(); } - startDemoAnimation() { - if (!this.demoState || !this.demoState.isRunning) return; - - this.demoState.frameCount++; - const elapsed = (Date.now() - this.demoState.startTime) / 1000; - - // Generate animated pose data - const animatedPoseData = this.generateAnimatedPoseData(elapsed); - - // Render the animated data - this.renderPoseData(animatedPoseData); - - // Continue animation - this.demoAnimationFrame = requestAnimationFrame(() => this.startDemoAnimation()); - } - - generateAnimatedPoseData(time) { - const persons = []; - - // Person 1: Walking animation - const person1 = this.generateWalkingPerson( - this.demoState.animations.person1.centerX, - this.demoState.animations.person1.centerY, - time * 2 // Walking speed - ); - persons.push(person1); - - // Person 2: Waving animation - const person2 = this.generateWavingPerson( - this.demoState.animations.person2.centerX, - this.demoState.animations.person2.centerY, - time * 3 // Waving speed - ); - persons.push(person2); - - // Person 3: Dancing animation - const person3 = this.generateDancingPerson( - this.demoState.animations.person3.centerX, - this.demoState.animations.person3.centerY, - time * 2.5 // Dancing speed - ); - persons.push(person3); - - return { - timestamp: new Date().toISOString(), - frame_id: `demo_frame_${this.demoState.frameCount.toString().padStart(6, '0')}`, - persons: persons, - zone_summary: { - demo_zone: persons.length - }, - processing_time_ms: 12 + Math.random() * 8, - metadata: { - mock_data: true, - source: 'animated_demo', - fps: Math.round(this.demoState.frameCount / ((Date.now() - this.demoState.startTime) / 1000)) - } - }; + updateLocalPoseButton() { + return; } - generateWalkingPerson(centerX, centerY, time) { - // Walking cycle parameters - const walkCycle = Math.sin(time) * 0.3; - const stepPhase = Math.sin(time * 2) * 0.2; - - // Base keypoint positions for walking - const keypoints = [ - // Head (nose, eyes, ears) - slight bob - { x: centerX, y: centerY - 80 + Math.sin(time * 4) * 2, confidence: 0.95 }, - { x: centerX - 8, y: centerY - 85 + Math.sin(time * 4) * 2, confidence: 0.92 }, - { x: centerX + 8, y: centerY - 85 + Math.sin(time * 4) * 2, confidence: 0.93 }, - { x: centerX - 15, y: centerY - 82 + Math.sin(time * 4) * 2, confidence: 0.88 }, - { x: centerX + 15, y: centerY - 82 + Math.sin(time * 4) * 2, confidence: 0.89 }, - - // Shoulders - subtle movement - { x: centerX - 35 + walkCycle * 5, y: centerY - 40 + Math.sin(time * 4) * 1, confidence: 0.94 }, - { x: centerX + 35 - walkCycle * 5, y: centerY - 40 + Math.sin(time * 4) * 1, confidence: 0.95 }, - - // Elbows - arm swing - { x: centerX - 25 + walkCycle * 20, y: centerY + 10 + walkCycle * 10, confidence: 0.91 }, - { x: centerX + 25 - walkCycle * 20, y: centerY + 10 - walkCycle * 10, confidence: 0.92 }, - - // Wrists - follow elbows - { x: centerX - 15 + walkCycle * 25, y: centerY + 55 + walkCycle * 15, confidence: 0.87 }, - { x: centerX + 15 - walkCycle * 25, y: centerY + 55 - walkCycle * 15, confidence: 0.88 }, - - // Hips - slight movement - { x: centerX - 18 + walkCycle * 3, y: centerY + 60, confidence: 0.96 }, - { x: centerX + 18 - walkCycle * 3, y: centerY + 60, confidence: 0.96 }, - - // Knees - walking motion - { x: centerX - 20 + stepPhase * 15, y: centerY + 120 - Math.abs(stepPhase) * 10, confidence: 0.93 }, - { x: centerX + 20 - stepPhase * 15, y: centerY + 120 - Math.abs(-stepPhase) * 10, confidence: 0.94 }, - - // Ankles - foot placement - { x: centerX - 22 + stepPhase * 20, y: centerY + 180, confidence: 0.90 }, - { x: centerX + 22 - stepPhase * 20, y: centerY + 180, confidence: 0.91 } - ]; - - return { - person_id: 'demo_walker', - confidence: 0.94 + Math.sin(time) * 0.03, - bbox: this.calculateBoundingBox(keypoints), - keypoints: keypoints, - zone_id: 'demo_zone', - activity: 'walking' - }; + startLocalPoseAnimation() { + return; } - generateWavingPerson(centerX, centerY, time) { - // Waving parameters - const wavePhase = Math.sin(time) * 0.8; - const armWave = Math.sin(time * 1.5) * 30; - - const keypoints = [ - // Head - stable - { x: centerX, y: centerY - 80, confidence: 0.96 }, - { x: centerX - 8, y: centerY - 85, confidence: 0.94 }, - { x: centerX + 8, y: centerY - 85, confidence: 0.94 }, - { x: centerX - 15, y: centerY - 82, confidence: 0.90 }, - { x: centerX + 15, y: centerY - 82, confidence: 0.91 }, - - // Shoulders - { x: centerX - 35, y: centerY - 40, confidence: 0.95 }, - { x: centerX + 35, y: centerY - 40, confidence: 0.95 }, - - // Elbows - left arm stable, right arm waving - { x: centerX - 55, y: centerY + 10, confidence: 0.92 }, - { x: centerX + 65 + armWave * 0.3, y: centerY - 10 - Math.abs(armWave) * 0.5, confidence: 0.93 }, - - // Wrists - dramatic wave motion - { x: centerX - 60, y: centerY + 60, confidence: 0.88 }, - { x: centerX + 45 + armWave, y: centerY - 30 - Math.abs(armWave) * 0.8, confidence: 0.89 }, - - // Hips - stable - { x: centerX - 18, y: centerY + 60, confidence: 0.97 }, - { x: centerX + 18, y: centerY + 60, confidence: 0.97 }, - - // Knees - slight movement - { x: centerX - 20, y: centerY + 120 + Math.sin(time * 0.5) * 5, confidence: 0.94 }, - { x: centerX + 20, y: centerY + 120 + Math.sin(time * 0.5) * 5, confidence: 0.95 }, - - // Ankles - stable - { x: centerX - 22, y: centerY + 180, confidence: 0.92 }, - { x: centerX + 22, y: centerY + 180, confidence: 0.93 } - ]; - - return { - person_id: 'demo_waver', - confidence: 0.91 + Math.sin(time * 0.7) * 0.05, - bbox: this.calculateBoundingBox(keypoints), - keypoints: keypoints, - zone_id: 'demo_zone', - activity: 'waving' - }; - } - - generateDancingPerson(centerX, centerY, time) { - // Dancing parameters - more complex movement - const dancePhase1 = Math.sin(time * 1.2) * 0.6; - const dancePhase2 = Math.cos(time * 1.8) * 0.4; - const bodyBob = Math.sin(time * 3) * 8; - const hipSway = Math.sin(time * 1.5) * 15; - - const keypoints = [ - // Head - dancing bob - { x: centerX + dancePhase1 * 5, y: centerY - 80 + bodyBob, confidence: 0.96 }, - { x: centerX - 8 + dancePhase1 * 5, y: centerY - 85 + bodyBob, confidence: 0.94 }, - { x: centerX + 8 + dancePhase1 * 5, y: centerY - 85 + bodyBob, confidence: 0.94 }, - { x: centerX - 15 + dancePhase1 * 5, y: centerY - 82 + bodyBob, confidence: 0.90 }, - { x: centerX + 15 + dancePhase1 * 5, y: centerY - 82 + bodyBob, confidence: 0.91 }, - - // Shoulders - dance movement - { x: centerX - 35 + dancePhase1 * 10, y: centerY - 40 + bodyBob * 0.5, confidence: 0.95 }, - { x: centerX + 35 + dancePhase2 * 10, y: centerY - 40 + bodyBob * 0.5, confidence: 0.95 }, - - // Elbows - both arms dancing - { x: centerX - 45 + dancePhase1 * 25, y: centerY + 0 + dancePhase1 * 20, confidence: 0.92 }, - { x: centerX + 45 + dancePhase2 * 25, y: centerY + 0 + dancePhase2 * 20, confidence: 0.93 }, - - // Wrists - expressive arm movements - { x: centerX - 40 + dancePhase1 * 35, y: centerY + 50 + dancePhase1 * 30, confidence: 0.88 }, - { x: centerX + 40 + dancePhase2 * 35, y: centerY + 50 + dancePhase2 * 30, confidence: 0.89 }, - - // Hips - dancing sway - { x: centerX - 18 + hipSway * 0.3, y: centerY + 60 + bodyBob * 0.3, confidence: 0.97 }, - { x: centerX + 18 + hipSway * 0.3, y: centerY + 60 + bodyBob * 0.3, confidence: 0.97 }, - - // Knees - dancing steps - { x: centerX - 20 + hipSway * 0.5 + Math.sin(time * 2.5) * 10, y: centerY + 120 + Math.abs(Math.sin(time * 2.5)) * 15, confidence: 0.94 }, - { x: centerX + 20 + hipSway * 0.5 + Math.cos(time * 2.5) * 10, y: centerY + 120 + Math.abs(Math.cos(time * 2.5)) * 15, confidence: 0.95 }, - - // Ankles - feet positioning - { x: centerX - 22 + hipSway * 0.6 + Math.sin(time * 2.5) * 12, y: centerY + 180, confidence: 0.92 }, - { x: centerX + 22 + hipSway * 0.6 + Math.cos(time * 2.5) * 12, y: centerY + 180, confidence: 0.93 } - ]; - - return { - person_id: 'demo_dancer', - confidence: 0.89 + Math.sin(time * 1.3) * 0.07, - bbox: this.calculateBoundingBox(keypoints), - keypoints: keypoints, - zone_id: 'demo_zone', - activity: 'dancing' - }; - } - - calculateBoundingBox(keypoints) { - const validPoints = keypoints.filter(kp => kp.confidence > 0.1); - if (validPoints.length === 0) return { x: 0, y: 0, width: 50, height: 50 }; - - const xs = validPoints.map(kp => kp.x); - const ys = validPoints.map(kp => kp.y); - - const minX = Math.min(...xs) - 10; - const maxX = Math.max(...xs) + 10; - const minY = Math.min(...ys) - 10; - const maxY = Math.max(...ys) + 10; - - return { - x: minX, - y: minY, - width: maxX - minX, - height: maxY - minY - }; - } - - generateDemoKeypoints(centerX, centerY) { - // COCO keypoint order: nose, left_eye, right_eye, left_ear, right_ear, - // left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, - // left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle - const offsets = [ - [0, -80], // nose - [-10, -90], // left_eye - [10, -90], // right_eye - [-20, -85], // left_ear - [20, -85], // right_ear - [-40, -40], // left_shoulder - [40, -40], // right_shoulder - [-60, 10], // left_elbow - [60, 10], // right_elbow - [-65, 60], // left_wrist - [65, 60], // right_wrist - [-20, 60], // left_hip - [20, 60], // right_hip - [-25, 120], // left_knee - [25, 120], // right_knee - [-25, 180], // left_ankle - [25, 180] // right_ankle - ]; - - return offsets.map(([dx, dy]) => ({ - x: centerX + dx, - y: centerY + dy, - confidence: 0.8 + (Math.random() * 0.2) - })); - } - - showDemoNotification(message = '🎭 Demo Mode Active') { - const notification = document.createElement('div'); - notification.style.cssText = ` - position: absolute; - top: 10px; - left: 10px; - background: rgba(111, 66, 193, 0.9); - color: white; - padding: 10px 15px; - border-radius: 4px; - font-size: 14px; - z-index: 20; - pointer-events: none; - box-shadow: 0 2px 8px rgba(0,0,0,0.3); - `; - notification.textContent = message; - - const overlay = document.getElementById(`overlay-${this.containerId}`); - - // Remove any existing notifications - const existingNotifications = overlay.querySelectorAll('div[style*="background: rgba(111, 66, 193"]'); - existingNotifications.forEach(n => n.remove()); - - overlay.appendChild(notification); - - // Remove notification after 3 seconds - setTimeout(() => { - if (notification.parentNode) { - notification.parentNode.removeChild(notification); - } - }, 3000); + generateAnimatedPoseData() { + return null; } // Configuration methods @@ -1519,8 +1179,8 @@ export class PoseDetectionCanvas { this.stop(); } - // Stop demo animation - this.stopDemo(); + // Clear disabled local-pose state. + this.clearLocalPoseState(); // Dispose settings panel if (this.settingsPanel) { @@ -1550,4 +1210,4 @@ export class PoseDetectionCanvas { this.logger.error('Error during disposal', { error: error.message }); } } -} \ No newline at end of file +} diff --git a/ui/components/SensingTab.js b/ui/components/SensingTab.js index 33387eefe7..8d8670597a 100644 --- a/ui/components/SensingTab.js +++ b/ui/components/SensingTab.js @@ -33,6 +33,16 @@ export class SensingTab { _buildDOM() { this.container.innerHTML = `

Live WiFi Sensing

+
+

Sensing README

+

This page visualizes live CSI-derived signal state from the RuView sensing service.

+
    +
  • The viewport updates only from live messages or HTTP status-derived hardware frames.
    • +
  • RSSI, variance, motion, breathing, and spectral values stay blank until reported.
    • +
  • Disconnected and stale states are shown explicitly; local fallback frames are disabled.
    • +
  • How data is used: CSI-derived features drive the 3D field, meters, classification label, sparkline, and node panels.
    • +
+
- 0 + --
- 0 + --
- 0 + --
- 0 + --
@@ -96,11 +106,11 @@ export class SensingTab {
Classification
-
ABSENT
+
UNKNOWN
- 0% + --
@@ -110,9 +120,8 @@ export class SensingTab {
About This Data

Metrics are computed from WiFi Channel State Information (CSI). - With 0 ESP32 node(s) you get presence detection, breathing - estimation, and gross motion. Add 3-4+ ESP32 nodes - around the room for spatial resolution and limb-level tracking. + Connected ESP32 node count is reported by the live backend. More nodes improve spatial resolution when + the backend provides calibrated CSI streams.

@@ -127,10 +136,10 @@ export class SensingTab {
Details
- Dominant Freq0 Hz + Dominant Freq--
- Change Points0 + Change Points--
Sample Rate-- @@ -150,15 +159,26 @@ export class SensingTab { return; } - return new Promise((resolve, reject) => { + return new Promise((resolve) => { + let settled = false; + const finish = (loaded) => { + if (settled) return; + settled = true; + this._threeLoaded = loaded; + resolve(); + }; const script = document.createElement('script'); script.src = 'https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js'; - script.onload = () => { - this._threeLoaded = true; - resolve(); + script.onload = () => finish(true); + script.onerror = () => { + console.warn('[SensingTab] Three.js CDN unavailable; using canvas renderer'); + finish(false); }; - script.onerror = () => reject(new Error('Failed to load Three.js')); document.head.appendChild(script); + setTimeout(() => { + console.warn('[SensingTab] Three.js load timed out; using canvas renderer'); + finish(false); + }, 2500); }); } @@ -172,13 +192,17 @@ export class SensingTab { viewport.innerHTML = ''; try { - this.splatRenderer = new GaussianSplatRenderer(viewport, { + const Renderer = window.THREE ? GaussianSplatRenderer : CanvasSensingRenderer; + this.splatRenderer = new Renderer(viewport, { width: viewport.clientWidth, height: viewport.clientHeight || 500, }); } catch (e) { console.error('[SensingTab] Failed to init splat renderer:', e); - viewport.innerHTML = '
3D rendering unavailable
'; + this.splatRenderer = new CanvasSensingRenderer(viewport, { + width: viewport.clientWidth, + height: viewport.clientHeight || 500, + }); } } @@ -210,15 +234,22 @@ export class SensingTab { const banner = this.container.querySelector('#sensingSourceBanner'); if (dot && text) { + const dataSource = sensingService.dataSource; const stateLabels = { disconnected: 'Disconnected', connecting: 'Connecting...', connected: 'Connected', reconnecting: 'Reconnecting...', - simulated: 'Simulated', }; - dot.className = 'sensing-dot ' + state; - text.textContent = stateLabels[state] || state; + const sourceLabels = { + live: 'Live', + stale: 'Stale', + }; + const displayState = dataSource === 'live' ? 'connected' + : dataSource === 'stale' ? 'reconnecting' + : state; + dot.className = 'sensing-dot ' + displayState; + text.textContent = sourceLabels[dataSource] || stateLabels[state] || state; } if (banner) { @@ -226,9 +257,8 @@ export class SensingTab { const dataSource = sensingService.dataSource; const bannerConfig = { 'live': { text: 'LIVE \u2014 ESP32 HARDWARE', cls: 'sensing-source-live' }, - 'server-simulated': { text: 'SIMULATED \u2014 NO HARDWARE', cls: 'sensing-source-server-sim' }, + 'stale': { text: 'STALE \u2014 FEATURE STATE', cls: 'sensing-source-stale' }, 'reconnecting': { text: 'RECONNECTING...', cls: 'sensing-source-reconnecting' }, - 'simulated': { text: 'OFFLINE \u2014 CLIENT SIMULATION', cls: 'sensing-source-simulated' }, }; const cfg = bannerConfig[dataSource] || bannerConfig.reconnecting; banner.textContent = cfg.text; @@ -248,31 +278,35 @@ export class SensingTab { if (countEl) countEl.textContent = String(nodeCount); // RSSI - this._setText('sensingRssi', `${(f.mean_rssi || -80).toFixed(1)} dBm`); - this._setText('sensingSource', data.source || ''); + const meanRssi = this._finiteNumber(f.mean_rssi); + this._setText('sensingRssi', meanRssi == null ? '-- dBm' : `${meanRssi.toFixed(1)} dBm`); + this._setText('sensingSource', this._sourceLabel(data.source)); // Bars (scale to 0-100%) - this._setBar('barVariance', f.variance, 10, 'valVariance', f.variance); - this._setBar('barMotion', f.motion_band_power, 0.5, 'valMotion', f.motion_band_power); - this._setBar('barBreath', f.breathing_band_power, 0.3, 'valBreath', f.breathing_band_power); - this._setBar('barSpectral', f.spectral_power, 2.0, 'valSpectral', f.spectral_power); + this._setBar('barVariance', f.variance, 10, 'valVariance', this._formatMetric(f.variance)); + this._setBar('barMotion', f.motion_band_power, 0.5, 'valMotion', this._formatMetric(f.motion_band_power)); + this._setBar('barBreath', f.breathing_band_power, 0.3, 'valBreath', this._formatMetric(f.breathing_band_power)); + this._setBar('barSpectral', f.spectral_power, 2.0, 'valSpectral', this._formatMetric(f.spectral_power)); // Classification const label = this.container.querySelector('#classLabel'); if (label) { - const level = (c.motion_level || 'absent').toUpperCase(); + const levelRaw = typeof c.motion_level === 'string' && c.motion_level ? c.motion_level : 'unknown'; + const level = levelRaw.toUpperCase(); label.textContent = level; - label.className = 'sensing-class-label ' + (c.motion_level || 'absent'); + label.className = 'sensing-class-label ' + levelRaw; } - const confPct = ((c.confidence || 0) * 100).toFixed(0); - this._setBar('barConfidence', c.confidence, 1.0, 'valConfidence', confPct + '%'); + const confidence = this._finiteNumber(c.confidence); + const confPct = confidence == null ? '--' : `${(confidence * 100).toFixed(0)}%`; + this._setBar('barConfidence', confidence, 1.0, 'valConfidence', confPct); // Details - this._setText('valDomFreq', (f.dominant_freq_hz || 0).toFixed(3) + ' Hz'); - this._setText('valChangePoints', String(f.change_points || 0)); - const srcLabel = (data.source === 'simulated' || data.source === 'simulate') ? 'sim' : data.source || 'live'; - this._setText('valSampleRate', srcLabel); + const dominantFreq = this._finiteNumber(f.dominant_freq_hz); + const changePoints = this._finiteNumber(f.change_points); + this._setText('valDomFreq', dominantFreq == null ? '--' : `${dominantFreq.toFixed(3)} Hz`); + this._setText('valChangePoints', changePoints == null ? '--' : String(changePoints)); + this._setText('valSampleRate', this._sourceLabel(data.source)); // Sparkline this._drawSparkline(); @@ -286,7 +320,8 @@ export class SensingTab { _setBar(barId, value, maxVal, valId, displayVal) { const bar = this.container.querySelector('#' + barId); if (bar) { - const pct = Math.min(100, Math.max(0, ((value || 0) / maxVal) * 100)); + const numericValue = this._finiteNumber(value) ?? 0; + const pct = Math.min(100, Math.max(0, (numericValue / maxVal) * 100)); bar.style.width = pct + '%'; } if (valId && displayVal != null) { @@ -295,6 +330,21 @@ export class SensingTab { } } + _finiteNumber(value) { + const number = Number(value); + return Number.isFinite(number) ? number : null; + } + + _formatMetric(value) { + const number = this._finiteNumber(value); + return number == null ? '--' : number.toFixed(3); + } + + _sourceLabel(source) { + if (!source) return '--'; + return source; + } + _drawSparkline() { const canvas = this.container.querySelector('#sensingSparkline'); if (!canvas) return; @@ -400,3 +450,154 @@ export class SensingTab { sensingService.stop(); } } + +class CanvasSensingRenderer { + constructor(container, opts = {}) { + this.container = container; + this.canvas = document.createElement('canvas'); + this.ctx = this.canvas.getContext('2d'); + this.width = opts.width || container.clientWidth || 800; + this.height = opts.height || 500; + this._lastData = null; + this._animFrame = null; + this.resize(this.width, this.height); + container.appendChild(this.canvas); + this._animate(); + } + + update(data) { + this._lastData = data; + } + + resize(width, height) { + this.width = Math.max(240, Math.floor(width || this.container.clientWidth || 800)); + this.height = Math.max(240, Math.floor(height || this.container.clientHeight || 500)); + const dpr = Math.min(window.devicePixelRatio || 1, 2); + this.canvas.width = Math.floor(this.width * dpr); + this.canvas.height = Math.floor(this.height * dpr); + this.canvas.style.width = `${this.width}px`; + this.canvas.style.height = `${this.height}px`; + this.ctx.setTransform(dpr, 0, 0, dpr, 0, 0); + } + + dispose() { + if (this._animFrame) cancelAnimationFrame(this._animFrame); + if (this.canvas.parentNode) this.canvas.parentNode.removeChild(this.canvas); + } + + _animate() { + this._animFrame = requestAnimationFrame(() => this._animate()); + this._draw(); + } + + _draw() { + const ctx = this.ctx; + const data = this._lastData; + ctx.clearRect(0, 0, this.width, this.height); + this._drawBackground(ctx); + this._drawSignalField(ctx, data); + this._drawNodes(ctx, data); + this._drawPresence(ctx, data); + this._drawOverlay(ctx, data); + } + + _drawBackground(ctx) { + const gradient = ctx.createLinearGradient(0, 0, this.width, this.height); + gradient.addColorStop(0, '#071015'); + gradient.addColorStop(1, '#101421'); + ctx.fillStyle = gradient; + ctx.fillRect(0, 0, this.width, this.height); + + ctx.strokeStyle = 'rgba(50, 184, 198, 0.14)'; + ctx.lineWidth = 1; + const grid = 10; + for (let i = 0; i <= grid; i++) { + const x = (i / grid) * this.width; + const y = (i / grid) * this.height; + ctx.beginPath(); + ctx.moveTo(x, 0); + ctx.lineTo(x, this.height); + ctx.stroke(); + ctx.beginPath(); + ctx.moveTo(0, y); + ctx.lineTo(this.width, y); + ctx.stroke(); + } + } + + _drawSignalField(ctx, data) { + const values = data?.signal_field?.values; + const size = Array.isArray(data?.signal_field?.grid_size) + ? data.signal_field.grid_size[0] + : 20; + if (!Array.isArray(values) || values.length === 0) return; + + const cellW = this.width / size; + const cellH = this.height / size; + for (let y = 0; y < size; y++) { + for (let x = 0; x < size; x++) { + const value = Math.max(0, Math.min(1, Number(values[y * size + x]) || 0)); + const hue = 205 - value * 190; + const alpha = 0.18 + value * 0.62; + ctx.fillStyle = `hsla(${hue}, 90%, ${32 + value * 34}%, ${alpha})`; + ctx.fillRect(x * cellW, y * cellH, cellW + 1, cellH + 1); + } + } + } + + _drawNodes(ctx, data) { + const nodes = Array.isArray(data?.nodes) ? data.nodes : []; + const colors = ['#32b8c6', '#ff8a3d', '#35d88f', '#d96cff']; + nodes.forEach((node, index) => { + const point = this._roomToCanvas(node.position || [0, 0, 0]); + ctx.beginPath(); + ctx.arc(point.x, point.y, 9, 0, Math.PI * 2); + ctx.fillStyle = colors[index % colors.length]; + ctx.fill(); + ctx.lineWidth = 2; + ctx.strokeStyle = 'rgba(255,255,255,0.85)'; + ctx.stroke(); + ctx.fillStyle = '#f5f7fb'; + ctx.font = '11px sans-serif'; + ctx.fillText(`Node ${node.node_id}`, point.x + 12, point.y + 4); + }); + } + + _drawPresence(ctx, data) { + if (!data?.classification?.presence) return; + const confidence = Math.max(0, Math.min(1, Number(data.classification.confidence) || 0)); + const motion = Math.max(0, Math.min(1, Number(data.features?.motion_band_power) || 0)); + const pulse = 0.7 + Math.sin(Date.now() * 0.006) * 0.18; + const radius = (52 + motion * 80) * pulse; + const x = this.width * (0.5 + Math.sin(Date.now() * 0.00045) * 0.11); + const y = this.height * (0.52 + Math.cos(Date.now() * 0.00033) * 0.08); + const gradient = ctx.createRadialGradient(x, y, 4, x, y, radius); + gradient.addColorStop(0, `rgba(255, 80, 62, ${0.35 + confidence * 0.35})`); + gradient.addColorStop(0.5, `rgba(53, 216, 143, ${0.18 + confidence * 0.22})`); + gradient.addColorStop(1, 'rgba(50, 184, 198, 0)'); + ctx.fillStyle = gradient; + ctx.beginPath(); + ctx.arc(x, y, radius, 0, Math.PI * 2); + ctx.fill(); + } + + _drawOverlay(ctx, data) { + const label = data + ? `${(data.classification?.motion_level || 'live').toUpperCase()} ${(data.features?.mean_rssi ?? 0).toFixed(0)} dBm` + : 'WAITING FOR LIVE CSI'; + ctx.fillStyle = 'rgba(0, 0, 0, 0.42)'; + ctx.fillRect(14, 14, 220, 34); + ctx.fillStyle = '#e8f9fb'; + ctx.font = '13px sans-serif'; + ctx.fillText(label, 24, 36); + } + + _roomToCanvas(position) { + const x = Array.isArray(position) ? Number(position[0]) || 0 : 0; + const z = Array.isArray(position) ? Number(position[2]) || 0 : 0; + return { + x: this.width * (0.5 + x / 20), + y: this.height * (0.5 + z / 20), + }; + } +} diff --git a/ui/components/TrainingPanel.js b/ui/components/TrainingPanel.js index d3b8f6d57d..d4b56ae792 100644 --- a/ui/components/TrainingPanel.js +++ b/ui/components/TrainingPanel.js @@ -4,56 +4,74 @@ import { trainingService } from '../services/training.service.js'; const TP_STYLES = ` -.tp-panel{background:rgba(17,24,39,.9);border:1px solid rgba(56,68,89,.6);border-radius:8px;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;color:#e0e0e0;overflow:hidden} -.tp-header{display:flex;align-items:center;justify-content:space-between;padding:14px 16px;background:rgba(13,17,23,.95);border-bottom:1px solid rgba(56,68,89,.6)} -.tp-title{font-size:14px;font-weight:600;color:#e0e0e0} -.tp-badge{font-size:11px;font-weight:600;padding:2px 8px;border-radius:10px} -.tp-badge-idle{background:rgba(108,117,125,.2);color:#8899aa;border:1px solid rgba(108,117,125,.3)} -.tp-badge-active{background:rgba(40,167,69,.2);color:#51cf66;border:1px solid rgba(40,167,69,.3);animation:tp-pulse 1.5s ease-in-out infinite} -.tp-badge-done{background:rgba(102,126,234,.2);color:#8ea4f0;border:1px solid rgba(102,126,234,.3)} -@keyframes tp-pulse{0%,100%{opacity:1}50%{opacity:.6}} -.tp-error{background:rgba(220,53,69,.15);color:#f5a0a8;border:1px solid rgba(220,53,69,.3);border-radius:4px;padding:8px 12px;margin:10px 12px 0;font-size:12px} -.tp-section{padding:12px;border-bottom:1px solid rgba(56,68,89,.3)} +.tp-panel{background:var(--color-surface);border:1px solid var(--color-card-border);border-radius:8px;color:var(--color-text);overflow:hidden} +.tp-header{display:flex;align-items:center;justify-content:space-between;padding:14px 16px;border-bottom:1px solid var(--color-card-border-inner);background:var(--color-background)} +.tp-title{font-size:14px;font-weight:650;color:var(--color-text)} +.tp-badge{font-size:11px;font-weight:650;padding:3px 9px;border-radius:999px;border:1px solid var(--color-border);text-transform:uppercase} +.tp-badge-idle{background:var(--color-secondary);color:var(--color-text-secondary)} +.tp-badge-active{background:rgba(var(--color-success-rgb),.14);color:var(--color-success);border-color:rgba(var(--color-success-rgb),.3);animation:tp-pulse 1.5s ease-in-out infinite} +.tp-badge-done{background:rgba(var(--color-primary-rgb,33,128,141),.14);color:var(--color-primary);border-color:rgba(var(--color-primary-rgb,33,128,141),.3)} +@keyframes tp-pulse{0%,100%{opacity:1}50%{opacity:.65}} +.tp-error{background:rgba(var(--color-error-rgb),.1);color:var(--color-error);border:1px solid rgba(var(--color-error-rgb),.28);border-radius:6px;padding:9px 12px;margin:12px 14px 0;font-size:12px} +.tp-section{padding:14px 16px;border-bottom:1px solid var(--color-card-border-inner)} .tp-section:last-child{border-bottom:none} -.tp-section-title{font-size:11px;font-weight:600;text-transform:uppercase;letter-spacing:.5px;color:#8899aa;margin-bottom:8px} -.tp-empty{color:#6b7a8d;font-size:12px;padding:12px 0;text-align:center} -.tp-rec-row{display:flex;align-items:center;justify-content:space-between;padding:6px 8px;margin-bottom:4px;background:rgba(13,17,23,.6);border:1px solid rgba(56,68,89,.3);border-radius:4px} -.tp-rec-info{display:flex;flex-direction:column;gap:2px} -.tp-rec-name{font-size:12px;color:#c8d0dc;font-weight:500} -.tp-rec-meta{font-size:10px;color:#6b7a8d} -.tp-rec-actions{margin-top:8px} -.tp-config-header{display:flex;align-items:center;justify-content:space-between;margin-bottom:6px} -.tp-config-form{display:flex;flex-direction:column;gap:6px} -.tp-label{font-size:12px;color:#8899aa;display:block;margin-bottom:2px} -.tp-input-row{display:flex;justify-content:space-between;align-items:center;gap:8px} -.tp-input-row .tp-label{flex:1;margin-bottom:0} -.tp-input{width:110px;padding:4px 8px;background:rgba(30,40,60,.8);border:1px solid rgba(56,68,89,.6);border-radius:4px;color:#c8d0dc;font-size:12px} -.tp-input:focus{outline:none;border-color:#667eea} -.tp-ds-container{display:flex;flex-direction:column;gap:4px;margin-bottom:4px;max-height:100px;overflow-y:auto} -.tp-ds-item{display:flex;align-items:center;gap:6px;font-size:12px;color:#c8d0dc;cursor:pointer} -.tp-ds-item input{width:14px;height:14px} -.tp-train-actions{display:flex;gap:6px;margin-top:10px} -.tp-progress-bar{height:6px;background:rgba(30,40,60,.8);border-radius:3px;overflow:hidden;margin-bottom:4px} -.tp-progress-fill{height:100%;background:linear-gradient(90deg,#667eea,#764ba2);border-radius:3px;transition:width .3s} -.tp-progress-label{font-size:11px;color:#8899aa;text-align:center;margin-bottom:10px} -.tp-chart-row{display:flex;gap:8px;margin-bottom:10px;flex-wrap:wrap} -.tp-chart-row canvas{border:1px solid rgba(56,68,89,.4);border-radius:4px;flex:1;min-width:120px} -.tp-metrics-grid{display:grid;grid-template-columns:1fr 1fr;gap:6px} -.tp-metric-cell{background:rgba(13,17,23,.6);border:1px solid rgba(56,68,89,.3);border-radius:4px;padding:6px 8px} -.tp-metric-label{font-size:10px;color:#6b7a8d;text-transform:uppercase;letter-spacing:.3px} -.tp-metric-value{font-size:13px;color:#c8d0dc;font-weight:500;margin-top:2px} -.tp-btn{padding:5px 12px;border-radius:4px;font-size:12px;font-weight:500;cursor:pointer;border:1px solid transparent;transition:all .15s} +.tp-section-title{font-size:11px;font-weight:700;text-transform:uppercase;letter-spacing:0;color:var(--color-text-secondary);margin-bottom:10px} +.tp-readme{display:grid;gap:10px;padding:14px 16px;border-bottom:1px solid var(--color-card-border-inner);background:var(--color-background)} +.tp-readme-title{font-size:13px;font-weight:700;color:var(--color-text)} +.tp-readme-copy{font-size:12px;line-height:1.45;color:var(--color-text-secondary);margin:0} +.tp-readme-list{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:8px;margin:0;padding:0;list-style:none} +.tp-readme-list li{font-size:12px;color:var(--color-text);background:var(--color-surface);border:1px solid var(--color-card-border-inner);border-radius:6px;padding:8px 9px;min-width:0} +.tp-empty{color:var(--color-text-secondary);font-size:12px;padding:14px 0;text-align:center} +.tp-rec-row{display:grid;grid-template-columns:minmax(0,1fr) auto;align-items:center;gap:10px;padding:9px 10px;margin-bottom:8px;background:var(--color-background);border:1px solid var(--color-card-border-inner);border-radius:6px} +.tp-rec-info{display:flex;flex-direction:column;gap:2px;min-width:0} +.tp-rec-name{font-size:12px;color:var(--color-text);font-weight:600;overflow:hidden;text-overflow:ellipsis;white-space:nowrap} +.tp-rec-meta{font-size:11px;color:var(--color-text-secondary)} +.tp-rec-actions{margin-top:10px} +.tp-config-header{display:flex;align-items:center;justify-content:space-between;margin-bottom:8px} +.tp-config-form{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:10px 12px} +.tp-label{font-size:12px;color:var(--color-text-secondary);display:block;margin-bottom:4px} +.tp-input-row{display:flex;flex-direction:column;gap:4px} +.tp-input{width:100%;padding:8px 10px;background:var(--color-background);border:1px solid var(--color-border);border-radius:6px;color:var(--color-text);font-size:13px} +.tp-input:focus{outline:none;border-color:var(--color-primary);box-shadow:var(--focus-ring)} +.tp-ds-container{grid-column:1/-1;display:grid;grid-template-columns:repeat(auto-fit,minmax(160px,1fr));gap:8px;margin-bottom:2px;max-height:132px;overflow-y:auto} +.tp-ds-item{display:flex;align-items:center;gap:8px;padding:7px 9px;border:1px solid var(--color-card-border-inner);border-radius:6px;background:var(--color-background);font-size:12px;color:var(--color-text);cursor:pointer;min-width:0} +.tp-ds-item span{overflow:hidden;text-overflow:ellipsis;white-space:nowrap} +.tp-ds-item input{width:14px;height:14px;flex:0 0 auto} +.tp-train-actions{display:flex;gap:8px;margin-top:12px;flex-wrap:wrap} +.tp-progress-bar{height:8px;background:var(--color-secondary);border-radius:999px;overflow:hidden;margin-bottom:6px} +.tp-progress-fill{height:100%;background:var(--color-primary);border-radius:999px;transition:width .3s} +.tp-progress-label{font-size:12px;color:var(--color-text-secondary);text-align:right;margin-bottom:12px} +.tp-chart-row{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:10px;margin-bottom:12px} +.tp-chart-row canvas{border:1px solid var(--color-card-border-inner);border-radius:6px;width:100%;min-width:0;background:var(--color-background)} +.tp-metrics-grid{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:8px} +.tp-metric-cell{background:var(--color-background);border:1px solid var(--color-card-border-inner);border-radius:6px;padding:8px 10px;min-width:0} +.tp-metric-label{font-size:10px;color:var(--color-text-secondary);text-transform:uppercase;letter-spacing:0} +.tp-metric-value{font-size:14px;color:var(--color-text);font-weight:650;margin-top:2px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap} +.tp-rvf-intro{font-size:12px;line-height:1.45;color:var(--color-text-secondary);margin:0 0 10px} +.tp-rvf-grid{display:grid;grid-template-columns:repeat(6,minmax(0,1fr));gap:8px;margin-bottom:12px} +.tp-rvf-status{display:inline-flex;align-items:center;gap:6px;font-size:11px;font-weight:650;padding:4px 8px;border-radius:999px;border:1px solid var(--color-border);color:var(--color-text-secondary);margin-bottom:10px} +.tp-rvf-status-ready{background:rgba(var(--color-success-rgb),.12);color:var(--color-success);border-color:rgba(var(--color-success-rgb),.25)} +.tp-rvf-status-warn{background:rgba(var(--color-warning-rgb,168,75,47),.12);color:var(--color-warning,#a84b2f);border-color:rgba(var(--color-warning-rgb,168,75,47),.25)} +.tp-rvf-files{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:8px;margin-bottom:12px} +.tp-file-cell{background:var(--color-background);border:1px solid var(--color-card-border-inner);border-radius:6px;padding:8px 10px;min-width:0} +.tp-file-label{font-size:10px;color:var(--color-text-secondary);text-transform:uppercase;letter-spacing:0} +.tp-file-value{font-size:12px;color:var(--color-text);font-weight:600;margin-top:3px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap} +.tp-cmd-list{display:grid;gap:8px} +.tp-cmd-row{display:grid;grid-template-columns:minmax(0,1fr) auto;gap:8px;align-items:start;background:var(--color-background);border:1px solid var(--color-card-border-inner);border-radius:6px;padding:9px 10px} +.tp-cmd-label{font-size:11px;font-weight:700;color:var(--color-text);margin-bottom:5px} +.tp-cmd-code{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:11px;line-height:1.45;color:var(--color-text-secondary);white-space:pre-wrap;word-break:break-word} +.tp-btn{padding:7px 12px;border-radius:6px;font-size:12px;font-weight:650;cursor:pointer;border:1px solid transparent;transition:background .15s,border-color .15s,color .15s} .tp-btn:disabled{opacity:.5;cursor:not-allowed} -.tp-btn-success{background:rgba(40,167,69,.2);color:#51cf66;border-color:rgba(40,167,69,.3)} -.tp-btn-success:hover:not(:disabled){background:rgba(40,167,69,.35)} -.tp-btn-danger{background:rgba(220,53,69,.2);color:#ff6b6b;border-color:rgba(220,53,69,.3)} -.tp-btn-danger:hover:not(:disabled){background:rgba(220,53,69,.35)} -.tp-btn-secondary{background:rgba(30,40,60,.8);color:#b0b8c8;border-color:rgba(56,68,89,.6)} -.tp-btn-secondary:hover:not(:disabled){background:rgba(40,50,75,.9)} -.tp-btn-rec{background:rgba(220,53,69,.15);color:#ff6b6b;border-color:rgba(220,53,69,.3)} -.tp-btn-rec:hover:not(:disabled){background:rgba(220,53,69,.3)} -.tp-btn-muted{background:transparent;color:#6b7a8d;border-color:rgba(56,68,89,.4);font-size:11px;padding:3px 8px} -.tp-btn-muted:hover:not(:disabled){color:#b0b8c8;border-color:rgba(56,68,89,.8)} +.tp-btn-success{background:var(--color-primary);color:var(--color-btn-primary-text);border-color:var(--color-primary)} +.tp-btn-success:hover:not(:disabled){background:var(--color-primary-hover)} +.tp-btn-danger{background:rgba(var(--color-error-rgb),.1);color:var(--color-error);border-color:rgba(var(--color-error-rgb),.28)} +.tp-btn-danger:hover:not(:disabled){background:rgba(var(--color-error-rgb),.18)} +.tp-btn-secondary,.tp-btn-rec{background:var(--color-secondary);color:var(--color-text);border-color:var(--color-border)} +.tp-btn-secondary:hover:not(:disabled),.tp-btn-rec:hover:not(:disabled){background:var(--color-secondary-hover)} +.tp-btn-muted{background:transparent;color:var(--color-text-secondary);border-color:var(--color-border);font-size:11px;padding:4px 8px} +.tp-btn-muted:hover:not(:disabled){color:var(--color-text);border-color:var(--color-primary)} +@media(max-width:980px){.tp-rvf-grid{grid-template-columns:repeat(2,minmax(0,1fr))}.tp-rvf-files{grid-template-columns:1fr}} +@media(max-width:760px){.tp-config-form,.tp-chart-row,.tp-metrics-grid,.tp-readme-list,.tp-rvf-grid{grid-template-columns:1fr}.tp-progress-label{text-align:left}.tp-cmd-row{grid-template-columns:1fr}} `; export default class TrainingPanel { @@ -64,6 +82,7 @@ export default class TrainingPanel { this.state = { recordings: [], trainingStatus: null, isRecording: false, + rvfReadiness: null, copiedCommand: null, configOpen: true, loading: false, error: null }; this.config = { @@ -100,12 +119,14 @@ export default class TrainingPanel { async refresh() { this._set({ loading: true, error: null }); try { - const [recordings, status] = await Promise.all([ + const [recordings, status, rvfReadiness] = await Promise.all([ trainingService.listRecordings().catch(() => []), - trainingService.getTrainingStatus().catch(() => null) + trainingService.getTrainingStatus().catch(() => null), + trainingService.getRvfReadiness().catch(() => null) ]); if (status && !status.active) this.progressData = { losses: [], pcks: [] }; - this._set({ recordings, trainingStatus: status, loading: false }); + const isRecording = recordings.some(rec => rec.status === 'recording'); + this._set({ recordings, trainingStatus: status, rvfReadiness, isRecording, loading: false }); } catch (e) { this._set({ loading: false, error: e.message }); } } @@ -142,7 +163,6 @@ export default class TrainingPanel { this._set({ loading: true, error: null }); this.progressData = { losses: [], pcks: [] }; try { - trainingService.connectProgressStream(); const payload = { dataset_ids: this.config.selectedRecordings, config: { @@ -152,7 +172,10 @@ export default class TrainingPanel { ...extraCfg } }; - await trainingService[method](payload); + const data = await trainingService[method](payload); + if (data?.active !== false && data?.status !== 'completed') { + trainingService.connectProgressStream(); + } await this.refresh(); } catch (e) { this._set({ loading: false, error: `Training failed: ${e.message}` }); } } @@ -172,8 +195,10 @@ export default class TrainingPanel { el.innerHTML = ''; const panel = this._el('div', 'tp-panel'); panel.appendChild(this._renderHeader()); + panel.appendChild(this._renderReadme()); if (this.state.error) panel.appendChild(this._el('div', 'tp-error', this.state.error)); panel.appendChild(this._renderRecordings()); + panel.appendChild(this._renderRvfWorkflow()); const ts = this.state.trainingStatus; const active = ts && ts.active; if (active) panel.appendChild(this._renderProgress()); @@ -194,6 +219,24 @@ export default class TrainingPanel { return h; } + _renderReadme() { + const s = this._el('div', 'tp-readme'); + s.appendChild(this._el('div', 'tp-readme-title', 'Training README')); + s.appendChild(this._el('p', 'tp-readme-copy', + 'This page turns live RuView CSI packets into reusable datasets, then uses those datasets to request pose-model training and manage exported .rvf model files.' + )); + const list = this._el('ul', 'tp-readme-list'); + [ + 'Record: capture live CSI frames into data/recordings as .csi.jsonl sessions.', + 'Select: choose recorded sessions as the dataset for supervised, pretraining, or LoRA runs.', + 'Train: send normalized training requests and show loss, PCK, OKS, learning rate, and phase.', + 'Models: list, load, unload, delete, and inspect .rvf model files from data/models.', + 'How data is used: recordings become datasets; training output becomes .rvf models; loaded models can consume live features for inference.' + ].forEach(item => list.appendChild(this._el('li', null, item))); + s.appendChild(list); + return s; + } + _renderRecordings() { const s = this._el('div', 'tp-section'); s.appendChild(this._el('div', 'tp-section-title', 'CSI Recordings')); @@ -228,6 +271,72 @@ export default class TrainingPanel { return s; } + _renderRvfWorkflow() { + const data = this.state.rvfReadiness; + const s = this._el('div', 'tp-section'); + s.appendChild(this._el('div', 'tp-section-title', 'Real .rvf Training')); + if (!data) { + s.appendChild(this._el('div', 'tp-empty', 'RVF readiness unavailable')); + return s; + } + + const sum = data.summary || {}; + const ready = data.status === 'ready'; + const status = this._el('div', `tp-rvf-status ${ready ? 'tp-rvf-status-ready' : 'tp-rvf-status-warn'}`, + ready ? 'Ready to train from paired data' : 'Needs camera labels or paired data' + ); + s.appendChild(status); + s.appendChild(this._el('p', 'tp-rvf-intro', + 'Use camera labels only while collecting ground truth, align them with CSI, then export a real .rvf into data/models for model inference.' + )); + + const metric = (label, value) => { + const cell = this._el('div', 'tp-metric-cell'); + cell.appendChild(this._el('div', 'tp-metric-label', label)); + cell.appendChild(this._el('div', 'tp-metric-value', String(value ?? '--'))); + return cell; + }; + const grid = this._el('div', 'tp-rvf-grid'); + grid.appendChild(metric('Live Nodes', `${sum.live_nodes ?? 0}/${sum.recommended_nodes ?? 4}`)); + grid.appendChild(metric('CSI', sum.recordings ?? 0)); + grid.appendChild(metric('Labels', sum.ground_truth ?? 0)); + grid.appendChild(metric('Paired', sum.paired ?? 0)); + grid.appendChild(metric('Real RVF', sum.real_rvf ?? 0)); + grid.appendChild(metric('Placeholders', sum.placeholder_rvf ?? 0)); + s.appendChild(grid); + + const latest = data.latest || {}; + const files = this._el('div', 'tp-rvf-files'); + [ + ['Latest CSI', latest.recording?.name || latest.recording?.id || 'none'], + ['Latest Labels', latest.ground_truth?.name || 'none'], + ['Latest Paired', latest.paired?.name || 'none'] + ].forEach(([label, value]) => { + const cell = this._el('div', 'tp-file-cell'); + cell.appendChild(this._el('div', 'tp-file-label', label)); + cell.appendChild(this._el('div', 'tp-file-value', value)); + files.appendChild(cell); + }); + s.appendChild(files); + + const commands = this._el('div', 'tp-cmd-list'); + (data.commands || []).forEach(cmd => commands.appendChild(this._renderCommand(cmd))); + s.appendChild(commands); + return s; + } + + _renderCommand(cmd) { + const row = this._el('div', 'tp-cmd-row'); + const body = this._el('div'); + body.appendChild(this._el('div', 'tp-cmd-label', cmd.label || cmd.id || 'Command')); + body.appendChild(this._el('div', 'tp-cmd-code', cmd.command || '')); + row.appendChild(body); + const copied = this.state.copiedCommand === cmd.id; + const copy = this._btn(copied ? 'Copied' : 'Copy', 'tp-btn tp-btn-muted', () => this._copyCommand(cmd)); + row.appendChild(copy); + return row; + } + _renderConfig() { const s = this._el('div', 'tp-section'); const hdr = this._el('div', 'tp-config-header'); @@ -340,8 +449,8 @@ export default class TrainingPanel { // --- Chart drawing --- _drawCharts() { - this._drawChart('tp-loss-chart', this.progressData.losses, { color: '#ff6b6b', label: 'Loss', yMin: 0, yMax: null }); - this._drawChart('tp-pck-chart', this.progressData.pcks, { color: '#51cf66', label: 'PCK', yMin: 0, yMax: 1 }); + this._drawChart('tp-loss-chart', this.progressData.losses, { color: '#c0152f', label: 'Loss', yMin: 0, yMax: null }); + this._drawChart('tp-pck-chart', this.progressData.pcks, { color: '#21808d', label: 'PCK', yMin: 0, yMax: 1 }); } _drawChart(id, data, opts) { @@ -349,23 +458,23 @@ export default class TrainingPanel { if (!cv) return; const ctx = cv.getContext('2d'), w = cv.width, h = cv.height; const p = { t: 20, r: 10, b: 24, l: 44 }; - ctx.fillStyle = '#0d1117'; ctx.fillRect(0, 0, w, h); - ctx.fillStyle = '#8899aa'; ctx.font = '11px -apple-system,sans-serif'; ctx.fillText(opts.label, p.l, 14); - if (!data.length) { ctx.fillStyle = '#6b7a8d'; ctx.fillText('No data', w / 2 - 20, h / 2); return; } + ctx.fillStyle = '#f7f7f4'; ctx.fillRect(0, 0, w, h); + ctx.fillStyle = '#626c71'; ctx.font = '11px -apple-system,sans-serif'; ctx.fillText(opts.label, p.l, 14); + if (!data.length) { ctx.fillStyle = '#626c71'; ctx.fillText('No data', w / 2 - 20, h / 2); return; } const pw = w - p.l - p.r, ph = h - p.t - p.b; let yMin = opts.yMin ?? Math.min(...data), yMax = opts.yMax ?? Math.max(...data); if (yMax === yMin) yMax = yMin + 1; - ctx.strokeStyle = 'rgba(255,255,255,.08)'; ctx.lineWidth = 1; + ctx.strokeStyle = 'rgba(94,82,64,.18)'; ctx.lineWidth = 1; for (let i = 0; i <= 4; i++) { const y = p.t + (ph / 4) * i; ctx.beginPath(); ctx.moveTo(p.l, y); ctx.lineTo(w - p.r, y); ctx.stroke(); const v = yMax - ((yMax - yMin) / 4) * i; - ctx.fillStyle = '#6b7a8d'; ctx.font = '9px sans-serif'; ctx.fillText(v.toFixed(v >= 1 ? 2 : 3), 2, y + 3); + ctx.fillStyle = '#626c71'; ctx.font = '9px sans-serif'; ctx.fillText(v.toFixed(v >= 1 ? 2 : 3), 2, y + 3); } const xl = Math.min(data.length, 5); for (let i = 0; i < xl; i++) { const idx = Math.round((data.length - 1) * (i / (xl - 1 || 1))); - ctx.fillStyle = '#6b7a8d'; ctx.fillText(String(idx + 1), p.l + (pw * idx) / (data.length - 1 || 1) - 4, h - 4); + ctx.fillStyle = '#626c71'; ctx.fillText(String(idx + 1), p.l + (pw * idx) / (data.length - 1 || 1) - 4, h - 4); } ctx.strokeStyle = opts.color; ctx.lineWidth = 1.5; ctx.beginPath(); data.forEach((v, i) => { @@ -382,6 +491,18 @@ export default class TrainingPanel { // --- Helpers --- + async _copyCommand(cmd) { + try { + await navigator.clipboard.writeText(cmd.command || ''); + this._set({ copiedCommand: cmd.id || null }); + window.setTimeout(() => { + if (this.state.copiedCommand === cmd.id) this._set({ copiedCommand: null }); + }, 1400); + } catch (e) { + this._set({ error: `Copy failed: ${e.message}` }); + } + } + _el(tag, cls, txt) { const e = document.createElement(tag); if (cls) e.className = cls; diff --git a/ui/components/dashboard-hud.js b/ui/components/dashboard-hud.js index 8349a3778d..4d450026de 100644 --- a/ui/components/dashboard-hud.js +++ b/ui/components/dashboard-hud.js @@ -14,7 +14,7 @@ export class DashboardHUD { fps: 0, confidence: 0, personCount: 0, - sensingMode: 'Mock', // CSI, RSSI, Mock + sensingMode: 'Unavailable', latency: 0, messageCount: 0, uptime: 0 @@ -61,7 +61,7 @@ export class DashboardHUD { text-transform: uppercase; z-index: 110; } - .hud-banner.mock { + .hud-banner.offline { background: linear-gradient(90deg, rgba(180,100,0,0.85) 0%, rgba(200,120,0,0.85) 50%, rgba(180,100,0,0.85) 100%); color: #fff; border-bottom: 2px solid #ff8800; @@ -211,7 +211,8 @@ export class DashboardHUD { border: 1px solid #8800ff; color: #ddaaff; } - .hud-mode-badge.mock { + .hud-mode-badge.offline, + .hud-mode-badge.unavailable { background: rgba(120, 80, 0, 0.7); border: 1px solid #ff8800; color: #ffddaa; @@ -244,7 +245,7 @@ export class DashboardHUD { -
MOCK DATA
+
NO LIVE DATA
@@ -298,7 +299,7 @@ export class DashboardHUD {
-
MOCK
+
OFFLINE
WiFi DensePose
@@ -371,8 +372,8 @@ export class DashboardHUD { this._els.banner.textContent = 'REAL DATA - LIVE STREAM'; this._els.banner.className = 'hud-banner real'; } else { - this._els.banner.textContent = 'MOCK DATA - DEMO MODE'; - this._els.banner.className = 'hud-banner mock'; + this._els.banner.textContent = 'NO LIVE DATA'; + this._els.banner.className = 'hud-banner offline'; } // Connection status @@ -416,7 +417,7 @@ export class DashboardHUD { this._els.confidenceFill.style.background = `hsl(${confHue}, 100%, 45%)`; // Sensing mode - const modeLower = (state.sensingMode || 'Mock').toLowerCase(); + const modeLower = (state.sensingMode || 'Unavailable').toLowerCase(); this._els.modeBadge.textContent = state.sensingMode.toUpperCase(); this._els.modeBadge.className = `hud-mode-badge ${modeLower}`; } diff --git a/ui/components/environment.js b/ui/components/environment.js index 116b218623..aef60cedc9 100644 --- a/ui/components/environment.js +++ b/ui/components/environment.js @@ -415,26 +415,9 @@ export class Environment { } } - // Generate a demo confidence heatmap centered on given positions - static generateDemoHeatmap(personPositions, cols, rows, roomWidth, roomDepth) { - const map = new Float32Array(cols * rows); - const cellW = roomWidth / cols; - const cellD = roomDepth / rows; - - for (const pos of (personPositions || [])) { - for (let r = 0; r < rows; r++) { - for (let c = 0; c < cols; c++) { - const cx = (c + 0.5) * cellW - roomWidth / 2; - const cz = (r + 0.5) * cellD - roomDepth / 2; - const dx = cx - (pos.x || 0); - const dz = cz - (pos.z || 0); - const dist = Math.sqrt(dx * dx + dz * dz); - const conf = Math.exp(-dist * dist * 0.5) * (pos.confidence || 0.8); - map[r * cols + c] = Math.max(map[r * cols + c], conf); - } - } - } - return map; + // Runtime local heatmap synthesis is disabled. + static generateUnavailableHeatmap(personPositions, cols, rows, roomWidth, roomDepth) { + return new Float32Array(cols * rows); } // Animate AP and RX markers (subtle pulse) diff --git a/ui/components/signal-viz.js b/ui/components/signal-viz.js index af60f68e6b..27f59963ce 100644 --- a/ui/components/signal-viz.js +++ b/ui/components/signal-viz.js @@ -412,42 +412,9 @@ export class SignalVisualization { } } - // Generate synthetic demo signal data - static generateDemoData(elapsed) { - const subcarriers = 30; - const dopplerBars = 16; - - // Amplitude: sinusoidal pattern with noise simulating human movement - const amplitude = new Float32Array(subcarriers); - for (let i = 0; i < subcarriers; i++) { - const baseFreq = Math.sin(elapsed * 2 + i * 0.3) * 0.3; - const bodyEffect = Math.sin(elapsed * 0.8 + i * 0.15) * 0.25; - const noise = (Math.random() - 0.5) * 0.1; - amplitude[i] = Math.max(0, Math.min(1, 0.4 + baseFreq + bodyEffect + noise)); - } - - // Phase: linear with perturbations from movement - const phase = new Float32Array(subcarriers); - for (let i = 0; i < subcarriers; i++) { - const linearPhase = (i / subcarriers) * Math.PI * 2; - const bodyPhase = Math.sin(elapsed * 1.5 + i * 0.2) * 0.8; - phase[i] = linearPhase + bodyPhase; - } - - // Doppler: spectral peaks from movement velocity - const doppler = new Float32Array(dopplerBars); - const centerBin = dopplerBars / 2 + Math.sin(elapsed * 0.7) * 3; - for (let i = 0; i < dopplerBars; i++) { - const dist = Math.abs(i - centerBin); - doppler[i] = Math.max(0, Math.exp(-dist * dist * 0.15) * (0.6 + Math.sin(elapsed * 1.2) * 0.3)); - doppler[i] += (Math.random() - 0.5) * 0.05; - doppler[i] = Math.max(0, Math.min(1, doppler[i])); - } - - // Motion energy: pulsating - const motionEnergy = (Math.sin(elapsed * 0.5) + 1) / 2 * 0.7 + 0.15; - - return { amplitude, phase, doppler, motionEnergy }; + // Runtime local signal synthesis is disabled. + static generateUnavailableData() { + return null; } getGroup() { diff --git a/ui/config/api.config.js b/ui/config/api.config.js index a8109182d5..109d9bc6ad 100644 --- a/ui/config/api.config.js +++ b/ui/config/api.config.js @@ -12,10 +12,10 @@ export const API_CONFIG = { WS_PREFIX: 'ws://', WSS_PREFIX: 'wss://', - // Mock server configuration (only for testing) + // Runtime mock data is disabled; unavailable backends stay unavailable. MOCK_SERVER: { - ENABLED: false, // Set to true only for testing without backend - AUTO_DETECT: false, // Disabled — sensing tab uses its own WebSocket on :8765 + ENABLED: false, + AUTO_DETECT: false, }, // API Endpoints @@ -132,4 +132,4 @@ export function buildWsUrl(endpoint, params = {}) { } return url; -} \ No newline at end of file +} diff --git a/ui/index.html b/ui/index.html index 857ebf2f60..8a33bcfa13 100644 --- a/ui/index.html +++ b/ui/index.html @@ -13,7 +13,7 @@ - +
@@ -29,9 +29,9 @@

WiFi DensePose

-
+
+ +
+
+

Node Status

+ Waiting for packets +
+
+
+

System Metrics

@@ -148,41 +167,41 @@

Zone Occupancy

🏠

Through Walls

-

Works through solid barriers with no line of sight required

+

Reports only the current sensing state from connected hardware.

🔒

Privacy-Preserving

-

No cameras or visual recording - just WiFi signal analysis

+

No camera feed is shown on this dashboard.

⚡
-

Real-Time

-

Maps 24 body regions in real-time at 100Hz sampling rate

+

Live Only

+

Disconnected or stale sources are shown as unavailable.

💰
-

Low Cost

-

Built using $30 commercial WiFi hardware

+

Local Runtime

+

Reads from the local RuView server on this machine.

- 24 - Body Regions + -- + Live Keypoints
- 100Hz - Sampling Rate + -- + Live Sample Rate
- 87.2% - Accuracy (AP@50) + -- + Session Accuracy
- $30 - Hardware Cost + -- + Active Nodes
@@ -191,22 +210,32 @@

Low Cost

- -