hx711s: make frame validity explicit - #5
Draft
pdscomp wants to merge 11 commits into
Draft
Conversation
Transmit raw channel counts with MCU-owned frame quality metadata, use deterministic reset and settling qualification, and abort an active load-cell probe immediately on invalid acquisition. Remove host-side held-value synthesis and the divergent fixed 100000-count classifier so invalid frames cannot reach tare, filters, or trigger decisions. Also use a 30-sample median tare baseline, require two consecutive qualified filtered samples to trigger, repair command-time SOS filter updates, and harden fixed-point bounds.
A four-channel HX711 frame occupies 20 bytes while the shared sensor_bulk buffer holds 51 bytes. After two frames, appending another before checking capacity writes past the end of the buffer and can corrupt MCU state. Flush an existing packet before appending a frame that would not fit.
Frame faults are handled from task context while the polling timer remains linked in the MCU scheduler. Adding that same timer again corrupts the timer list and eventually manifests as serial garbage, disconnects, or Invalid oid type shutdowns. Remove the timer before changing its wake time and re-adding it, and use the same reset path for command-started acquisition.
Make the multi-channel HX711 path fail closed on invalid frames and recover without corrupting MCU state. Timestamp each frame, validate the wire format, honor HX711 ready-line timing, close stop and scheduler races, and latch terminal probe outcomes. Preserve timestamped fault details through Kalico, exclude recovery frames from tare failures, use summed channel ranges, and expose health counters. Add rate-independent trigger confirmation, focused regression tests, and documentation for the new protocol and diagnostics.
During carbon2u bed meshes, HX711S protocol faults such as not_ready, extra_low, and post_read_low correctly become an invalid-sample load-cell probe error. However, one torn HX711 frame during a probe point should retire that measurement and let the existing probe/mesh retry loop collect the point again instead of aborting the whole mesh. Handle only the explicit invalid HX711 sample diagnostic inside PrinterProbe._run_probe_with_retries(). Retract and retry the same probe point once, then allow any second invalid sample to propagate as the original hard error. Other load-cell failures, including drift safety, fixed-point overflow, watchdog timeout, and ordinary homing timeouts, still raise immediately. Add focused regression tests covering a successful one-shot retry, repeated invalid samples remaining fatal, and force safety errors not being retried.
A single HX711 acquisition fault can surface through LoadCellSampleCollector validation as the generic "Sensor reported 1 acquisition errors and 0 bulk overflows while sampling" error. That happens during probe tare or tap sample validation before the existing PrinterProbe invalid-HX711 retry sees the MCU probe error string, so BED_MESH_CALIBRATE can abort on a transient recovery sample instead of retrying the probe.\n\nNormalize only HX711/hx711s probe-boundary collector failures with exactly one acquisition error and zero bulk overflows to the existing invalid HX711 sample probe error. The existing PrinterProbe retry limit then retries once and keeps repeated faults fatal. Bulk overflows, non-HX711 sensors, safety faults, and all generic LoadCell validation outside probing continue to fail normally.\n\nAdd regression coverage for collector acquisition retry, tare conversion, repeated acquisition faults, bulk overflow, and non-HX711 behavior.
A poll or read racing the free-running HX711 conversion clock produces a NOT_READY frame (window closed before the read) or an EXTRA_LOW|POST_READ_LOW frame (a conversion completed mid-read). At 80 SPS on the CC1 bed this happens about once per 12 s, and each event forced a full power-down reset, a 7-frame recovery, and a probe-aborting hard fault. No bed mesh could survive the steady-state fault rate even with a one-retry budget at two layers. Treat the three timing flags as timing misses instead of protocol faults: the frame remains excluded from tare, filter, and probe decisions, but the MCU neither resets nor reports a probe fault, and the host counts it in a new "timing" health bucket instead of failing the collection window. READ_OVERRUN and SATURATED keep the existing fault and reset semantics. A stuck ADC still fails closed: 40 consecutive timing misses (~0.5 s at 80 SPS) tag the frame with the new Q_TIMING_STREAK bit, which the host classifies as a hard fault, and the MCU reports a probe fault and resets. Old hosts see the unknown bit and fail closed as well. Validation: 15 host tests (timing classification, escalation marking, collector behavior), ruff clean, STM32F401 build passes.
… band The drift-safety range was checked before trigger evaluation on every sample. With the 12.5 ms confirm, the trigger needs two or more consecutive above-threshold samples at 80 SPS, so a fast genuine contact ramp (cleaning-pass probes, stiff edge contact) crossed the raw drift band in one or two samples and the homing move died as ERROR_SAFETY_RANGE before the confirm could complete. Upstream triggers on the first filtered crossing and never sees this race; the confirm window is a rewrite addition for impulse rejection. Evaluate the drift band only when no confirm is armed: the first sample of a contact streak must still be inside the band (a lost tare or gross spike still errors), but once armed the confirm owns the outcome and may complete across the band. Below the trigger threshold the band still applies and any armed confirm resets. Reproduced on CC1: hot LOADCELL_Z_HOME failed intermittently with "force exceeded drift_safety_limit before triggering!" while 2 mm/s PROBE passed; cleaning-pass contact peaks reach ~475 g against the +/-1000 g band. STM32F401 build passes; host-side tests unaffected (15 passed, ruff clean).
Every reference implementation (upstream Kalico, garethky's community driver, the OpenCentauri fork, and Prusa's loadcell) triggers on the first filtered threshold crossing; the 12.5 ms confirmation window was an Elegoo-stock-ism carried into the rewrite. At 80 SPS the window is one sample period, so the trigger needed two or more above-threshold samples, and the extra post-contact travel pushed stiff-contact ramps across the raw drift band before the confirm could complete. With confirm_time defaulting to 0 the MCU triggers on the first within-band filtered crossing, exactly matching upstream semantics; the arming-gate restructure then degenerates to upstream's ordering. The parameter remains for configs that want impulse rejection, and the first crossing is always the reported contact time.
The r14 mesh campaign failed again with "force exceeded drift_safety_limit before triggering!" — with first-crossing triggering that requires the first above-threshold sample to be beyond the tare +/-1000 g band, i.e. a single-sample jump of >1000 g on the sum. A real 2 mm/s contact cannot do that in 25 um of travel; these are one-frame electrical glitches (ch3 dominant, matching both the old Elegoo 100k-count heuristic and the spike filter on OpenCentauri/kalico hx711s-new). Port the hx711s-new per-channel spike filter: a sample whose channel delta vs the last good value exceeds 100k counts is held back from the probe (the raw frame still streams to the host for diagnostics). A second consecutive sample near the suspicious value confirms a genuine force step and is forwarded; an immediate return to the last good value confirms an isolated glitch. State re-seeds after a power-cycle reset. Genuine fast ramps are delayed by at most one sample (~12 ms), one-frame spikes can no longer trip the drift band.
The round-5 mesh campaign reached 5 consecutive passes before attempt 6 died to "Sensor reported 0 acquisition errors and 1 bulk overflows while sampling" — a single lost bulk message after ~45 minutes of streaming, almost certainly a transient host scheduling hiccup. Failing a whole 7.5-minute macro for one lost frame is far too strict: hx711s frames are individually timestamped, so a lost message is a known gap rather than silent corruption, and the consumers are gap-tolerant by design (tare uses a median over many frames; tap data is shape-validated by TapAnalysis; the MCU trigger path does not use bulk data at all). Add a max_tolerated_overflows sensor capability (2 for hx711s, 0 default so other sensors keep the old strict behavior) and honor it in LoadCell.validate_samples: (0, <=2) logs a warning and continues, larger or mixed losses stay fatal. Covers both the tare (avg_counts) and tap (TappingMove) collection paths.
Author
|
Test build available 🧪 The Cosmos PR build carrying this stack (kalico This exact artifact went 10/10 consecutive Feedback especially welcome on:
Raw evidence and per-commit rationale: design record. |
Collaborator
|
I've just had a quick look through this but I thought I could let you know my thoughts even though it's still just a draft. The changes here are quite invasive and seem to make a lot of changes in shared files, such as It looks like a lot of the changes to the shared files are to add extra debug functionality which is fine for testing, but it would be nice to have a bit more focus to the PR, which will hopefully make it a bit easier to review. I also noticed that the PR description references PRs from upstream kalico repo which I think are incorrect and should be referencing the cosmos repo instead. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cosmos integration
This is consumed by the corresponding draft Cosmos PR #265. The detailed protocol comparison, design decisions, reverse-engineering deliverables, and validation plan are available in the HX711 reliability design record.
Annotated commit list
Base driver (earlier work):
5189df58–ca304125— hx711s multi-cell driver: lockstep reads of 4 HX711s, readiness gating, torn-read detection.cec9b2af–92fa07fc— probe plumbing: internal clients, per-channel tare counts,LOAD_CELL_CALIBRATE TARE=TRUE, pullback options.fa59d011+8433f46d— frame validity made MCU-owned and explicit (quality bits ride with the raw counts).Hardening (pre-campaign):
9ed1574a— flush bulk buffer before appending a frame (4-channel timestamped frames could overrun the 51-byte bulk buffer).28aca926— safe timer reschedule during recovery (scheduler list corruption).9044d16e— recovery/timing/probe-fault hardening.9c1936ef/64593629— retry one invalid sample / transient collector fault during probing.2026-07-23 mesh campaign stack (in order, each fixing a measured failure):
13952564— timing misses → non-fatal sample skips. Reads racing the free-running ADCs (~0.1% of frames) caused full resets ~1/9s andinvalid HX711 samplemacro deaths. Now skipped silently; a persistent streak (genuinely stuck ADC) still escalates and resets.426ac30b— drift band gates the arming sample only. The ±1000g lost-tare band was evaluated before the 12.5ms confirm on every sample; fast contact ramps crossed it mid-confirm →drift_safety_limitaborts. An armed confirm now owns completion.a4ceadd3— first-crossing trigger default (trigger_confirm_time: 0). Matches every reference implementation (Kalico main, garethky, Prusa); confirm remains as an option.87f1c4f7— per-channel spike rejection. Single-sample >100k-count glitches (ch3-dominant, ~1/800 frames) reached the trigger as fake >1000g force. A suspicious sample is held one frame: revert = glitch (dropped from probe path, still streamed to host), repeat = genuine step (accepted). Ported fromhx711s-new.c368f84e— bounded bulk-overflow tolerance. One lost bulk message after 45 min of streaming killed a 7.5-min macro. Timestamped frames make a lost message a known gap, not corruption: ≤2/session warns and continues, more stays fatal. Other sensors unchanged.Cosmos side: PR KalicoCrew#265 pins this stack;
LOADCELL_Z_HOMEcleaning passes gotSAMPLES_TOLERANCE=0.1(ooze management after squish-mangled edge contact; the precision center probe keeps 0.05).Validation on hardware (CC1 test printer)
BED_MESH_CALIBRATE BED_TEMP=60BED_MESH_CALIBRATE BED_TEMP=100Why this version is good:
Offline validation
test_hx711_reliability.py), ruff clean.