Skip to content

hx711s: read-on-DRDY acquisition loop with in-driver recovery - #6

Closed
pdscomp wants to merge 3 commits into
hx711s-newfrom
paul/hx711s-new
Closed

hx711s: read-on-DRDY acquisition loop with in-driver recovery#6
pdscomp wants to merge 3 commits into
hx711s-newfrom
paul/hx711s-new

Conversation

@pdscomp

@pdscomp pdscomp commented Jul 26, 2026

Copy link
Copy Markdown

Summary

Built on james's hx711s-new branch (multi-sensor driver, corrupt-frame filter, tap recorder) — this PR rewrites its acquisition loop around read-on-DRDY with in-driver recovery, replacing poll-and-classify with a design where recoverable faults never reach the host.

Validated on CC1 hardware (4× HX711, 80 SPS): 30/30 consecutive full bed-mesh runs with zero faults — 10× at 60 °C bed, 20× at 100 °C (chamber ~60 °C). Supersedes #5 (same base branch; validated old stack remains at c368f84e).

The driver polls each chip's DRDY, verifies every frame, and re-reads on the next DRDY instead of consuming or discarding raced frames — the same acquisition contract used by Prusa's HX717, the Linux IIO hx711 driver, and upstream Klipper's hx71x. Protocol corruption is cleared by an in-driver power cycle with the datasheet-mandated settling conversions discarded, instead of a host-forced sensor restart that kills any active probe session.

Files changed

Only 2 files, both owned by the hx711s driver. No changes to shared probing, retry, or load-cell framework logic.

File Change Justification
src/sensor_hx711s.c Acquisition loop rewrite (+164/−15) DRDY verify + bounded retry for torn frames (datasheet: DOUT low = data ready; every reference impl waits for DRDY), sum-impulse hold-confirm for the probe trigger, in-driver power-cycle recovery for desync/stuck (datasheet: SCK >60 µs = power down; 4 settling conversions @80 SPS), stuck-ADC watchdog
klippy/extras/load_cell/hx711s.py Host decode (+~20 lines) RECOVERED marker decode (torn-retry + recovery tallies logged, no forced restart)

Everything else in the driver (per-channel corrupt-frame filter, lockstep read, tap recorder) is james's hx711s-new, unchanged.

Failure taxonomy fixed (all captured on hardware)

Class Signature Rate Handling
Torn read Chip latches a conversion mid-read (~50 µs read window vs 11.6 ms period) ~0.4% of frames Re-read on next DRDY (bounded ×2), never emitted
Corrupt frame Protocol-valid frame, impossible value (267907 → −1 on one channel) ~1/800 frames Per-channel hold-confirm filter, held from probe path
Single-sample impulse 87–220 g blip, one sample wide, instantly reverts (Z-accel kicks, travel ring) ~1/40 taps Sum-impulse hold-confirm below trigger_force — see below
Desync Gain-pulse count mismatch = corrupt protocol state rare In-driver power cycle (~115 ms), host notified via marker
Stuck ADC No DRDY for ~2.5 conversion periods not yet observed Same power-cycle path

The impulse class was the campaign-killer, in two captured manifestations:

  1. Phantom trigger: a ~220 g single-sample blip at probing-move start fired a first-crossing trigger 1.2 mm above real contact → flat tap window → TAP_CHRONOLOGY abort.
  2. Pre-movement trigger: an isolated +87 g blip 0.5 s after travel settled, caught in a 182k-sample raw stream capture, landing in the arming window → Probe triggered prior to movement.

Real contact at 2 mm/s rises ≤ ~80 g/sample sustained; impulses exceed trigger force for exactly one sample. The driver holds any single-sample jump > 67 g (just under the 75 g trigger) from the trigger feed and releases it only if the next sample confirms it sustained (sign-aware: an up-jump confirms if the level holds, so steep real ramps pass with ≤1 sample latency). The raw stream is untouched — impulses still reach the host for diagnostics.

Why MCU-side sum with in-driver retries (not 4 independent load cells)

  1. Generic-side blast radius. Per-channel emission forces load_cell_probe.c (shared with hx71x/ADS131M0X) to grow an aggregation state machine for 4 asynchronous streams. MCU-side summing keeps shared code untouched: one clean force stream, as with a single-cell sensor.
  2. The trigger is MCU-real-time. Z halt evaluates total force with sub-ms determinism; the sum must exist MCU-side regardless.
  3. Fewer torn reads. Reading each chip at its own DRDY (vs waiting for all-4-ready and reading stale-ready peers near their next conversion) shrinks the tear window; the sum is held until all 4 verified with a bounded deadline — same worst-case trigger latency as lockstep.
  4. Retry locality. A retried channel is invisible to the wire protocol; per-channel emission would force the host to understand gaps or partial sums.
  5. Topology precedent. Every 4-cell implementation (Elegoo stock, james's hx711s) sums MCU-side; single-cell designs (Prusa, upstream hx71x) have no sum problem to solve.

Testing

  • Full BED_MESH_CALIBRATE campaigns on a 4× HX711 CC1: 10/10 @ 60 °C, 20/20 @ 100 °C, zero faults, zero recoveries needed.
  • Raw force capture (load_cell/dump_force) across full macro runs; the new filter was validated offline by replaying 182k captured samples: all 72 phantom crossings removed, all 433 real contact ramps still trigger.
  • Per-commit rationale and citations in the commit messages.

Full design record, failure forensics (tap curves, raw stream excerpts), and campaign logs: https://gist.github.com/pdscomp/b3ab4f3c3712ce771f0fa637f643be26

TL;DR / lingo for normal humans

  • DRDY ("data ready"): the HX711's DOUT pin. The chip finishes a conversion and pulls it low to say "fresh stable value, safe to read." Reading at any other time is how you get garbage.
  • Torn read: the chip latches a new conversion while we're clocking out the previous one, so the bits arrive half-old/half-new. Happens ~0.4% of the time by physics; unfixable by polling faster, fixable only by re-reading.
  • Tap: the nozzle touching the bed during a probe. The force-vs-time curve of one tap gets validated for shape before its z value is trusted.
  • Tare: zeroing the scale before each probe (like the TARE button on a kitchen scale).
  • Trigger / trigger_force: when measured force crosses 75 g, the MCU stops the Z move instantly — that's the "we touched the bed" signal.
  • First-crossing trigger: trigger on the first sample above 75 g (no confirmation window). Fast and precise, but believes any liar big enough — hence the impulse filter.
  • Arming window: the brief moment between "probe is ready" and "Z starts moving." A force blip there reads as "already touching" — the Probe triggered prior to movement error.
  • Phantom trigger: a one-sample force blip (motor accel kicking the bed frame) crossing 75 g with no actual contact. The probe "touches air" 1 mm above the bed.
  • TAP_CHRONOLOGY / TAP_SHAPE_INVALID: the tap validator saying "this force curve doesn't look like a real touch" — the symptom phantoms left behind.
  • Hold-confirm: when a suspiciously large jump appears, hold it back for exactly one sample. If the next sample agrees it was a real step; if it snaps back, it was noise — drop it.
  • Settling conversions: after a power cycle the HX711's first 4 readings are garbage by design (datasheet says so); the driver throws them away.
  • SPS: samples per second (80, per chip).

pdscomp added 3 commits July 25, 2026 21:52
Make the MCU driver self-healing so recoverable acquisition faults never
reach the host as fatal errors:

- Torn reads (a chip latching a new conversion mid-read, ~0.4% of frames
  from the ~50us read window vs the 11.6ms conversion period) are no
  longer consumed or discarded. The driver emits nothing and re-reads on
  the next DRDY, bounded at 2 retries before degrading to the host's
  benign hold-last-value path. Every reference implementation waits for
  DRDY and re-reads rather than consume a raced frame: Prusa HX717
  (src/common/hx717.cpp), Linux IIO (drivers/iio/adc/hx711.c), upstream
  Klipper (src/sensor_hx71x.c), and Elegoo's stock firmware (bounded
  ready-poll plus protected transfer, per the 1.6.9/1.7.0 images).
- A gain-pulse desync means the serial protocol state no longer matches
  the chip, which only a power cycle clears (HX711F datasheet, power-down
  control: SCK high >60us). The driver now power-cycles in-driver and
  discards the 4 settling conversions the datasheet requires at 80 SPS
  before resuming the stream (~115ms pause). Previously the host
  force-restarted the whole sensor, which killed any active probe/tare
  session. Elegoo's stock firmware performs the same power-cycle
  recovery.
- A chip that stops signaling DRDY for ~2.5 conversion periods (its
  oscillator free-runs, so silence is never legitimate) takes the same
  power-cycle path instead of freezing the stream.
- The host is notified once per recovery via a RECOVERED marker frame
  carrying the torn-retry and recovery tallies, so reliability stays
  observable in the klippy log without escalating to a restart.

The per-channel spike filter is unchanged: those glitches pass protocol
framing, so no read-timing fix can catch them.
Move-start acceleration kicks (500 mm/s^2 Z moves) ring the bed frame and
produce single-sample force impulses of ~200-250g on the channel sum that
fully revert on the next sample (measured from recorded tap curves on the
CC1 test printer: the same ~220g one-sample blip appears at probing-move
start and at pullback-move start). They pass the per-channel glitch filter
(100k-count threshold) because they are too small, and they pass the tare
because they are brief. When one lands in the arming window of a probing
move it fires a first-crossing trigger up to 1.2mm above the real contact
point: the resulting tap window contains no contact ramp and the tap
validator aborts with TAP_CHRONOLOGY.

Real contact at 2mm/s rises by at most ~80g/sample sustained (also from
the recorded taps), so any single-sample jump over ~190g on the sum is
now held for one sample and only forwarded to the trigger if the next
sample confirms it as a genuine step (the same hold-confirm pattern as
the per-channel glitch filter). Real contact ramps never engage the
hold, so trigger latency is unchanged; genuine steps are delayed by at
most one sample. The raw per-channel stream still carries the impulse to
the host for diagnostics.
The previous 190g threshold missed the small end of the impulse class: a
captured raw force stream (182k samples, full mesh macro runs on the CC1
test printer) shows the killer event was an isolated +87g single-sample
blip 0.5s after travel vibration settled -- 87g is under the 190g hold
threshold but over the 75g trigger force, so it fired 'probe triggered
prior to movement' in the arming window. The impulse class spans at
least 87-220g, all single-sample and instantly reverting, so the hold
threshold now sits just under trigger_force (67g): any blip big enough
to trigger is held, and only released if the next sample confirms it.

The confirm check is now sign-aware: an up-jump confirms when the next
sample stays above (pending - threshold), which steep real contact ramps
(+80g/sample sustained) satisfy, while phantoms revert and are dropped.
The old symmetric |current - pending| <= threshold check would never
confirm a monotonic ramp (each sample is more than threshold above the
previous), needlessly stalling the trigger feed.

Validated by replaying the full 182k-sample capture through a model of
the filter: all 72 phantom crossings (1-sample blips, 60-220g) are
removed; all 433 real contact ramps still trigger; the fatal +87g blip
at the failure timestamp is held and dropped.
@pdscomp

pdscomp commented Jul 26, 2026

Copy link
Copy Markdown
Author

Closing because OBE (rebasing fixes on james-new branch!)

@pdscomp pdscomp closed this Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant