Skip to content

fix(native): quiet send-cancelled-on-stop Error + bump collector 0.6.2 / agent 0.5.28#1230

Merged
lotgon merged 1 commit into
masterfrom
fix/send-on-stop-not-error
Jul 3, 2026
Merged

fix(native): quiet send-cancelled-on-stop Error + bump collector 0.6.2 / agent 0.5.28#1230
lotgon merged 1 commit into
masterfrom
fix/send-on-stop-not-error

Conversation

@lotgon

@lotgon lotgon commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

What

HttpSendBatch logged every failed send at Error — including the in-flight POST that StopWorker cancels during a graceful stop (HTTP 0 Operation was aborted by an application callback), seen on every restart. That's the same "expected shutdown noise" class as the stop-drop (made Debug in #1225).

Now: if the failure happened while send_cancelled_ is set (the stop path), log it at Debug; a genuine send failure while running still logs at Error (deduplicated). send_cancelled_ is read under hang_mutex_ — safe, because the caller's guard scope ends before sender_(batch).

Versions (the discipline)

  • Collector behavior change, no ABI changePATCH 0.6.1 → 0.6.2 (HSM_COLLECTOR_VERSION + CMakeLists.txt). → src/native/collector/AGENTS.md
  • The collector ships inside the agent → agent 0.5.27 → 0.5.28. → src/agent/AGENTS.md

Stacked on #1229 (agent 0.5.27); base retargets to master when #1229 merges.

Verify

After deploy, a graceful stop/restart shows no [ERROR] Failed to send … aborted (now Debug, filtered by the agent's Info min-level) — matching the already-quiet stop-drop.

🤖 Generated with Claude Code

… bump collector 0.6.2 / agent 0.5.28)

HttpSendBatch logged every failed send at Error, including the in-flight POST that StopWorker cancels during a graceful stop ('HTTP 0 Operation was aborted by an application callback') — noise on every restart, the same class as the stop-drop (already Debug). Now: if send_cancelled_ is set (stop path), log at Debug; a genuine failure while running still logs Error. Read under hang_mutex_ (not held by the caller there).

Per src/native/collector/AGENTS.md this is a behavior change with no ABI change -> collector PATCH 0.6.1 -> 0.6.2; per src/agent/AGENTS.md the collector ships in the agent -> agent 0.5.27 -> 0.5.28.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

Review — PR #1230 fix(native): quiet send-cancelled-on-stop Error

Overall this is a small, well-targeted fix. The logic is correct, the locking is sound, and the version discipline (collector PATCH 0.6.1 → 0.6.2, agent 0.5.27 → 0.5.28, HSM_COLLECTOR_VERSION_PATCH kept in sync) is consistent with the ABI-versioning rule. I have no blocking concerns. A few observations below.

Correctness — looks good

  • Locking is safe. send_cancelled_ is read under hang_mutex_ (matching every other access: writes at StartWorker/StopWorker, and the read in TrySendBatch's hang wait). The guard scope closes before LogMessage/LogError are called, so there is no lock nesting and no deadlock with logger_mutex_. Good.
  • The stop window is covered end-to-end. StopWorker() sets send_cancelled_ = true before joining the worker, and DrainQueueOnStop() runs afterward on the same stop thread with the flag still set. So both the in-flight cancelled POST (inside the worker) and any failing send during the post-join drain are now downgraded to Debug — which is the full intent. The flag is reset to false in StartWorker(), so a later restart correctly re-arms Error logging.
  • No double-emit problem: the Debug line here and the existing "Collector stop dropped N pending value(s)" breadcrumb are both Debug and both expected during a graceful stop.

Observation — the downgrade is broader than the comment implies (non-blocking)

The comment reads "A send whose in-flight POST was cancelled by a graceful stop…", but the actual condition is simply "any send failure while a stop is in progress." Once send_cancelled_ is true, every failed batch is logged at Debug — including a genuine, unrelated failure (e.g. the server happens to return 500, or DNS fails) that coincides with shutdown. That is almost certainly the desired trade-off (a graceful stop must not spam Error and must not block on a dead transport), but it is worth being explicit that this suppresses all send errors during the stop window, not only cancellation-caused ones. Consider tightening the comment wording to match the actual behavior so a future reader doesn't assume the check is narrower than it is.

Test coverage — no regression test added (minor)

There's no test asserting the new Debug-vs-Error split. This path only runs in the HSM_COLLECTOR_HTTP build (the default RecordBatch sender always returns true), so exercising it requires the HTTP transport plus a fail/cancel injection during stop — harder to set up than the corpus tests. If the HTTP test harness can force a failing send during a stop, a small regression test (assert the captured log for a cancelled-on-stop send is Debug, and that a failing send while running stays Error) would lock in the behavior and guard against a future refactor of send_cancelled_ re-introducing the shutdown Error noise. Not required to merge.

Nits

  • The msg string is now built unconditionally inside the if (!response.IsSuccess()) block. That's fine — it's only constructed on the failure path, so no measurable cost.

Verdict

LGTM. The fix is correct and minimal; the only follow-ups are cosmetic (comment precision) and optional (a regression test).

@lotgon
lotgon merged commit 91a2f64 into master Jul 3, 2026
21 checks passed
@lotgon
lotgon deleted the fix/send-on-stop-not-error branch July 3, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant