Skip to content

fix(worker): avoid heartbeat sync storm - #971

Open
RerankerGuo wants to merge 2 commits into
agentscope-ai:mainfrom
RerankerGuo:fix/issue-949-worker-sync-storm
Open

fix(worker): avoid heartbeat sync storm#971
RerankerGuo wants to merge 2 commits into
agentscope-ai:mainfrom
RerankerGuo:fix/issue-949-worker-sync-storm

Conversation

@RerankerGuo

@RerankerGuo RerankerGuo commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • ignore Local->Remote-excluded paths when deciding whether the OpenClaw Worker sync loop should run
  • stop treating HEARTBEAT.md as a Worker-pushed prompt file so runtime reads/touches do not cause repeated storage uploads
  • add a static worker sync regression test and changelog entry

Fixes #949.

Tests

  • bash worker/tests/test-worker-sync-excludes.sh
  • git diff --check

Temporary dependency

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

📊 CI Metrics Report

Summary

Metric Current Baseline Change
LLM Calls 177 81 +96 ↑ +118.5%
Input Tokens 5335554 2803871 +2531683 ↑ +90.3%
Output Tokens 46523 16791 +29732 ↑ +177.1%
Total Tokens 5382077 2820662 +2561415 ↑ +90.8%

By Role

Role Metric Current Baseline Change
🧠 Manager LLM Calls 106 68 +38 ↑ +55.9%
Input Tokens 3565472 2502214 +1063258 ↑ +42.5%
Output Tokens 20746 13725 +7021 ↑ +51.2%
Total Tokens 3586218 2515939 +1070279 ↑ +42.5%
🔧 Workers LLM Calls 71 13 +58 ↑ +446.2%
Input Tokens 1770082 301657 +1468425 ↑ +486.8%
Output Tokens 25777 3066 +22711 ↑ +740.7%
Total Tokens 1795859 304723 +1491136 ↑ +489.3%

Per-Test Breakdown

Test Mgr Calls Wkr Calls Δ Calls Mgr In Wkr In Mgr Out Wkr Out Δ Tokens Trend
02-create-worker 10 0 -2 ↓ -16.7% 291830 0 1271 0 -65521 ↓ -18.3% ✅ improved
03-assign-task 11 5 +1 ↑ +6.7% 369632 125140 1864 722 +23702 ↑ +5.0% ⚠️ regressed
04-human-intervene 23 12 +22 ↑ +169.2% 741373 274552 3233 1214 +587374 ↑ +135.7% ⚠️ regressed
05-heartbeat 9 3 +5 ↑ +71.4% 347553 76613 1648 575 +151137 ↑ +54.9% ⚠️ regressed
06-multi-worker 53 51 +70 ↑ +205.9% 1815084 1293777 12730 23266 +1864723 ↑ +145.7% ⚠️ regressed

Trends

1 test(s) improved (fewer LLM calls)
⚠️ 4 test(s) regressed (more LLM calls)


Generated by HiClaw CI on 2026-07-13 15:54:55 UTC


📦 Download debug logs & test artifacts

@RerankerGuo
RerankerGuo force-pushed the fix/issue-949-worker-sync-storm branch from e6353f1 to 5352b9d Compare July 8, 2026 07:46
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction / mgr=copaw / wk=hermes)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@RerankerGuo

Copy link
Copy Markdown
Contributor Author

@shiyiyue1102 Hi! Just rebased onto current main (was conflicting after main moved). CI ran but failed on the SHARD_A copaw/hermes flake — same root cause as #972 / #973 / #975 / #976 / #983 / #984 (Manager didn't invoke hiclaw create worker --name bob in the copaw/hermes run; bob worker session was never created).

Good news: PR #985 (test: make multi-worker bob setup deterministic) just turned 18/18 green on its rebase and pinged you this morning as a priority — it replaces the LLM-driven bob creation with a direct hiclaw apply worker --name bob call so the SHARD_A flake cannot reoccur. If #985 lands first, re-running CI on this PR (and the other Tier-3 PRs) should clear them automatically.

This PR's actual change: OpenClaw Worker sync loop excludes HEARTBEAT.md / runtime files from the bulk mirror path so empty cycles no longer trigger. Could you take a look once #985 is in?

@RerankerGuo
RerankerGuo force-pushed the fix/issue-949-worker-sync-storm branch from 1e964c6 to 28c4ebf Compare July 14, 2026 11:19
@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction-2 / mgr=copaw / wk=copaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction / mgr=copaw / wk=copaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction / mgr=openclaw / wk=openclaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction-2 / mgr=openclaw / wk=openclaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr / mgr=copaw / wk=hermes)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr-2 / mgr=copaw / wk=copaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (llm-interaction-2 / mgr=copaw / wk=hermes)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr / mgr=copaw / wk=copaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@github-actions

Copy link
Copy Markdown
Contributor

❌ Integration Tests Failed (controller-cr / mgr=openclaw / wk=openclaw)

Commit: 28c4ebf
Workflow run: #1653

Test Results
No test output captured.
Debug Log (tail)
No debug logs available.

📦 Download full debug logs & test artifacts

@shiyiyue1102

Copy link
Copy Markdown
Collaborator

Avoiding repeated empty storage mirrors from runtime-touched or excluded files is operationally valuable. The current PR conflicts with main and includes the unrelated multi-worker setup commit, which also appears in several other branches. Please rebase onto the latest main branch, remove the test-06 dependency, and keep only the Worker sync-loop change, its focused regression, and the current changelog entry. Please also make the intended HEARTBEAT.md ownership explicit because changing it from bidirectional to pull-only is a behavioral contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

work 容器假死,manager无法唤醒。 || work The container is suspended and the manager cannot wake up.

2 participants