Skip to content

Guard terminal runs against late heartbeats#3919

Open
owenshuo wants to merge 1 commit into
orchestration-agent:mainfrom
owenshuo:fix/3913-heartbeat-terminal-guard
Open

Guard terminal runs against late heartbeats#3919
owenshuo wants to merge 1 commit into
orchestration-agent:mainfrom
owenshuo:fix/3913-heartbeat-terminal-guard

Conversation

@owenshuo
Copy link
Copy Markdown

Summary

  • add a run heartbeat monitor with durable terminal states for completed, failed, and cancelled runs
  • reject late worker heartbeats once a run reaches a terminal outcome so the run cannot be revived as running
  • wire the monitor into task execution and expose regression coverage for terminal idempotency and engine-level late heartbeat handling

Fixes #3913

Validation

  • python3 -m pytest tests/test_heartbeat_monitor.py tests/test_scheduler.py tests/test_config.py -q
  • python3 -m flake8 src/orchestrator/engine.py src/orchestrator/init.py tests/test_heartbeat_monitor.py
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ Bounty $3k ] [ Runtime ] Prevent worker heartbeat from reviving completed runs — heartbeat monitor

1 participant