Skip to content

Add jittered retries for queue failures#3934

Open
owenshuo wants to merge 1 commit into
orchestration-agent:mainfrom
owenshuo:fix/3929-jittered-queue-retries
Open

Add jittered retries for queue failures#3934
owenshuo wants to merge 1 commit into
orchestration-agent:mainfrom
owenshuo:fix/3929-jittered-queue-retries

Conversation

@owenshuo
Copy link
Copy Markdown

Summary

  • store scheduled tasks with their queue metadata so delayed retries can be dequeued with the original task payload
  • retry transient queue failures with bounded exponential backoff and injectable jitter for deterministic tests
  • preserve task identity across retries and record terminal completed/cancelled/failed outcomes to reject stale retry paths

Fixes #3929

Validation

  • python3 -m pytest tests/test_scheduler.py tests/test_config.py -q
  • python3 -m flake8 src/orchestrator/scheduler.py src/orchestrator/engine.py tests/test_scheduler.py
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ Bounty $10k ] [ Runtime ] Retry transient queue failures with jitter — queue runtime

1 participant