Skip to content

feat: resilient background job retry & monitoring#707

Open
shaidshark wants to merge 3 commits intorohitdash08:mainfrom
shaidshark:job-retry-monitoring-from-agent
Open

feat: resilient background job retry & monitoring#707
shaidshark wants to merge 3 commits intorohitdash08:mainfrom
shaidshark:job-retry-monitoring-from-agent

Conversation

@shaidshark
Copy link
Copy Markdown

Implements resilient retry handling and basic monitoring for reminder/background job delivery.

Included

  • retry tracking fields on reminders
  • exponential backoff retry flow
  • failed/retrying status handling
  • admin monitoring endpoints for reminder failures/metrics
  • OpenAPI + README updates
  • tests for retry behavior

This is intentionally scoped to the existing reminder/background processing flow rather than introducing a new queueing system.

Implements weekly financial digests with:
- Comprehensive weekly summary (income, expenses, net flow, savings rate)
- Week-over-week trends analysis
- Spending breakdown by category
- Notable transactions highlights
- Upcoming bills preview
- AI-generated financial insights
- Email delivery via existing SMTP infrastructure
- Scheduled weekly generation (Sundays 9 AM UTC via APScheduler)

New endpoints:
- GET /digest/weekly - Get weekly digest data
- POST /digest/weekly/send - Send digest email
- GET /digest/weekly/preview - Preview email content
- GET /scheduler/status - Check scheduler status

CLI commands:
- flask generate-digest --user-id ID --send-email

Fixes rohitdash08#121
- Add retry tracking fields to Reminder model (retry_count, last_retry_at, next_retry_at, failure_reason, status)
- Implement dispatch_reminder with exponential backoff (5min base, max 5 retries)
- Update /reminders/run to use dispatch_reminder and handle retrying reminders
- Add admin monitoring endpoints: /admin/reminders/metrics and /admin/reminders/failures
- Extend OpenAPI spec with new fields and admin endpoints
- Update README with retry mechanism and admin monitoring documentation
- Add observability events for sent, retry, failed
- Add comprehensive tests for retry logic and admin access
- Ensure schema compatibility for existing PostgreSQL deployments

Closes rohitdash08#130
@shaidshark shaidshark requested a review from rohitdash08 as a code owner April 2, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant