refactor: optimize cache management and restore risk change detection#59
Merged
refactor: optimize cache management and restore risk change detection#59
Conversation
6 tasks
jubalm
changed the title
Add graceful degradation for RPC rate limiting during event queries: - Detect rate limit errors (429, 1015, "Too Many Requests") - Add 100ms delay between chunk queries to reduce request rate - Implement exponential backoff (2s, 4s, 8s) when rate limits hit - Stop after 5 consecutive failures and use partial data - Track successful vs total chunks for better monitoring - Continue with partial results instead of complete failure This prevents workflow failures when public RPC endpoints hit rate limits during hourly fork risk monitoring runs.
Major performance optimization for fork risk monitoring: - Add EventCache and SerializedEventLog interfaces - Implement cache management functions (load, save, validate, serialize, prune) - Modify getActiveDisputes() for incremental queries - Add finality protection (re-query last 32 blocks) - Merge cached events with new blockchain queries - Log efficiency metrics (RPC queries saved) - Change schedule from hourly to 6-hourly (00:05, 06:05, 12:05, 18:05 UTC) - Add cache retrieval from gh-pages branch - Implement smart commit detection: - Immediate: When risk data changes - Daily: Cache refresh at midnight (00:05 UTC) - Skip: No meaningful changes - Preserve cache/ directory in gh-pages deployment - Create docs/rpc-caching-strategy.md (comprehensive public documentation) - Update README.md with caching information - Add inline workflow comments explaining caching strategy - Create cache/ directory with .gitkeep placeholder - RPC queries: ~1,200/day → ~8/day (99.3% reduction) - Execution time: ~60s → ~10s per run (83% reduction) - Git commits: ~720/month → ~40/month (95% reduction) - Rate limit errors: Frequent → Rare - Cache publicly accessible at https://augur.net/cache/event-cache.json - Full git history audit trail on gh-pages branch - Detailed GitHub Actions logs for verification - Cache maintains 7-day sliding window of events - Ethereum finality protection (32 blocks) - Graceful fallback to full query if cache unavailable - Cache version: 1.0.0 See docs/rpc-caching-strategy.md for complete technical documentation.
jubalm
force-pushed
the
fix-rpc-rate-limiting
branch
from
125fa77 to
5c2d8ac
Compare
## Improvements
### Cache Path Optimization
- Move cache from root `cache/` to `public/cache/`
- Leverages Astro's automatic public folder copying to dist/
- Eliminates manual copy steps in workflow
- Cache automatically included in artifact and deployed
### Restored Risk Change Detection
- Re-add immediate commits when fork risk changes meaningfully
- Compare current vs previous risk from git history
- Maintains full audit trail of risk changes
- Improves transparency vs. efficiency balance
### Workflow Optimization
- Eliminate redundant artifact download
- Remove unused cache file copy step
- Checkout gh-pages directly instead of main
- Cleaner, more efficient deploy job (fewer steps)
### First-Run Cache Initialization
- Replace malformed `{}` fallback with valid EventCache structure
- Ensures validation passes immediately
- Prevents unnecessary rescue logic on first run
### Cleanup
- Remove unused `cache/.gitkeep` placeholder
## Technical Details
- Cache path: `public/cache/event-cache.json` (auto-copied by Astro to dist/)
- Risk detection: Compares `riskPercentage` between current and previous commits
- Empty cache: Valid JSON matching EventCache interface with all required fields
- Smart commits: Immediate (risk change), Daily (cache refresh), Skip (no changes)
See docs/rpc-caching-strategy.md for complete caching architecture documentation.
jubalm
changed the title
- Update cache storage location to reflect public/cache/ during build - Update smart commit detection logic to show git history comparison - Clarify the two-phase workflow (build job → deploy job separation) - Explain how Astro copies public/ to dist/ automatically - Document valid empty cache structure instead of malformed fallback - Add detailed execution flow diagram with job breakdown
- Fix YAML syntax error in heredoc (line 73) - Replace heredoc with jq command for clean JSON generation - Use dynamic timestamps via date -u (UTC) instead of hardcoded values - Eliminates indentation issues from YAML block scalar parsing - Improves code clarity and maintainability This fixes: 1. GitHub Actions workflow validation error 2. Hardcoded timestamp issue (cache metadata always stale) 3. Ensures consistent UTC timezone usage
## Critical Fixes
### 1. Floating-Point String Comparison (Line 160)
Changed from string comparison to numeric comparison with rounding tolerance.
- Old: `[ "$CURRENT_RISK" != "$PREVIOUS_RISK" ]` (string compare fails: "2.1" != "2.10")
- New: `awk BEGIN {if (int(curr*100) != int(prev*100))}` (numeric compare at 2 decimals)
- Impact: Prevents false positives/negatives in risk change detection
### 2. Timezone Bug (Line 168)
Changed to UTC to match cron schedule
- Old: `HOUR=$(date +%H)` (uses runner's local timezone)
- New: `HOUR=$(date -u +%H)` (UTC)
- Impact: Daily cache refresh now triggers at midnight UTC, not runner's local time
### 3. Cache File Assumption (Line 165)
Added explicit file existence verification
- Old: `EVENTS=$(jq ... cache/event-cache.json)` (fails silently)
- New: Check `[ -f cache/event-cache.json ]` before reading
- Impact: Prevents silent fallback to "0" if cache wasn't deployed
### 4. Block Number Handling (Line 154)
Changed from "N/A" to "Unknown" for clarity
- Better commit message clarity if data is missing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comprehensive RPC rate limiting solution with incremental event caching, smart commits, and workflow optimizations. Reduces RPC calls by 99.3% (1,200/day → 8/day) while maintaining full transparency and auditability.
Problem Solved
Original implementation queried ~50,400 blocks (7 days) every hour via ~51 RPC calls, totaling 1,200 queries/day. This caused:
Solution Architecture
Incremental Event Caching
Smart Commit Strategy
Workflow Optimization
Commits Included
1. feat: implement 6-hour smart commit with incremental event caching
2. refactor: optimize cache management and restore risk change detection
cache/→ Astro'spublic/cache/(auto-copied to dist/)3. docs: align caching strategy documentation
4. fix: resolve YAML syntax error and improve cache initialization
jqfor clean JSON generationdate -u(UTC)5. fix: resolve critical smart commit detection issues
date -u +%Hfor midnight detection[ -f cache/event-cache.json ]check before readingPerformance Impact
Transparency & Auditability
git log gh-pages -- cache/event-cache.jsonTest Plan
Migration Notes
Code Quality
Related: Fixes #57 (RPC rate limiting issues)
Branch: fix-rpc-rate-limiting → main
Status: Ready for review and testing