Add RubyLLM::Agents.track for multi-agent request grouping#16
Merged
Conversation
Introduces the core data structures for the track block feature: - Tracker: internal collector that accumulates Results during a block - TrackReport: public aggregation class with cost/token/timing summaries Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every Result (base, embedding, speech, transcription, image) now: - Stores agent_class_name for cost_breakdown reporting - Auto-registers with active Thread.current tracker via Trackable mixin - Includes agent_class_name in to_h output Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the core .track API: - Wraps block, collects all Results, returns TrackReport - Supports nesting with result bubbling to parent tracker - Captures errors without re-raising (report.error) - Merges shared defaults (tenant, tags) into agent options - Cleans up thread-local in ensure block Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The instrumentation middleware now reads @_track_request_id and @_track_tags from the agent instance and injects them into execution records, enabling dashboard grouping by request_id and tag filtering. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New dashboard page that groups executions by request_id: - Index view with sortable columns, pagination, aggregate stats - Show view with execution timeline, summary stats, timing details - Navigation link in header (desktop + mobile) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds SummarizeAgent + ClassifyAgent as helper agents for demonstrating RubyLLM::Agents.track, with comprehensive usage examples in comments and a new "Multi-Agent Tracking" section in the example README. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Seeds 8 multi-agent tracked requests with diverse scenarios: customer support, content pipeline, error handling, batch processing, streaming conversation, multi-model, in-progress, and single-agent cached requests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add llm_executions method to TestOrganization so stubs work when verify_partial_doubles is enabled by earlier spec files - Remove invalid allow_any_instance_of(ActiveRecord::Relation) stub for avg_time_to_first_token since the actual method works correctly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RubyLLM::Agents.trackblock wrapper that collects everyResultproduced by agent calls inside it, aggregates cost/token/timing stats, and returns aTrackReportrequest_idandtagsinto execution records automatically, enabling grouped observabilityTrackablemixinWhat's included
Core (
lib/ruby_llm/agents/)Tracker— internal collector that accumulates Results during a tracked block usingThread.currentTrackReport— public aggregation class withtotal_cost,total_tokens,duration_ms,cost_breakdown,models_used,errors,all_successful?, and moreResults::Trackablemixin — included in all 12 result classes (text, embedding, speech, transcription, 8 image types) for automatic tracker registrationBaseAgentintegration — merges tracker defaults (tenant, tags) into agent optionsInstrumentationmiddleware — injectsrequest_idand tags from the active tracker into execution recordsDashboard (
app/)RequestsControllerwith index (grouped byrequest_id, sortable, paginated) and show (execution timeline, summary stats)Example app & seeds
SummarizeAgent+ClassifyAgenttracking examples with comprehensive usage docsTests (TDD — 116 new specs)
tracker_spec.rb— 10 specs for Tracker internalstrack_report_spec.rb— 30 specs for TrackReport aggregationresult_tracking_spec.rb— 7 specs for Result hook across all typestrack_integration_spec.rb— 19 specs for.trackmethod (nesting, errors, defaults)track_instrumentation_spec.rb— 5 specs for request_id/tags injectionrequests_controller_spec.rb— 8 specs for dashboard controllerUsage
Test plan
🤖 Generated with Claude Code