v3.13.0

Added

• Two-tier data retention — soft_purge_after (default 30 days) and hard_purge_after (default 365 days) config options with a new RetentionJob and rake ruby_llm_agents:purge task. Soft pass destroys execution_details and tool_executions while preserving execution records (and a truncated error_message in metadata) so error-rate trends survive the purge; hard pass destroys executions in batches with cascading deletes. Either tier can be set to nil to disable it
• Searchable multi-select tenant filter — Executions page tenant filter now supports multi-select with a client-side search input. _filter_dropdown partial gains an opt-in searchable: true flag; dropdown stays open while selecting and batches submissions on close
• Routing auto_delegate: false — Opt out of router auto-delegation to return a classification-only RoutingResult with agent_class populated for manual dispatch (fixes #24)
• Soft-purge execution UI — Banner on execution show page; detail-backed sections hide silently after purge; system config page surfaces both purge windows

Changed

• Bumped ruby_llm dependency to >= 1.14.1
• Deprecated retention_period — Now an alias for hard_purge_after
• Dashboard override UI now renders for regular agents — show.html.erb renders config_<kind> for all agent kinds instead of skipping "agent"
• Replaced internal-class mocks in specs — Image execution, reliability, builder, speaker, transcriber, and instrumentation specs now exercise real code paths; mocks retained only at external boundaries (LLM APIs, DB error injection, Rails cache edges)

Fixed

• Router stream block forwarding — Caller's stream block is now captured and forwarded to the delegated agent so streaming flows end-to-end (fixes #24)
• Failed image executions not persisted — ImageOperationExecution#record_failed_execution passed error_message: to Execution.create!, raising ActiveModel::UnknownAttributeError that was silently swallowed. Split into Execution.create! + execution.create_detail!(error_message:) to match the instrumentation middleware pattern
• Audio player response.dig on String — Added Execution#response_hash normalizer that returns {} when the response is a String, Array, or nil; fixes undefined method 'dig' for String when agents write plain-text responses to the JSON column
• Override reset button — Reset controls use button_to with turbo_confirm; "reset all" moved outside the save-overrides form to stop submitting the outer PATCH

Full changelog: v3.12.0...v3.13.0

v3.12.0

Added

• Knowledge DSL — Inject domain knowledge into agent system prompts with knows :refund_policy (static files) or knows(:data) { ... } (dynamic blocks). Supports conditional inclusion via if: option, multi-entry knows :a, :b, :c, and automatic ## Heading formatting. Default knowledge_path is app/agents/knowledge
• Analytics page — Filterable cost intelligence dashboard with period-over-period comparison, daily cost chart overlay, model efficiency table, error cost breakdown, monthly projection, and savings opportunity detection
• Dashboard-overridable agent settings — Mark DSL fields with overridable: true (e.g., model "gpt-4o", overridable: true) to allow tuning from the dashboard without code deploys. Stored in agent_overrides table
• Tenant dashboard enhancements — Analytics section with trend chart, usage breakdowns by model/agent, and refresh counters button on tenant show page
• Average cost/tokens per call — Agent show page now displays average cost and token usage per execution

Fixed

• Instrumentation middleware — model_provider, chosen_model_id, and finish_reason now correctly populate on execution records (fixes #23)
• check_total_timeout! math bug — Timeout error now reports the configured timeout value instead of an inflated number
• Prior period boundary overlap — Analytics prior period scope uses exclusive range to prevent boundary-day double-counting
• Budget middleware double tenant lookup — Tenant record is now resolved once per request instead of twice (check + record)
• trend_analysis N+1 queries — Replaced 28 individual queries (4 per day × 7 days) with a single GROUP BY query
• daily_report excessive queries — Consolidated 11 individual count/sum queries into a single aggregated query
• Analytics view XSS hardening — Replaced raw helper with .to_json.html_safe for data injection in JavaScript blocks

Removed

• Orchestration/sub-agents feature — Removed as out of scope for this gem

---

Full Changelog: v3.11.0...v3.12.0

v3.11.0

Added

• agents DSL — Separate DSL for declaring sub-agent delegates, distinct from tools. Supports simple form (agents [AgentA, AgentB], forward: [:workspace_path]) and block form with per-agent timeout: and description: overrides, forward parameter auto-injection, and instructions text
• forward parameter injection — Auto-injects parent agent params into sub-agent calls and removes them from the LLM-facing tool schema, eliminating token waste and reducing failure points
• System prompt auto-generation — Agents with the agents DSL automatically get "## Direct Tools" and "## Agents" sections appended to their system prompt, helping the LLM distinguish between cheap local tools and expensive autonomous agents
• StreamEvent value object — Typed streaming events (:chunk, :tool_start, :tool_end, :agent_start, :agent_end, :error) for full execution lifecycle visibility. Opt-in via stream_events: true on .call() — default raw chunk behavior unchanged
• Router auto-delegation — Routes with agent: mapping now automatically invoke the mapped agent after classification. RoutingResult exposes delegated?, delegated_to, routing_cost, and total_cost for full visibility into the classification + delegation pipeline
• OrchestratorAgent example — New example agent demonstrating the agents DSL with block form, forward, and instructions

v3.10.0

Added

• RubyLLM::Agents::Tool base class — Extends RubyLLM::Tool with agent context access, per-tool timeouts, and automatic error handling. Users implement execute() (standard RubyLLM convention) while call() wraps with context, timeout, and error recovery
• ToolContext accessor — Read-only context wrapper providing method-style (context.container_id) and hash-style (context[:container_id]) access to agent params, tenant ID, and execution ID from within tools
• Per-tool timeout DSL — timeout 30 class-level DSL on tools, with config.default_tool_timeout global fallback
• ToolExecution model — Real-time per-tool tracking with individual database records (INSERT on start, UPDATE on complete), enabling live dashboard views and queryable tool-level analytics
• Cancellation support — on_cancelled: proc checked before each tool execution. Raises CancelledError which BaseAgent catches to return Result with cancelled? = true
• Result#cancelled? — Query whether an agent execution was cancelled mid-run
• Example coding agent — CodingAgent with FileReaderTool demonstrating tool context and timeout features

Fixed

• Undefined method error? on Context — Fixed budget middleware breaking tenant budget tracking when Pipeline::Context lacked the error? method

v3.9.0

Added

• RubyLLM::Agents.track block API — Group multiple agent executions into a single tracked request with request_id and tags. Includes Tracker and TrackReport classes for aggregated cost/token analytics
• Requests dashboard — New dashboard page for viewing and filtering tracked request groups with per-request detail views
• Convenience query API — RubyLLM::Agents.executions, .recent, .for_agent, .for_model class-level query methods for accessing execution data from host apps
• doctor generator — rails generate ruby_llm_agents:doctor validates installation, configuration, database tables, and RubyLLM connectivity
• demo generator — rails generate ruby_llm_agents:demo scaffolds a working example agent with demo rake task
• Structured logging & pipeline tracing — Middleware base class now provides structured log_info/log_warn/log_error helpers with trace context for debugging pipeline execution
• Trackable results — All Result subclasses register with the active tracker for automatic request-level aggregation
• Wiki guide — "Using Data In Your App" wiki page documenting the convenience query API

Changed

• Modernized generators — Agent and install generators produce cleaner, DSL-focused templates with better inline documentation
• Improved error messages — Actionable error messages for common configuration mistakes (missing API key, missing model, etc.)
• README quickstart — Streamlined golden-path getting started section
• Bumped ruby_llm dependency to v1.13.2

Fixed

• Tenant API keys — Use RubyLLM::Context#chat instead of passing context as keyword arg, fixing tenant-scoped API key isolation
• Test isolation — Fixed verify_partial_doubles issues in specs

v3.8.0

Added

• Inception Labs Mercury provider — New provider for diffusion-based LLMs via Inception Labs API. Includes model registry, capabilities detection, and chat integration. Configure with config.inception_api_key
• AgentRegistry service — Centralized agent discovery combining filesystem scanning with execution history for dashboard agent listings
• Dashboard performance indexes — New composite indexes ([status, created_at], [model_id, status], [cache_hit, created_at]) for dashboard query optimization. Available via upgrade generator

Changed

• Dashboard query optimization — Rewrote analytics queries to use SQL GROUP BY with conditional aggregation instead of loading records into Ruby. Reduces dashboard page load from ~75 SQL queries to ~10
• Thin controllers — Extracted query logic from dashboard and agents controllers into models and AgentRegistry service
• Consolidated instrumentation — Deprecated legacy Instrumentation module in favor of pipeline middleware, DRY'd up shared logic
• Thread-safe tenant API keys — Store tenant API keys on pipeline context instead of mutating global configuration

Fixed

• Code quality — Fixed error logging, race conditions in budget tracking, tenant scoping, XSS in views, and test mock violations
• Flaky test — Fixed tenant_auto_create_spec test pollution from leaked track_executions config
• Test coverage — Added specs for TenantsController, SystemConfigController, ExecutionDetail, and Mercury provider

v3.7.2

Fixed

• Fix missing trace_id and request_id extraction in pipeline instrumentation middleware — The instrumentation middleware was not properly extracting trace_id and request_id from the pipeline context, causing these fields to be missing in execution logs.

v3.7.1

Fixed

• Upgrade generator missing usage counter columns migration — The upgrade generator had a template for adding usage counter columns (monthly_cost_spent, daily_cost_spent, etc.) to the tenants table but never wired it into the generator steps. Users upgrading from older versions would hit PG::UndefinedColumn errors on the dashboard. Running rails generate ruby_llm_agents:upgrade now correctly detects and generates the missing migration.

v3.7.0

Added

• Evaluation framework — New RubyLLM::Agents::Eval module for testing and benchmarking agents:
  • EvalSuite — Define test cases with expected outputs, run evaluations with built-in scorers (exact match, includes, semantic similarity, custom), and generate summary reports
  • EvalResult — Structured result objects with pass/fail status, scores, and metadata
  • EvalRun — Batch evaluation runner with parallel execution support and aggregate statistics
• Evaluation examples in example app (SchemaAgentEval, SupportRouterEval)
• Evaluation framework documentation in README, wiki, and LLMS.txt
• Comprehensive unit and integration specs for eval framework components

Fixed

• Dashboard overview stats overflow on mobile viewports

Removed

• Dead redaction references left over from v3.0.0 removal

v3.6.0

Added

• Agent-as-tool composition — Wrap any agent as a RubyLLM tool with AgentTool adapter. Use tools AgentTool.from(OtherAgent) to let agents call other agents. Execution hierarchy tracked via parent_execution_id in metadata
• Queryable agents — Query execution history directly from agent classes: .executions, .last_run, .failures(since:), .total_spent(since:), .stats(since:), .cost_by_model(since:), .with_params(**params). Includes replay support for A/B testing models on real inputs
• ActiveSupport Notifications — Instrument the full pipeline with ruby_llm_agents.* events:
  • Execution lifecycle: execution.start, execution.complete, execution.error
  • Cache: cache.hit, cache.miss, cache.write
  • Budget: budget.check, budget.record, budget.exceeded
  • Reliability: reliability.fallback, reliability.exhausted
• Custom middleware — Inject custom middleware into the pipeline globally via config.middleware or per-agent via middleware DSL. Supports before/after/around positioning relative to built-in middleware
• Agent rename support — Three complementary approaches for handling agent class renames:
  • Aliases DSL — aliases "OldName", "AnotherOldName" on agent classes makes by_agent scopes, analytics, and budget checks automatically include records from all previous names
  • Programmatic helper — RubyLLM::Agents.rename_agent("Old", to: "New") updates execution records and tenant budget keys in-place, with dry_run: true support
  • Migration generator — rails generate ruby_llm_agents:rename_agent OldName NewName creates a reversible migration
  • Rake task — rake ruby_llm_agents:rename_agent FROM=Old TO=New [DRY_RUN=1] as a CLI alternative
• Debugging helpers — Configuration#to_h, BaseAgent.config_summary, Result#execution for console inspection
• Parameter descriptions — desc: / description: keyword on param DSL for documenting agent parameters
• Dashboard enhancements — Custom date range picker for stats/charts, cache savings display, top tenants overview, favicons

Changed

• Tenant middleware now supports objects responding to llm_tenant_id in addition to string/symbol tenant IDs
• Deep symbolize hash keys in agent response processing for consistent access patterns
• Replaced test doubles with real objects across spec suite for more reliable tests

Removed

• Dead code cleanup: removed unused Reliability::Executor sub-classes, duplicate error classes, orphaned async_max_concurrency config, unimplemented redaction stub, hardcoded pricing constants, dead Pipeline convenience methods and Instrumentation diagnostic methods