diff --git a/CHANGELOG.md b/CHANGELOG.md index 1eb3a0ba..275a0d56 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,11 @@ # Legion Changelog +## [1.9.43] - 2026-06-25 + +### Fixed +- API: replaced deleted `MicrosoftTeams::Helpers::TokenCache` reference in `AuthTeams` with `Identity::Entra::Helpers::TokenManager.save_token`, fixing `legionio auth` failures (fixes #229) + + ## [1.9.42] - 2026-06-07 ### Performance diff --git a/README.md b/README.md index 9122eec4..b81da830 100644 --- a/README.md +++ b/README.md @@ -1,621 +1,178 @@

LegionIO

- One Ruby gem that is a distributed async job engine, an AI coding assistant, an MCP server, - and a cognitive-computing platform — and runs with zero required infrastructure. + A modular Ruby framework for async jobs and AI infrastructure.
+ Start with a task engine that runs on zero infrastructure. Add LLM routing with + measured context curation, an MCP server, RBAC, a knowledge store, or an experimental + cognitive layer. Each is an independent gem. All of it is open source. Nothing is gated.

Gem Version Ruby License - HA

-``` - ╭──────────────────────────────────────╮ - │ L E G I O N I O │ - │ │ - │ async jobs · AI chat · MCP │ - │ REST API · HA · cognitive AI │ - │ zero-infra lite mode · Vault │ - ╰──────────────────────────────────────╯ -``` - -> Schedule tasks, chain services into dependency graphs, run them concurrently across a RabbitMQ -> fleet, and orchestrate AI-powered workflows — from a single `legion` command. Then run the whole -> thing with **no RabbitMQ, no Redis, nothing** via lite mode. - -## Why LegionIO - -- 🧩 **Four products in one gem.** A RabbitMQ-backed async **job engine**, an **AI coding assistant** (chat, commit, review, PR, multi-agent), an **MCP server** that exposes your infrastructure to any agent, and a brain-modeled **cognitive platform** — all in one `gem install`. -- 🪶 **Zero-infrastructure lite mode.** `LEGION_MODE=lite` swaps RabbitMQ for in-process pub/sub and Redis/Memcached for an in-memory cache. Every feature still works — `gem install` to a running daemon in seconds, no services to stand up. -- 🔗 **Dependency-graph orchestration.** Chain tasks with JSON conditions and ERB transformations, fan out in parallel, and scale by simply launching more processes — RabbitMQ distributes the work automatically (tested to 100+ workers). -- 🤖 **AI workflows built in.** `legion chat`, `commit`, `review`, `pr`, multi-agent `swarm`, persistent cross-session memory, and a shared knowledge store — powered by [legion-llm](https://github.com/LegionIO/legion-llm)'s any-client → any-provider routing. -- 🧠 **Cognitive architecture.** 240+ brain-modeled extensions across 18 domains (emotion, reasoning, social, metacognition…), coordinated by a tick-cycle scheduler ([legion-gaia](https://github.com/LegionIO/legion-gaia)). -- 🔌 **MCP-native.** Exposes itself as an MCP server (stdio or streamable HTTP), so Claude Desktop or any agent SDK can run tasks, manage extensions, and query your infrastructure directly. -- 🛡️ **Operational from day one.** Vault secrets, AES-256 message encryption, RBAC, JWT / API-key auth, sliding-window rate limiting, Prometheus metrics, an OpenAPI 3.1 spec, and live `SIGHUP` reload. **No paid tiers, no feature gates, full HA out of the box.** - -## The Legion Ecosystem - -LegionIO is the orchestrator; the heavy lifting lives in a family of focused, independently-versioned gems. Here's the one-line version — follow a link to dig in: - -| Gem | What it is | -|-----|-----------| -| [legion-llm](https://github.com/LegionIO/legion-llm) | Universal LLM proxy — any client dialect → any provider, with routing, escalation, and metering | -| [legion-gaia](https://github.com/LegionIO/legion-gaia) | Cognitive coordination layer — tick-cycle scheduler + weighted routing across cognitive modules | -| [legion-apollo](https://github.com/LegionIO/legion-apollo) | Shared + local knowledge store — RAG retrieval, embeddings, and a knowledge graph | -| [legion-data](https://github.com/LegionIO/legion-data) | Persistence — task history, scheduling, and chains over SQLite / PostgreSQL / MySQL | -| [legion-transport](https://github.com/LegionIO/legion-transport) | Messaging abstraction — RabbitMQ AMQP plus the in-process lite adapter | -| [legion-cache](https://github.com/LegionIO/legion-cache) | Caching abstraction — Redis / Memcached plus the in-memory lite adapter | -| [legion-crypt](https://github.com/LegionIO/legion-crypt) | Secrets & encryption — Vault integration, AES-256, JWT auth | -| [legion-rbac](https://github.com/LegionIO/legion-rbac) | Role-based access control with Vault-style flat policies | -| [legion-mcp](https://github.com/LegionIO/legion-mcp) | Model Context Protocol server/client implementation | -| [legion-settings](https://github.com/LegionIO/legion-settings) | Layered configuration + secret resolution (`vault://`, `env://`) | -| [legion-logging](https://github.com/LegionIO/legion-logging) | Structured logging used across every gem | -| [legion-tty](https://github.com/LegionIO/legion-tty) | Terminal UI components — spinners, tables, prompts | - -Capabilities (`lex-*` extensions) are a separate, much larger catalog — see [Extensions](#extensions) below. - -## What Does It Do? - -LegionIO routes work between services asynchronously. Tasks chain into dependency graphs with conditions and transformations controlling data flow: - -``` -Task A ──→ [condition] ──→ Task B ──→ [transform] ──→ Task C - └──→ Task D (parallel) - └──→ Task E ──→ Task F -``` - -When A completes, B runs. B triggers C, D, and E in parallel. Conditions gate execution. Transformations reshape payloads between steps. Add more workers by running more processes — RabbitMQ handles distribution automatically. - -But that's just the foundation. LegionIO is also: - -- **An AI coding assistant** — interactive chat with tools, code review, commit messages, PR generation, and multi-agent workflows -- **An MCP server** — 60 tools that let any AI agent run tasks, manage extensions, and query your infrastructure -- **A cognitive computing platform** — 242 brain-modeled extensions across 18 cognitive domains -- **A digital worker platform** — AI-as-labor with governance, risk tiers, and cost tracking - -## Quick Start - ```bash gem install legionio -legionio check # verify subsystem connections -legionio start # start the daemon -``` - -For the AI features: - -```bash -legion # launch the interactive TTY shell -legion chat # interactive AI REPL with 40 built-in tools -legion commit # AI-generated commit message from staged changes -legion review # AI code review of your code -``` - -### Two Binaries - -| Binary | Purpose | -|--------|---------| -| `legion` | Interactive TTY shell + dev-workflow commands (`chat`, `commit`, `review`, `plan`, `memory`, `init`) | -| `legionio` | Daemon lifecycle + all operational commands (`start`, `stop`, `lex`, `task`, `config`, `mcp`, and 40+ more) | - -`legion` with no args drops into the interactive TTY shell. `legionio` is the full operational CLI. - -## Installation - -```bash -gem install legionio -``` - -Or add to your Gemfile: - -```ruby -gem 'legionio' -``` - -### Optional Capabilities - -| Gem | What It Unlocks | -|-----|-----------------| -| `legion-data` | Task history, scheduling, chains (SQLite/PostgreSQL/MySQL) | -| `legion-llm` | AI chat, commit, review, agents, multi-provider LLM routing | -| `legion-cache` | Redis/Memcached caching for extensions | -| `legion-crypt` | Vault integration, encryption, JWT auth | -| `legion-tty` | TTY UI components (spinners, tables, prompts) | - -## Zero-Infrastructure Mode (Lite) - -Run LegionIO without RabbitMQ, Redis, or Memcached: - -```bash -LEGION_MODE=lite legion start # environment variable -legion start --lite # CLI flag -``` - -In lite mode, `legion-transport` uses an in-process pub/sub adapter (no RabbitMQ required) and `legion-cache` uses a pure in-memory store with TTL (no Redis/Memcached required). All extensions and features work normally. Useful for single-machine development, CI, and trying LegionIO with no infrastructure. - -## Natural Language Intent Router - -```bash -legion do "list all running tasks" -legion do "start the email extension" -``` - -`legion do` routes free-text to the Capability Registry. Routes through the running daemon (MCP Tier 0 fast path) when available, or runs in-process otherwise. - -## Infrastructure - -| Component | Role | Required? | -|-----------|------|-----------| -| **RabbitMQ** | Task distribution (AMQP 0.9.1) | No (lite mode replaces with InProcess adapter) | -| **SQLite/PostgreSQL/MySQL** | Persistence (tasks, scheduling, chains) | Optional | -| **Redis/Memcached** | Extension caching | No (lite mode replaces with Memory adapter) | -| **HashiCorp Vault** | Secrets, PKI, encrypted settings | Optional | - -## The CLI - -Operational commands run through `legionio`. Dev-workflow and AI commands run through `legion`. - -### Daemon & Health - -```bash -legionio start # foreground -legionio start -d # daemonize -legionio start --http-port 8080 # custom API port -legionio status # service status -legionio stop # graceful shutdown -legionio check # smoke-test all connections -legionio check --extensions # also verify extensions -legionio check --full # full boot including API -``` - -### Extensions (LEX) - -Extensions are gems named `lex-*`, auto-discovered at startup: - -```bash -legion lex list # installed extensions -legion lex info # runners, actors, dependencies -legion lex create # scaffold a new extension -legion lex enable # enable / disable -``` - -### Tasks - -```bash -legion task run http.request.get url:https://example.com # dot notation -legion task run -e http -r request -f get # explicit flags -legion task run # interactive picker -legion task list # recent tasks -legion task show # detail + logs -``` - -### AI Chat - -An interactive AI coding assistant with project awareness, persistent memory, tool use, and multi-agent coordination. Requires `legion-llm`. - -```bash -legion chat # interactive REPL -legion chat prompt "explain main.rb" # single-prompt mode -echo "fix the bug" | legion chat prompt - # pipe from stdin -``` - -**40 built-in tools**: read_file, write_file, edit_file, search_files, search_content, run_command, save_memory, search_memory, web_search, spawn_agent, search_traces, query_knowledge, ingest_knowledge, consolidate_memory, relate_knowledge, knowledge_maintenance, knowledge_stats, summarize_traces, list_extensions, manage_tasks, system_status, view_events, cost_summary, reflect, manage_schedules, worker_status, detect_anomalies, view_trends, trigger_dream, generate_insights, budget_status, provider_health, model_comparison, shadow_eval_status, entity_extract, arbitrage_status, escalation_status, graph_explore, scheduling_status, memory_status - -**Slash commands**: `/help` `/quit` `/cost` `/status` `/clear` `/new` `/save` `/load` `/sessions` `/compact` `/fetch URL` `/search QUERY` `/diff` `/copy` `/rewind` `/memory` `/agent` `/agents` `/plan` `/swarm` `/review` `/permissions` `/personality` `/model` `/edit` `/commit` `/workers` `/dream` - -**Bang commands**: `!ls -la` — run shell commands with output injected into context - -**At-mentions**: `@reviewer check main.rb` — delegate to custom agents in `.legion/agents/` - -### AI Workflows - -```bash -legion commit # AI-generated commit message -legion pr # AI-generated PR title + description -legion pr --base develop --draft # target branch, draft mode -legion review # AI code review of staged changes -legion review src/main.rb # review specific files -legion review --diff # review uncommitted diff -``` - -### Multi-Agent Orchestration - -```bash -legion plan # read-only exploration mode (AI reasons, no writes) -legion swarm start deploy-pipeline # run multi-agent workflow -legion swarm list # available workflows -``` - -### Memory - -Persistent project and global memory that survives across sessions: - -```bash -legion memory list # project memories -legion memory add "always use rspec" -legion memory search "testing" -legion memory forget 3 -``` +LEGION_MODE=lite legion start # no RabbitMQ, no Redis, no database required +``` + +## What this is + +LegionIO is a distributed async job engine (RabbitMQ-backed, in the Sidekiq/Celery +family) that chains tasks into dependency graphs, plus a set of optional layers that +install as separate gems: + +| Layer | Gem | What it adds | Requires | +|-------|-----|--------------|----------| +| Task engine | [legionio](https://github.com/LegionIO/LegionIO) | Task chains, scheduling, 8 actor types, CLI, REST API | nothing (lite mode) | +| LLM gateway | [legion-llm](https://github.com/LegionIO/legion-llm) | Any-client-to-any-provider routing, tiered escalation, mid-stream failover, context curation, fleet dispatch to pooled workstation GPUs | usable standalone | +| MCP server | [legion-mcp](https://github.com/LegionIO/legion-mcp) | Exposes tasks and extensions as MCP tools (stdio or HTTP) | legion-llm | +| Knowledge store | [legion-apollo](https://github.com/LegionIO/legion-apollo) | RAG retrieval, embeddings, confidence-decayed knowledge | activated by lex-apollo / lex-knowledge | +| Access control | [legion-rbac](https://github.com/LegionIO/legion-rbac) | Vault-style flat policies | usable standalone | +| Cognitive layer | [legion-gaia](https://github.com/LegionIO/legion-gaia) | Experimental: tick-cycle scheduling over agentic extensions | see "The experimental part" below | -### Knowledge +The composition rules are simple and enforced by gemspecs, not documentation: every +layer is optional, dependencies between layers are declared where they exist +(legion-mcp depends on legion-llm; Apollo does little until a lex-* extension activates +it), and installing a core gem you don't use adds a contract, not overhead. You can run +the LLM gateway without the task engine, the task engine without any AI, or the whole +stack together. -Query and manage the Apollo shared knowledge store and local knowledge index: +There are no paid tiers, no enterprise editions, and no feature gates. RBAC, the audit +ledger, identity integration, and every operational control ship in the open-source +gems, because there is no commercial version for them to be held back for. -```bash -legion knowledge query "how does transport routing work?" -legion knowledge retrieve "embedding cosine similarity" --scope global -legion knowledge ingest /path/to/docs/ -legion knowledge status # index stats, embedding coverage -legion knowledge health # detect orphans, quality metrics -legion knowledge maintain # cleanup orphans, reindex -legion knowledge quality # quality report -``` +## The part most worth your skepticism budget: legion-llm -### Mind Growth +legion-llm is a proxy/gateway between AI clients and model backends, in the same +category as LiteLLM or OpenRouter, with one addition neither has: automatic context +curation for long agent sessions. -Autonomous cognitive architecture expansion system. Analyzes gaps, proposes new cognitive extensions, and builds them via a staged pipeline: +**Routing.** Every model is classified into a tier: +[`local(0) → direct(1) → fleet(2) → cloud(3) → frontier(4)`](https://github.com/LegionIO/legion-llm/blob/main/lib/legion/llm/router.rb). +Requests try the cheapest capable tier first and escalate on failure or capability +mismatch. A [health tracker](https://github.com/LegionIO/legion-llm/blob/main/lib/legion/llm/router/health_tracker.rb) +(300s window, 3 failures trips the circuit, 60s cooldown) keys availability per +provider instance, and a provider dying mid-stream fails over and continues the stream +rather than erroring the client. -```bash -legion mind-growth status # current growth cycle state -legion mind-growth analyze # gap analysis against 5 reference models -legion mind-growth propose # propose a new concept -legion mind-growth evaluate # evaluate a proposal -legion mind-growth build # run staged build pipeline -legion mind-growth list # list proposals -legion mind-growth approve # manually approve -legion mind-growth reject # manually reject -legion mind-growth profile # cognitive profile across all models -legion mind-growth health # extension fitness validation -``` +**Curation.** After each turn (async, off the request path), the +[Curator](https://github.com/LegionIO/legion-llm/blob/main/lib/legion/llm/context/curator.rb) +shrinks accumulated history with six deterministic strategies: `strip_thinking`, +`distill_tool_result`, `fold_resolved_exchanges`, `evict_superseded`, `dedup_similar`, +and `drop_and_archive` (overflow goes to the knowledge store, not the trash). -Requires `lex-mind-growth`. Also exposes 6 MCP tools in the `legion.mind_growth_*` namespace. - -### Digital Workers - -AI-as-labor with governance, risk tiers, and cost tracking: - -```bash -legion worker list # list workers -legion worker show # worker detail -legion worker create # register new worker (bootstrap state) -legion worker pause # pause / activate / retire -legion worker costs --days 30 # cost report -``` +**Measured, including where it does nothing.** Production ledger aggregates, all +requests, by conversation length: -### Code Generation +| Turns | 1 | 2–3 | 4–5 | 6–9 | 10–19 | 20–49 | 50+ | +|-------|---|-----|-----|-----|-------|-------|-----| +| Reduction vs. naive full-history resend | -0.1% | 9.6% | 13.3% | 23.6% | 54.3% | 72.8% | **97.7%** | -Run inside a `lex-*` directory: +Single-turn workloads gain nothing; long agent sessions go from an average 1.13M +naive tokens per turn to ~26K. The asymmetry is the point: curation doesn't make cheap +traffic cheaper, it bounds the runaway sessions where cost concentrates. Methodology, +baseline definition, raw numbers, and caveats: +[curation-production-metrics.md](https://github.com/LegionIO/legion-llm/blob/main/docs/curation-production-metrics.md). -```bash -legion generate runner # add runner + spec -legion generate actor # add actor + spec -legion g exchange # 'g' is an alias -``` +Nine provider adapters exist today (Anthropic, OpenAI, Bedrock, Gemini, Vertex, +Azure Foundry, Ollama, vLLM, MLX), each its own gem built on the +[lex-llm](https://github.com/LegionIO/lex-llm) contract layer. lex-llm defines what a +provider is; legion-llm decides where traffic goes. Install only the adapters you use. -### Scheduling +## The job engine -Requires `lex-scheduler`: +The original core, in production since before any of the AI layers existed: -```bash -legion schedule add alerts "*/5 * * * *" http.request.get -legion schedule add daily "every day at noon" report.generate.summary -legion schedule list -``` +- Task chains with conditions and transformations: + `Task A → [condition] → Task B → [transform] → Task C`, with parallel fan-out. +- Eight actor types (subscription, poll, every, once, loop, singleton, nothing, + absorber_dispatch) — see + [lib/legion/extensions/actors/](https://github.com/LegionIO/LegionIO/tree/main/lib/legion/extensions/actors). +- Distributed cron scheduling with interval locking. +- A JSONL disk spool that buffers messages through AMQP outages. +- Scale by starting more processes; RabbitMQ distributes the work. -### Configuration +`LEGION_MODE=lite` swaps RabbitMQ for in-process pub/sub and Redis for an in-memory +cache. Every feature works; nothing external is required. This is the recommended way +to evaluate the framework, and it takes about five minutes. -```bash -legion config show # resolved config (redacted) -legion config validate # verify settings + subsystem health -legion config scaffold # generate starter config files (auto-detects env vars) -``` +## The experimental part, labeled as such -`config scaffold` auto-detects environment variables (`ANTHROPIC_API_KEY`, `AWS_BEARER_TOKEN_BEDROCK`, `OPENAI_API_KEY`, `GEMINI_API_KEY`, `VAULT_TOKEN`, `RABBITMQ_USER`/`PASSWORD`) and a running Ollama instance, enabling providers and setting `env://` references automatically. +The `lex-agentic-*` gems are a research layer: 16 gems containing 369 actor and runner +modules that model cognition-inspired behaviors on top of the job engine. The honest +mechanical description is that each is a scheduled job or subscription that adjusts +persistent state. The interesting research idea is what that state does: task-routing +connections strengthen when chains succeed and +[decay](https://github.com/LegionIO/lex-synapse) when unused, so frequently-successful +paths get cheaper to select. A +[16-phase tick cycle](https://github.com/LegionIO/lex-tick/blob/main/lib/legion/extensions/tick/helpers/constants.rb) +schedules this work in budgeted modes (dormant 0.2s, sentinel 0.5s, full 5.0s), and a +10-phase idle-time cycle consolidates memory and feeds what it learns about your usage +back into RAG retrieval for the LLM layer. -Settings load from the first directory found: `/etc/legionio/` → `~/.legionio/settings/` → `~/legionio/` → `./settings/` +Whether this framing earns its keep is an open question we're running in production to +answer. If you don't install these gems, none of this exists in your deployment. -### Observability +## Interfaces ```bash -legion dashboard # TUI operational dashboard with auto-refresh -legion cost summary # cost overview (today/week/month) -legion cost worker # per-worker cost breakdown -legion trace search "failed tasks last hour" # natural language trace search -legion graph show --format mermaid # task relationship graph +legion start # the engine +legion chat # agentic REPL with tools, memory, subagents +legion task run http.request.get url:https://example.com +legion mcp # MCP server over stdio or streamable HTTP +curl http://localhost:4567/api/v1/tasks # REST API (OpenAPI 3.1 spec in legionio-spec) ``` -### Audit & RBAC +Every CLI command supports `--json`. MCP tools are discovered at runtime from +installed extensions, so the tool list reflects what your deployment can actually do. -```bash -legion audit list # query audit log -legion audit export --format csv -legion rbac roles # list roles -legion rbac check -``` +## Verify the claims yourself -### Diagnostics +Every number above is reproducible from public source. A few one-liners: ```bash -legion doctor # diagnose environment, suggest fixes -legion doctor --fix # auto-remediate fixable issues (stale PIDs, missing gems) -legion doctor --json # machine-readable output -``` +# tick and dream phase counts (16 and 10) +git clone https://github.com/LegionIO/lex-tick && \ + grep -A20 'PHASES = ' lex-tick/lib/legion/extensions/tick/helpers/constants.rb -Checks Ruby version, bundle status, config files, RabbitMQ, database, cache, Vault, extensions, PID files, and permissions. Exits 1 if any check fails. +# legion-llm test surface (269 spec files, ~3,050 examples) +git clone https://github.com/LegionIO/legion-llm && \ + find legion-llm/spec -name '*_spec.rb' | wc -l -### Updating - -```bash -legion update # update all legion gems in-place -legion update --dry-run # check what's available without installing +# router tiers and circuit-breaker defaults +grep -n 'TIER_RANK\|failure_threshold' legion-llm/lib/legion/llm/router.rb \ + legion-llm/lib/legion/llm/router/health_tracker.rb ``` -Uses the same Ruby that `legion` is running from — safe for Homebrew installs (updates go into the bundled gem directory, not your system Ruby). - -All commands support `--json` for structured output and `--no-color` to strip ANSI codes. - -## REST API - -The daemon exposes a REST API on port 4567 (configurable): - -| Route | Description | -|-------|-------------| -| `GET /api/health` | Health check | -| `GET /api/ready` | Readiness + component status | -| `GET/POST /api/tasks` | List / create tasks | -| `GET /api/extensions` | Installed extensions + runners | -| `GET /api/nodes` | Cluster nodes | -| `GET/POST/PUT/DELETE /api/schedules` | Cron / interval scheduling | -| `GET /api/settings` | Config (sensitive values redacted) | -| `GET /api/transport` | RabbitMQ connection status | -| `GET /api/events` | SSE event stream | -| `GET/POST/PUT/DELETE /api/workers` | Digital worker lifecycle | -| `GET /api/capacity` | Workforce capacity and forecasting | -| `GET /api/tenants` | Multi-tenant management | -| `GET /api/audit` | Audit log query and export | -| `GET /api/rbac/*` | Role-based access control | -| `GET /api/webhooks` | Webhook subscription management | -| `GET /api/openapi.json` | OpenAPI 3.1.0 specification | -| `GET /metrics` | Prometheus metrics | -| `POST /api/coldstart/ingest` | Context ingestion | - -```json -{ - "data": { "..." }, - "meta": { "timestamp": "2026-03-15T12:00:00Z", "node": "legion-01" } -} -``` - -## MCP Server - -LegionIO exposes itself as an [MCP](https://modelcontextprotocol.io/) server, letting any AI agent run tasks, manage extensions, and query infrastructure directly. - -```bash -legion mcp # stdio transport (Claude Desktop, agent SDKs) -legion mcp http # streamable HTTP on localhost:9393 -legion mcp http --port 8080 --host 0.0.0.0 -``` - -**60 tools** in the `legion.*` namespace: - -| Category | Tools | -|----------|-------| -| **Agentic** | `run_task`, `describe_runner` | -| **Tasks** | `list_tasks`, `get_task`, `delete_task`, `get_task_logs` | -| **Extensions** | `list_extensions`, `get_extension`, `enable_extension`, `disable_extension` | -| **Chains** | `list_chains`, `create_chain`, `update_chain`, `delete_chain` | -| **Relationships** | `list_relationships`, `create_relationship`, `update_relationship`, `delete_relationship` | -| **Schedules** | `list_schedules`, `create_schedule`, `update_schedule`, `delete_schedule` | -| **System** | `get_status`, `get_config` | -| **Workers** | `list_workers`, `show_worker`, `worker_lifecycle`, `worker_costs`, `team_summary` | -| **RBAC** | `rbac_assignments`, `rbac_check`, `rbac_grants` | -| **Analytics** | `routing_stats` | -| **Knowledge** | `query_knowledge`, `knowledge_health` | -| **Mind Growth** | `mind_growth_status`, `mind_growth_analyze`, `mind_growth_propose`, `mind_growth_evaluate`, `mind_growth_build`, `mind_growth_profile` | - -**Resources**: `legion://runners` (full runner catalog), `legion://extensions/{name}` (extension detail) - -## Task Relationships - -### Conditions - -JSON rule engine via `lex-conditioner`. Supports nested `all`/`any` with operators: - -```json -{ - "all": [ - {"fact": "pet.type", "value": "dog", "operator": "equal"}, - {"fact": "pet.hungry", "operator": "is_true"} - ] -} -``` - -### Transformations - -ERB templates via `lex-transformer`. Map data between services: - -```json -{"message": "Incident assigned to <%= assignee %> with priority <%= severity %>"} -``` - -Access Vault secrets inline: `<%= Legion::Crypt.read('pushover/token') %>` - -## Extensions - -Browse: [LegionIO GitHub](https://github.com/LegionIO) | [legionio topic](https://github.com/topics/legionio?l=ruby) - -### Core (14 operational extensions) - -`lex-node` `lex-tasker` `lex-conditioner` `lex-transformer` `lex-scheduler` `lex-health` `lex-log` `lex-ping` `lex-exec` `lex-lex` `lex-codegen` `lex-metering` `lex-telemetry` `lex-task_pruner` - -### Agentic (242 cognitive extensions) - -Brain-modeled cognitive architecture. 20 core orchestration extensions plus 222 expanded modules across 18 domains: - -| Domain | Examples | -|--------|----------| -| **Orchestration** | `lex-tick`, `lex-cortex`, `lex-dream`, `lex-memory`, `lex-identity` | -| **Emotion** | `lex-emotion`, `lex-mood`, `lex-empathy` | -| **Reasoning** | `lex-prediction`, `lex-planning`, `lex-logic` | -| **Social** | `lex-trust`, `lex-consent`, `lex-governance` | -| **Metacognition** | `lex-reflection`, `lex-awareness`, `lex-curiosity` | - -Coordinated by [legion-gaia](https://github.com/LegionIO/legion-gaia), the cognitive coordination layer with tick-cycle scheduling, channel abstraction, and weighted routing across cognitive modules. +Engineering docs are public in each repo, including the +[router design doc](https://github.com/LegionIO/legion-llm/tree/main/docs/work/planning) +and the [debugging methodology](https://github.com/LegionIO/legion-llm/blob/main/docs/work/planning/nxn-debugging-method.md) +the routing layer is held to. They are the most accurate picture of how this project +is actually built. -### AI / LLM - -`legion-llm` `lex-llm` `lex-llm-anthropic` `lex-llm-azure-foundry` `lex-llm-bedrock` `lex-llm-gemini` `lex-llm-ledger` `lex-llm-mlx` `lex-llm-ollama` `lex-llm-openai` `lex-llm-vertex` `lex-llm-vllm` - -Powered by [legion-llm](https://github.com/LegionIO/legion-llm) with provider-neutral model offerings, local and fleet routing, hosted cloud providers, health tracking, metering, and automatic model discovery. - -LLM API routes are mounted from `legion-llm` when available; LegionIO only hosts those route modules and does not provide a provider gateway fallback. - -### Service Integrations (8 common + 15 additional) - -**Common**: `lex-http` `lex-redis` `lex-s3` `lex-github` `lex-consul` `lex-tfe` `lex-vault` `lex-kerberos` `lex-microsoft_teams` - -**Additional**: `lex-ssh` `lex-slack` `lex-smtp` `lex-influxdb` `lex-pagerduty` `lex-elasticsearch` `lex-chef` `lex-pushover` `lex-twilio` `lex-todoist` `lex-pushbullet` `lex-sleepiq` `lex-elastic_app_search` `lex-memcached` `lex-sonos` - -### Build Your Own - -```bash -legion lex create myextension -cd lex-myextension -legion generate runner myrunner -legion generate actor myactor -bundle exec rspec -``` - -## Role Profiles - -Control which extensions load at startup via `settings/legion.json`: - -```json -{"role": {"profile": "dev"}} -``` - -| Profile | What loads | -|---------|-----------| -| *(default)* | Everything — no filtering | -| `core` | 14 core operational extensions only | -| `cognitive` | core + all agentic extensions | -| `service` | core + service + other integrations | -| `dev` | core + native LLM providers + essential agentic (~20 extensions) | -| `custom` | only what's listed in `role.extensions` | - -Faster boot and lower memory footprint for dedicated worker roles. - -## Scaling - -Task distribution uses RabbitMQ FIFO queues. Add workers by running more Legion processes — each subscribes to the same queues and picks up work automatically. Tested to 100+ workers. - -Run different LEX combinations per worker: 10 pods focused on `lex-ssh`, a separate pod for `lex-pagerduty` + `lex-log` notifications. - -No paid tiers. No feature gates. Full HA out of the box. - -## Security - -- **Message encryption**: AES-256-CBC via `legion-crypt` -- **Vault integration**: Secrets, PKI, encrypted settings -- **Node identity**: Each worker generates a keypair for inter-node communication -- **Cluster secret**: Generated at first startup, distributed via Vault or in-memory -- **JWT auth**: Bearer token authentication on the REST API -- **API key support**: `X-API-Key` header authentication -- **RBAC**: Role-based access control with Vault-style flat policies -- **Rate limiting**: Sliding-window per-IP/agent/tenant rate limiting -- **API versioning**: `/api/v1/` prefix with deprecation headers -- **Kerberos**: SPNEGO/GSSAPI authentication with LDAP group resolution - -## Docker - -```bash -docker pull legionio/legion -``` - -```dockerfile -FROM ruby:3-alpine -RUN gem install legionio -CMD ruby --yjit $(which legion) start -``` - -## Architecture - -Before any Legion code loads, the executable applies performance optimizations: - -- **YJIT** — `RubyVM::YJIT.enable` for 15-30% runtime throughput (Ruby 3.1+ builds) -- **GC tuning** — pre-allocates 600k heap slots and raises malloc limits (ENV overrides respected) -- **bootsnap** *(opt-in)* — set `LEGION_BOOTSNAP=true` to cache YARV bytecode and `$LOAD_PATH` resolution at `~/.legionio/cache/bootsnap/` - -``` -legion start - └── Legion::Service - ├── 1. Logging (legion-logging) - ├── 2. Settings (legion-settings — /etc/legionio, ~/legionio, ./settings) - ├── 3. Crypt (legion-crypt — Vault connection) - ├── 4. Transport (legion-transport — RabbitMQ) - ├── 5. Cache (legion-cache — Redis/Memcached) - ├── 6. Data (legion-data — database + migrations) - ├── 7. RBAC (legion-rbac — optional role-based access control) - ├── 8. LLM (legion-llm — AI provider setup + routing) - ├── 9. Apollo (legion-apollo — shared/local knowledge store) - ├── 10. GAIA (legion-gaia — cognitive coordination layer) - ├── 11. Telemetry (OpenTelemetry — optional) - ├── 12. Supervision (process supervision) - ├── 13. Extensions (two-phase parallel load: require+autobuild, then hook_all_actors) - ├── 14. Cluster Secret (distribute via Vault or memory) - └── 15. API (Sinatra/Puma on port 4567) -``` - -Each phase registers with `Legion::Readiness`. All phases are individually toggleable. - -`SIGHUP` triggers a live reload (`Legion.reload`) — subsystems shut down in reverse order and restart fresh without killing the process. Useful for rolling restarts and config changes. - -## Similar Projects - -| Project | Language | HA | AI | Cognitive | -|---------|----------|----|----|-----------| -| **LegionIO** | Ruby | Yes | Chat, MCP, agents, LLM routing | 242 extensions | -| [Node-RED](https://nodered.org/) | JS | No | No | No | -| [n8n.io](https://n8n.io/) | TS | Limited | Limited | No | -| [StackStorm](https://stackstorm.com/) | Python | Yes | No | No | -| [Huginn](https://github.com/huginn/huginn) | Ruby | No | No | No | - -## Development - -```bash -git clone https://github.com/LegionIO/LegionIO.git -cd LegionIO -bundle install -bundle exec rspec # ~3500+ examples, 0 failures -bundle exec rubocop # 0 offenses -``` +## Project status, honestly -Always run `bundle exec rspec` and `bundle exec rubocop -A` and fix all errors before committing. +LegionIO is built primarily by one engineer, with a disciplined process: PR-based flow, +CI on every repo, RSpec and RuboCop green before merge, conventional commits. The +GitHub org dates to 2018 (the job engine's first life); the AI platform is a 2025–2026 +rebuild, which is why most public repos are young. It runs production workloads daily. +It is early, it is small, and the code is real. Read the source before betting on it — +that's what it's there for. -### Project Structure +The commit history shows the shape plainly: a 2018-era org, early gems from 2019, and an intense 2025–2026 rebuild sprint. That rebuild was heavily AI-assisted — and the LLM traffic behind it was routed through LegionIO itself, which means the production metrics published here are largely the workload of building the thing they describe. The 1:1 test-to-code ratio, PR-gated flow, and conventional commits are what make that velocity safe; the audit ledger is its receipt trail. -| Path | Purpose | -|------|---------| -| `lib/legion.rb` | Entry point: `Legion.start`, `.shutdown`, `.reload` | -| `lib/legion/service.rb` | 15-phase startup orchestrator | -| `lib/legion/cli.rb` | Thor CLI: 40+ subcommands across two binaries | -| `lib/legion/api.rb` | Sinatra REST API with middleware stack | -| `lib/legion/extensions/` | LEX discovery, loading, actors, builders | -| `lib/legion/tools/` | Canonical tool layer (Registry, Discovery, EmbeddingCache) | -| `lib/legion/digital_worker/` | AI-as-labor governance platform | -| `lib/legion/cli/chat/` | Interactive AI REPL with 40 tools | -| `spec/` | RSpec suite (~3500+ examples) | +LegionIO is also published through [Optum Open Source](https://github.com/Optum/LegionIO), where it first shipped publicly; this org is the active development home, and updates are periodically merged back upstream. The enterprise provenance is why a framework this young ships RBAC, an audit ledger, and identity integration — a real production deployment required them. -### Contributing +## Requirements -1. Fork the repo and create a feature branch -2. Write specs for new functionality -3. Ensure `bundle exec rspec` passes with 0 failures -4. Ensure `bundle exec rubocop` passes with 0 offenses -5. Open a PR targeting `main` +- Ruby >= 3.4 +- Nothing else in lite mode. Full mode: RabbitMQ; optional PostgreSQL/MySQL/SQLite, + Redis/Memcached, HashiCorp Vault. ## License -Apache-2.0 +Core framework: [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0). +Extensions: [MIT](https://opensource.org/licenses/MIT). diff --git a/legionio.gemspec b/legionio.gemspec index 54b3144c..f259d970 100755 --- a/legionio.gemspec +++ b/legionio.gemspec @@ -10,8 +10,11 @@ Gem::Specification.new do |spec| spec.authors = ['Esity'] spec.email = ['matthewdiverson@gmail.com'] - spec.summary = 'The primary gem to run the LegionIO Framework' - spec.description = 'LegionIO is an extensible framework for running, scheduling and building relationships of tasks in a concurrent matter' + spec.summary = 'Modular Ruby framework: distributed async job engine with optional LLM gateway, MCP server, and RBAC' + spec.description = 'LegionIO chains tasks into dependency graphs across a RabbitMQ fleet, or fully in-process via ' \ + 'lite mode with zero infrastructure. Optional layers install as independent gems: legion-llm adds ' \ + 'tiered LLM routing with mid-stream failover and context curation, legion-mcp adds an MCP server, ' \ + 'legion-rbac adds access control. All open source, no feature gates.' spec.homepage = 'https://github.com/LegionIO/LegionIO' spec.license = 'Apache-2.0' spec.require_paths = ['lib'] diff --git a/lib/legion/api/auth_teams.rb b/lib/legion/api/auth_teams.rb index 8b3f6353..c8c83d55 100644 --- a/lib/legion/api/auth_teams.rb +++ b/lib/legion/api/auth_teams.rb @@ -129,15 +129,14 @@ def self.register_callback(app) module TeamsTokenHelper def store_teams_token(token_body, scopes) - require 'legion/extensions/microsoft_teams/helpers/token_cache' - cache = Legion::Extensions::MicrosoftTeams::Helpers::TokenCache.new - cache.store_delegated_token( + require 'legion/extensions/identity/entra/helpers/token_manager' + Legion::Extensions::Identity::Entra::Helpers::TokenManager.save_token( + :delegated, access_token: token_body[:access_token], refresh_token: token_body[:refresh_token], expires_in: token_body[:expires_in] || 3600, scopes: scopes ) - cache.save_to_vault Legion::Logging.info 'Teams delegated token stored' if defined?(Legion::Logging) rescue StandardError => e Legion::Logging.warn "Failed to store Teams token: #{e.message}" if defined?(Legion::Logging) diff --git a/lib/legion/version.rb b/lib/legion/version.rb index 484813b0..969995f6 100644 --- a/lib/legion/version.rb +++ b/lib/legion/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module Legion - VERSION = '1.9.42' + VERSION = '1.9.43' end