Operon

Operon is a runtime for AI tasks.
Operon standardizes how applications call AI by routing all requests through a shared execution layer. Instead of scattering prompts and model calls across a codebase, Operon provides a small set of well-defined task types (“opcodes”) with stable schemas and consistent behavior.

This means you can:

• Map and manage task-type → model routing from a single place
• Monitor and control cost and performance by task type
• Avoid duplicated AI-calling logic across the codebase

Operon is open source and maintained by DeepModel.

Why Operon Exists

While building Spark, our AI workspace, we realized we had more than 50 places in the codebase making AI calls. We hadn’t even shipped v1 yet, and it was already getting messy.

We didn’t know which tasks needed high-end models and which didn’t. Most tasks were using expensive models without a good reason. We didn’t have a way to track cost or performance by task type. And swapping models required touching application code.

This happens in every AI product:

• AI calls spread quickly
• Different teams write their own prompts
• Models get swapped ad-hoc
• Costs are unpredictable
• Logic gets duplicated across services

We needed a way to centralize and standardize how AI is executed.
That’s why Operon exists.

What Operon Does

Operon abstracts AI calls into reusable tasks. Application developers invoke tasks instead of wiring model calls directly.

With Operon:

• Each task type has a clear contract.
• Admins choose the model for each task.
• The runtime handles routing, execution, policies, and metering.

This keeps application code clean and gives organizations control over cost and behavior.

Operon provides:

• A fixed set of AI task types (“opcodes”)
• A single invocation API
• Model routing by task type
• Structured task configs and validation
• Root prompts for predictable behavior
• Cost and usage metering
• Request policies (intercept or log)
• Tracing and logging

Operon is not a framework.
It doesn’t tell you how to build your application.
It only standardizes how AI tasks are executed.

Task Types (Opcodes)

Operon defines a core set of operations:

• title_generation — generate concise titles
• summarization — produce structured or free-form summaries
• document_generation — generate longer documents (emails, specs, reports)
• image_generation — create images from prompts
• diagram_generation — generate diagrams (Mermaid, PlantUML, Graphviz)
• text_classification — label text with categories or scores
• data_extraction — extract structured fields from raw text
• data_validation — validate input against rules or schemas
• text_transform — rewrite tone, style, or structure
• data_transform — transform JSON/records into new formats
• recommendation — rank or suggest options with rationale
• reasoning — stepwise problem-solving or analysis
• streaming_chat — multi-turn streaming interactions
• orchestration — plan multi-step flows using other tasks

Each task has a simple, stable schema and optional config.

Request Schema (v0.1)

{
  "task": {
    "type": "summarization",
    "instruction": "Summarize this text concisely.",
    "config": {}
  },
  "input": "Text to summarize...",
  "tags": ["example"],
  "meta": {
    "user_id": "123",
    "workspace_id": "abc"
  }
}

The runtime uses:

• task.type to determine routing
• task.instruction and config to build the root prompt
• tags and meta to attribute cost and usage

Minimal Example (Python)

Placeholder. Runtime implementation coming soon.

from operon import OperonClient

client = OperonClient(base_url="http://localhost:8000")

result = client.invoke({
    "task": {
        "type": "summarization",
        "instruction": "Summarize briefly"
    },
    "input": "Long text here..."
})

print(result.output)

Roadmap

v0.1 — Prototype

• Basic runtime structure
• Summarization opcode
• Local model routing
• Simple metrics and tracing
• REST API for invocation

v0.2 — OSS Release

• Additional task types (classification, extraction, transform)
• Config schemas
• Policy engine
• Python + JS SDKs
• Documentation site

v0.3 — Hardening

• Streaming chat
• Orchestration task
• Logging integrations
• OpenTelemetry export

v1.0 — Stable

• Declarative model routing
• Full schema validation
• Dashboard (commercial)
• Quotas and governance (commercial)

Status

Operon is early and evolving quickly. The core runtime API and task schemas may change during v0.x.

License

Apache 2.0

Maintainer

Operon is an open-source project maintained by DeepModel. Website: https://operon.sh