Skip to content

AGENTS.md

Latest commit

 

History

History
446 lines (353 loc) · 21.5 KB

AGENTS.md

File metadata and controls

446 lines (353 loc) · 21.5 KB

Agent Development Guide

What Are Agents?

Agents are AI-powered assistants that automate tasks on your GitLab instance. Each agent specializes in a specific workflow (code review, pipeline debugging, etc.) and is orchestrated by the Codeward runtime.

Architecture

Webhook Event (MR / Note / Pipeline / Job)            Slash command on issue/MR
    │                                                          │
    ▼                                                          ▼
┌──────────┐     ┌────────────┐     ┌───────────────┐   ┌──────────────┐
│  Gateway │────▶│Orchestrator│────▶│ Agent Runner  │   │ Flow Runner  │
│ router.py│     │            │     │  (Agno Agent) │◀──│  (Redis)     │
└──────────┘     └────────────┘     └──────┬────────┘   └──────────────┘
                       │                   │
                 ┌─────┴──────┐       ┌────┴───────────┐
                 │ • Match    │       │ Agent Toolset  │
                 │ • Cooldown │       │ ┌────────────┐ │
                 │ • Rate lim │       │ │GitLabToolk.│ │
                 │ • Context  │       │ ├────────────┤ │
                 │ • Record   │       │ │External    │ │
                 │   run      │       │ │MCP tools   │ │
                 └────────────┘       │ │(per agent) │ │
                                      │ └────────────┘ │
                                      └────────────────┘

  1. Gateway receives a GitLab webhook and dispatches to the orchestrator as a background task.

  2. Orchestrator loads the project config (.codeward.yml), matches the event to agents, checks cooldowns/rate limits, builds context via ContextEngine, then runs each matched agent.

  3. Context Engine builds an AgentContext appropriate to the event type — MR diffs for code review, parsed job logs for pipeline debugging, security report artifacts for triage, etc.

  4. Agent is an Agno Agent whose toolset is the union of:

    • GitLabToolkit — posting inline comments, notes, labels, and fetching files from the GitLab REST API.
    • External MCP tools — any tools exported by the MCP servers the agent declares under mcp_servers: (see Using External MCP Servers from Agents). These are merged in by MCPConnectionManager.get_tools_for_agent in codeward/mcp_client/manager.py.

    The agent returns a structured result (ReviewResult, PipelineDebugResult, or SecurityTriageResult) for the summary.

  5. Flow Runner (separate path, in codeward/flows/runner.py) is invoked by slash commands like @codeward /full-review and chains multiple agent runs and GitLab actions through a Redis-backed state store. See Multi-Agent Flows below.

Built-in Agents

Code Review Agent (agents/code-reviewer.yml)

Automatically reviews merge request diffs when an MR is opened or updated.

Triggers: merge_request.open, merge_request.update Model alias: code (configurable in codeward.yml) Skips: Draft MRs, WIP MRs (configurable) Cooldown: 120 seconds between re-reviews on the same MR

What it does:

  • Posts inline diff comments on specific lines using the post_inline_comment tool
  • Posts a summary note with a severity table
  • Adds/removes labels based on findings
  • Respects skip_paths and review_rules from the project's .codeward.yml

Chat Agent (agents/chat.yml)

Responds to @codeward mentions in MR comments and issue discussions.

Triggers: note (with @codeward mention required) Model alias: strong Cooldown: N/A (conversation-driven)

What it does:

  • Answers questions about the MR diff, project code, or CI pipeline
  • Supports slash commands: /review, /debug, /security
  • Maintains conversation context via the conversation store
  • Replies in the same discussion thread

Pipeline Debugger (agents/pipeline-debugger.yml)

Automatically diagnoses CI/CD pipeline failures by analyzing job logs and .gitlab-ci.yml.

Triggers: pipeline.failed Model alias: strong Cooldown: 300 seconds Slash command: @codeward /debug

What it does:

  • Analyzes failed job logs using the log parser (error classification, context extraction)
  • Fetches and reviews .gitlab-ci.yml for misconfigurations
  • Classifies errors by type: test failure, dependency, docker, timeout, syntax, permission, network, config
  • Extracts exit codes and identifies flaky failures
  • Correlates failures with the MR diff (if a merge request triggered the pipeline)
  • Posts a structured summary note with per-job root cause analysis and fix suggestions
  • Falls back to posting a commit comment when no MR is associated

Output schema: PipelineDebugResult — summary, is_flaky flag, per-job analyses (FailedJobAnalysis), suggested actions.

Security Triage (agents/security-triage.yml)

Triages findings from GitLab security scanners (SAST, DAST, dependency scanning, secret detection, container scanning).

Triggers: build.success (filtered by job_names) Job names: semgrep-sast, sast, bandit-sast, gemnasium-dependency_scanning, dependency_scanning, dast, secret_detection, container_scanning Model alias: strong Cooldown: 600 seconds Slash command: @codeward /security

What it does:

  • Fetches security report artifacts from completed scan jobs (e.g., gl-sast-report.json)
  • Parses GitLab's standard security report format (v14.x/v15.x)
  • Prioritizes findings by severity (Critical > High > Medium > Low)
  • Identifies false positives based on common scanner patterns
  • Posts a structured triage report with risk level, confirmed findings, and false positive justifications
  • Applies labels when enabled (e.g., security::high, security::critical)

Opt-in: Auto-triggering is opt-in per project. Add security-triage to enabled_agents in .codeward.yml to activate:

codeward:
  enabled_agents:
    - code-reviewer
    - security-triage

Output schema: SecurityTriageResult — summary, risk level, per-finding triage (SecurityFindingResult), false positive count, label suggestions.

Agent YAML Schema

Agents are defined as YAML files in the directory specified by CODEWARD_AGENTS_DIR (default: ./agents).

# Required
name: my-agent              # Unique identifier
type: my-agent              # Agent type — maps to a CodewardAgent subclass (see dispatch.py)
version: "1.0"              # Semver string
description: >              # What the agent does
  Human-readable description.

# When to trigger
trigger:
  events:                   # List of "event_type.action" strings
    - merge_request.open    # MR opened
    - merge_request.update  # MR updated (new commits)
    - pipeline.failed       # Pipeline finished with failure
    - build.success         # Job (build) completed successfully
    - note                  # Comment/note posted (usually with mention)
  job_names:                # Filter build events to specific job names (optional)
    - semgrep-sast
    - sast
  cooldown: 120             # Seconds between re-runs on the same target
  skip_draft: true          # Skip draft MRs (MR events only)
  skip_wip: true            # Skip WIP-prefixed MRs (MR events only)

# LLM model
model:
  default: code             # Model alias from codeward.yml

# What context the agent needs
context:
  needs_diff: true
  needs_file_context: true
  needs_pipeline_logs: false
  needs_ci_config: false
  needs_security_reports: false
  max_files: 15

# System prompt template
# Available variables: {project_path}, {project_rules}, {source_branch}, {target_branch}
system_prompt: |
  You are an expert code reviewer for {project_path}.
  ...

# Output schema (for structured response)
output:
  format: json

# What actions the agent is allowed to perform
actions:
  inline_comments: true     # Post inline diff comments
  summary_note: true        # Post a summary comment
  labels: true              # Add/remove labels
  approve: false            # Auto-approve MRs (use with caution)

# Optional: external MCP servers whose tools should be exposed to this agent.
# Each name must match a key in `codeward.mcp_servers` in codeward.yml.
mcp_servers:
  - context7
  - sentry

Supported Event Types

Event object_kind Actions Description
Merge Request merge_request open, update, close, reopen, merge MR lifecycle events
Note note — Comments on MRs, issues, commits. Use mention: true to require @codeward
Pipeline pipeline failed, success, running Pipeline status changes
Job (Build) build success, failed Individual CI job completion. Use job_names to filter
Push push — Code pushed to a branch

Using External MCP Servers from Agents

Agents can be granted tools from any external Model Context Protocol server (e.g. Context7, Sentry, a vendor-specific knowledge base) without writing any Python. The flow is:

  1. Declare the server under codeward.mcp_servers in codeward.yml.
  2. Reference it from the agent YAML's mcp_servers: list.
  3. At agent run time, MCPConnectionManager (in codeward/mcp_client/manager.py) opens the connection (lazily, cached, with health checks) and merges the server's tools into the Agno agent's toolset alongside GitLabToolkit.

codeward.mcp_servers schema

The authoritative shape lives in codeward/mcp_client/registry.py (MCPServerConfig).

codeward:
  mcp_servers:
    context7:
      url: https://mcp.context7.com/mcp
      transport: streamable-http     # streamable-http | sse
      auth_type: bearer              # bearer | header | none
      auth_token_env: CONTEXT7_TOKEN # env var holding the bearer token
      enabled: true
      description: Context7 documentation lookups

    internal-kb:
      url: https://kb.internal/mcp
      transport: streamable-http
      auth_type: header
      headers:
        X-API-Key: ${INTERNAL_KB_KEY}  # ${VAR} expansion is supported
Field Type Notes
url string The MCP endpoint. ${ENV_VAR} is expanded from the process environment.
transport string streamable-http (default) or sse.
auth_type string bearer, header, or none.
auth_token_env string For bearer: the env var that holds the token.
headers mapping For header: the literal headers to send. ${ENV_VAR} is expanded.
enabled bool Disable a server without removing the entry. Defaults to true.
description string Free-form, surfaced in logs.

Per-project overrides

Projects can replace the MCP server list per agent through their .codeward.yml:

codeward:
  mcp_servers:
    chat:                           # agent name
      - context7                    # only context7 for the chat agent in this repo

If mcp_servers is set for an agent in a project's .codeward.yml, that list completely replaces the agent-level list (it is not merged). See MCPConnectionManager.get_tools_for_agent for the exact precedence.

Exposing Codeward as an MCP Server

Codeward also acts as an MCP server itself, mounted at /mcp by codeward/main.py and built with FastMCP in codeward/mcp_server/server.py. Any MCP-aware client (Claude Desktop, Cursor, an Agno agent in another project, etc.) can connect to it and use Codeward as a turn-key GitLab tool provider.

Highlights:

  • Transport: streamable HTTP at http://<host>:8420/mcp.
  • Auth: optional bearer token. Set CODEWARD_MCP_KEY in .env and the MCPAuthMiddleware will reject requests without the matching Authorization: Bearer … header. If unset, the mount is open.
  • Tools (codeward/mcp_server/tools.py): list_merge_requests, get_merge_request, get_merge_request_diff, post_merge_request_comment, list_issues, get_issue, create_issue, get_file_content, get_repository_tree, search_code, get_pipeline_status, get_failed_pipeline_logs, list_project_labels.
  • Resources (codeward/mcp_server/resources.py): gitlab://project/{path}/readme, gitlab://project/{path}/ci-config, gitlab://project/{path}/merge-requests/open, gitlab://project/{path}/issues/open, gitlab://project/{path}/pipelines/latest.
  • Prompts (codeward/mcp_server/prompts.py): review_merge_request, explain_pipeline_failure, summarize_issue.

See docs/mcp.md for client configuration examples and a worked walkthrough.

Multi-Agent Flows

A flow is a YAML-defined sequence of agent runs and GitLab API actions, executed by the Redis-backed worker (make worker). Flows are how Codeward implements the slash commands that touch more than one agent — e.g. running a code review, then a security triage, then posting a combined summary.

Built-in flows in agents/flows/:

Flow Trigger Steps
full-review @codeward /full-review on an MR code-reviewer → security-triage (skipped on failure) → combined summary comment
issue-to-mr @codeward /implement on an issue analyse issue → plan comment → branch → generate code → self-review → open MR → notify
pipeline-fix @codeward /fix-pipeline on an MR pipeline-debugger → diagnosis comment

Flows differ from individual agents in three ways:

  1. They are command-driven, not webhook-driven (each flow declares a slash command in its YAML trigger: block).
  2. They have state that survives across steps, persisted to Redis (codeward/flows/state.py). Subsequent steps can reference earlier outputs via {step_name.field} template placeholders.
  3. They run in the flow worker process (make worker), not inline in the webhook handler — so the GitLab webhook returns immediately while the flow continues asynchronously. Track progress via GET /flows/{flow_id}.

Full reference and YAML schema: docs/flows.md.

GitLab Toolkit (Agno Tools)

Agents have access to a GitLabToolkit with the following tools during LLM execution:

Tool Description Parameters
post_inline_comment Post a review comment on a specific diff line file, line, line_type (new/old), body, suggestion
post_note Post a general comment on the MR body
add_label Add a label to the MR label
remove_label Remove a label from the MR label
get_file_content Fetch a file from the repository file_path, ref

The toolkit automatically constructs GitLab position dicts for inline comments using the MR's diff_refs (base_sha, head_sha, start_sha). If positioning fails, it falls back to posting a general note.

Per-Project Configuration (.codeward.yml)

Projects can customize Codeward behavior by placing a .codeward.yml file in their repository root:

# Top-level or nested under "codeward:" key
codeward:
  # Control which agents run
  enabled_agents:           # If set, only these agents are allowed
    - code-reviewer
  disabled_agents:          # Agents to disable
    - security-scanner

  # Code review customization
  review_rules:             # Appended to the review agent's prompt
    - "Always use type hints for function signatures"
    - "Prefer dataclasses over plain dicts for structured data"
    - "All public functions must have docstrings"

  skip_paths:               # Glob patterns for files to exclude from review
    - "*.generated.py"
    - "vendor/**"
    - "migrations/**"

The config is cached for 5 minutes per project to avoid API spam.

Rate Limiting & Cooldowns

Rate limits are configured in codeward.yml and enforced via the agent_runs database table:

Limit Default Description
per_project_per_hour 30 Max agent runs per project per hour
per_user_per_hour 20 Max agent runs per user per hour
global_per_minute 10 Max agent runs globally per minute

Cooldown prevents the same agent from re-running on the same MR within a configurable window (default: 120 seconds). Only successful runs count toward cooldown.

Bot Loop Prevention

Codeward prevents infinite loops by checking is_bot_event() in the gateway:

  • Events authored by the bot username (CODEWARD_BOT_USERNAME) are skipped
  • MR updates where the last commit author matches the bot are skipped

Incremental Reviews

When a developer pushes new commits to an MR that was already reviewed:

  1. The orchestrator checks ReviewState for the MR (last reviewed SHA + discussion IDs)
  2. If the SHA is unchanged, the review is skipped entirely
  3. If the SHA changed, an inter-diff is fetched via GitLab's /repository/compare API
  4. Only the new changes are sent to the LLM for review
  5. Previous bot discussions on modified files are auto-resolved with a note: "This appears to have been addressed in the latest push. Resolving automatically."
  6. The summary note indicates it's an incremental review with the count of resolved threads
  7. If the inter-diff fetch fails, Codeward falls back to a full review

Thread resolution uses file-level matching: if the file referenced by a bot discussion was modified in the new push, the issue is assumed addressed.

Error Handling

If an agent fails during execution, the error is classified:

Status Trigger User Note
timeout Agent exceeds configurable timeout (default: 120s) "Timed out -- diff may be too large"
rate_limit LLM provider returns 429 or rate limit error "Rate limit hit -- will retry on next push"
error Any other exception Error type and message shown

The agent_runs table records the specific status (timeout, rate_limit, or error) with the error message. The orchestrator continues to the next matched agent.

Creating a Custom Agent

  1. Create a YAML file in the agents/ directory (or CODEWARD_AGENTS_DIR):

    agents/my-custom-agent.yml

  2. Define triggers, model, system prompt, and actions (see schema above).

  3. Restart Codeward - the registry loads all *.yml files at startup.

  4. The agent will automatically trigger on matching events. Use .codeward.yml in projects to enable/disable it per-repository.

Key Files

File Purpose
codeward/agents/base.py CodewardAgent base class - wraps Agno Agent, handles structured output
codeward/agents/code_reviewer.py CodeReviewAgent - MR diff review
codeward/agents/chat.py ChatAgent - conversational @codeward mentions
codeward/agents/pipeline_debugger.py PipelineDebuggerAgent - CI failure diagnosis
codeward/agents/security_triage.py SecurityTriageAgent - security scan triage
codeward/agents/models.py ReviewResult, PipelineDebugResult, SecurityTriageResult, AgentResult
codeward/agents/registry.py AgentRegistry - loads YAML definitions, TriggerSpec with job_names
codeward/agents/dispatch.py Agent type -> class mapping
codeward/agents/tools/gitlab_tools.py GitLabToolkit - Agno tools for GitLab
codeward/context/engine.py ContextEngine - builds AgentContext for each event type
codeward/context/log_parser.py CI log parsing - error classification, context extraction
codeward/context/security_report.py GitLab security report parsing (SAST, DAST, dependency)
codeward/orchestrator/orchestrator.py Orchestrator - event dispatch + result posting
codeward/orchestrator/matcher.py Event-to-agent matching (MR, Note, Pipeline, Job)
codeward/orchestrator/project_config.py Per-repo .codeward.yml loader
codeward/flows/runner.py FlowRunner — executes a flow's steps, applies templates and conditions
codeward/flows/loader.py Loads flow YAML definitions from agents/flows/
codeward/flows/state.py Redis-backed flow state store
codeward/worker/app.py Background worker process started by make worker
codeward/mcp_server/server.py FastMCP server + MCPAuthMiddleware, mounted at /mcp from main.py
codeward/mcp_server/{tools,resources,prompts}.py Registered tools / resources / prompts exposed to MCP clients
codeward/mcp_client/registry.py MCPRegistry and MCPServerConfig — parses codeward.mcp_servers
codeward/mcp_client/manager.py MCPConnectionManager — lazy connect, caching, per-agent tool merge