AgentLens

AgentLens is a self-hosted observability and debugging platform for AI agents. I built it to make agent execution easier to understand: instead of seeing only the final response, you can inspect the runs, tool calls, errors, timing, model metadata, token usage, and intermediate events that produced it.

The current release provides first-class instrumentation for Python agents and serves the collector, API, and dashboard as one self-hosted application.

Why I built AgentLens

An AI agent rarely performs one operation. A single request can involve several model calls, tool invocations, intermediate decisions, and external requests. When the result is slow, wrong, or incomplete, the final answer does not tell you enough about what happened internally.

AgentLens records that execution so I can answer practical debugging questions:

• Which step failed?
• Which tool was called, and what happened next?
• How long did each operation take?
• Which model was used?
• How many tokens were consumed?
• What happened immediately before the failure?
• Did the run finish successfully?

The goal is straightforward: turn an opaque agent run into an execution history that another engineer can inspect.

How it works

Python AI Agent
      ↓
AgentLens SDK
      ↓
FastAPI Collector
      ↓
PostgreSQL / Supabase
      ↓
AgentLens Dashboard

1. The Python SDK instruments run lifecycles, decorated tools, steps, and custom events. It converts them into a structured telemetry envelope with stable run, trace, span, and event identifiers.
2. The FastAPI collector authenticates the project key, validates the payload, applies safety limits, and derives the project scope on the server.
3. The event and its run lifecycle are persisted under that project. Stable event IDs make delivery retries idempotent.
4. The dashboard retrieves only the connected project's data and reconstructs run summaries, steps, timing, tool activity, model metadata, and failures.
5. A developer can move from a project overview into a single run and inspect the sequence of events that led to its result.

Local development uses a bounded, process-local memory adapter, so the complete workflow runs without a database. Durable deployments use PostgreSQL through Supabase and the included SQL migration chain.

What I built

AgentLens combines instrumentation, ingestion, storage, security, and a developer-facing debugging interface in one repository.

• Instrumentation layer: a Python tracing API built around context managers, contextvars, and sync/async decorators. It records run lifecycle events, nested spans, custom events, tool inputs/results, timing, safe error details, and explicit LLM metadata.
• Telemetry transport: a synchronous HTTP client with bounded timeouts, retries, backoff, stable event IDs, recursive redaction, and fail-open behavior in the high-level runtime so an unavailable collector does not break the instrumented application.
• Collector API: a FastAPI service with Pydantic validation, authenticated ingestion, project-scoped authorization, request and event limits, bounded pagination, stable error responses, and server-side run aggregation.
• Project access model: high-entropy bearer keys stored as SHA-256 digests, one-time plaintext key display, rotation and revocation, and signed HttpOnly dashboard sessions that keep ingestion keys out of browser storage.
• Persistence layer: interchangeable bounded-memory and Supabase storage adapters, ordered SQL migrations, integrity constraints, query indexes, forced Row Level Security, and direct browser-role denial for sensitive tables.
• Observability dashboard: project metrics, searchable and filterable runs, run timelines, grouped steps, errors, tool and LLM activity, timing, token usage, model metadata, and paginated raw events.
• Engineering checks: unit, integration, security, migration, storage, and end-to-end tests, plus linting, type checking, package build validation, secret scanning, dependency auditing, and GitHub Actions CI.

Tech stack

Backend

• Python 3.10+
• FastAPI
• Pydantic
• Uvicorn

SDK

• Python
• Context managers and contextvars
• Sync and async decorators
• Requests-based HTTP telemetry transport

Database

• PostgreSQL
• Supabase
• Ordered SQL migrations
• Row Level Security

Frontend

• HTML
• CSS
• JavaScript
• FastAPI-served dashboard and static assets

Security

• Bearer project API keys
• SHA-256 key storage
• Signed HttpOnly project sessions
• Project-scoped authorization
• XSS-safe DOM rendering and Content Security Policy
• Payload, query, and rate limits
• Recursive telemetry redaction
• Gitleaks secret scanning
• pip-audit dependency checks

Testing and development tooling

• pytest
• Ruff
• mypy
• GitHub Actions
• PostgreSQL migration and RLS tests
• End-to-end smoke testing
• Python source distribution and wheel validation

Core features

• Trace complete agent runs with stable run, trace, span, and event IDs.
• Instrument synchronous and asynchronous tools with @agentlens.tool.
• Record custom steps, final output, errors, tool activity, and explicit LLM metadata.
• Inspect project metrics, recent runs, status, duration, errors, tokens, models, and raw telemetry in the dashboard.
• Search run IDs, filter lifecycle status, and page through runs and events.
• Create, rotate, and revoke project API keys.
• Run locally with no database or use PostgreSQL/Supabase for persistence.

Quick start

Requirements: Git and Python 3.10 or newer. The bundled example is deterministic and does not require a model-provider account or paid API.

git clone https://github.com/WaseemGhanem98/AgentLens.git
cd AgentLens
python -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install -e ".[dev]"

Start the collector and leave it running:

uvicorn agentlens_collector.app:app --host 127.0.0.1 --port 8001 --reload

With no Supabase configuration, development uses an in-memory store capped at 10,000 events, 10,000 runs, and 1,000 projects. In a second terminal, activate the environment and create a project:

. .venv/bin/activate
python scripts/create_project.py local-demo

The command prints the project API key once. Store it outside the repository, then configure and run the example:

export AGENTLENS_BASE_URL="http://127.0.0.1:8001"
export AGENTLENS_API_KEY="<the al_ key printed above>"
python examples/quickstart.py

Open http://127.0.0.1:8001/login, connect with the same project key, and inspect the run URL printed by the example. The login flow exchanges the key for a signed, project-bound HttpOnly session and does not retain the key in browser storage.

To record and inspect a controlled failure:

python examples/failing_run.py

To test project creation, SDK ingestion, persistence, authenticated dashboard access, logout, and anonymous-read rejection in one command:

python scripts/e2e_smoke.py

The memory backend is ephemeral. Restarting the collector removes its local projects, keys, runs, and events.

SDK example

The canonical high-level import is agentlens:

import agentlens

agentlens.init(
    base_url="http://127.0.0.1:8001",
    api_key="al_your_project_key",
    agent_name="research-agent",
)

@agentlens.tool("local_search")
def local_search(query: str) -> list[str]:
    return [f"result for {query}"]

with agentlens.trace_run(
    run_id="research-001",
    objective="Find the relevant release note",
):
    agentlens.event(
        "agent.step.start",
        {"step_id": "search", "step_name": "Search local data"},
    )
    results = local_search("AgentLens 0.1.0")
    agentlens.event("agent.final", {"final_output": results[0]})
    agentlens.event(
        "agent.step.end",
        {"step_id": "search", "step_name": "Search local data", "status": "success"},
    )

Telemetry sends are synchronous and fail open by default. Configuration errors still fail clearly, while collector or network failures do not replace the application's result or exception. Tests and examples can use strict=True when missing telemetry should fail the script. The SDK guide covers retries, redaction, explicit LLM events, and shutdown behavior.

Dashboard

The dashboard is designed around the path from a project overview to one specific execution:

• /dashboard shows project metrics, recent activity, errors, and runs.
• /dashboard/runs provides run-ID search, lifecycle status filters, and pagination.
• /dashboard/run/{run_id} shows the run timeline, grouped steps, tool and LLM activity, timing, errors, model/token metadata, insights, and raw events.
• /dashboard/projects handles local project creation and key management.
• /dashboard/settings shows the connected project and runtime configuration.

The collector also exposes authenticated JSON endpoints and a generated API reference at /api/docs. See the API reference for the full request, response, authentication, and error contracts.

PostgreSQL and Supabase

The collector uses the bounded memory adapter for local development and Supabase/PostgreSQL for durable storage. Apply the migrations in lexical order before starting a durable deployment:

for migration in migrations/*.sql; do
  psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f "$migration"
done

The browser never connects directly to Supabase. The collector keeps the service-role credential server-side and scopes every storage operation to the authenticated project. Read the database guide and deployment guide for configuration and access-control details.

Security

Agent telemetry can contain prompts, model output, tool inputs, identifiers, and errors. The SDK recursively redacts common credential-shaped keys and strings, but the safest approach is still to avoid collecting sensitive data that is not needed for debugging.

Project keys are bearer credentials. AgentLens stores only their SHA-256 digests, keeps production infrastructure secrets on the server, and isolates telemetry by project. PostgreSQL migrations force RLS on projects, runs, and events, while browser-facing roles have no direct table access. Production traffic should be served over HTTPS.

See SECURITY.md for vulnerability reporting and the privacy guide for collection, redaction, retention, and incident-handling guidance.

Development and testing

The repository currently has 138 tests covering the SDK, collector, sessions, authentication, validation, storage adapters, run metrics, migrations, project isolation, security regressions, and hostile telemetry rendering.

Run the main local checks from an activated development environment:

PYTHONDONTWRITEBYTECODE=1 python -m pytest -p no:cacheprovider
ruff check agentlens agentlens_sdk agentlens_collector scripts examples tests
mypy agentlens agentlens_sdk agentlens_collector scripts examples
python -m compileall -q agentlens agentlens_sdk agentlens_collector scripts examples tests
python scripts/e2e_smoke.py
python scripts/check_no_secrets.py
python -m pip_audit --local
python -m build
git diff --check

CI runs the test suite on Python 3.10 and 3.12. It also validates a fresh PostgreSQL migration chain and RLS boundary, frontend JavaScript and rendering sinks, installed wheel contents, Gitleaks scans of the current tree and Git history, and the resolved dependency graph. See CONTRIBUTING.md for the development workflow.

Project structure

agentlens/             High-level tracing runtime and decorators
agentlens_sdk/         Low-level authenticated HTTP client
agentlens_collector/   FastAPI app, storage adapters, security, and dashboard
migrations/            Ordered PostgreSQL/Supabase schema and hardening chain
examples/              Deterministic successful and failed agent runs
scripts/               Provisioning, end-to-end smoke, and secret checks
tests/                 Unit, integration, security, SDK, and migration tests
docs/                  Architecture, API, SDK, database, deployment, and privacy

Documentation

• Architecture
• API reference
• SDK guide
• Event contract
• Database and migrations
• Deployment
• Telemetry privacy
• Contributing
• Security policy

Current scope

AgentLens 0.1.0 focuses on the core observability loop: instrument an agent, collect structured telemetry, persist it under the correct project, and inspect the execution in a dashboard. It is currently designed as a self-hosted developer tool rather than a hosted SaaS platform.

The current scope does not include automatic model-provider monkey-patching, a hosted control plane, an email/password account system, or an enterprise organization and RBAC layer. Model and usage telemetry is emitted explicitly so the application controls what AgentLens receives.

License

AgentLens is available under the MIT License.