AgentOps Workbench

AgentOps Workbench is an experimental local toolkit for turning structured project notes into deterministic, reviewable AI-agent workspace scaffolds.

It combines workspace generation with evaluation, approval, version-record, and architecture-health primitives. The current implementation is a local MVP, not a production agent platform.

Status: experimental preview. Deterministic by default. No built-in LLM/RAG calls, autonomous rule promotion, or production deployment automation.

What works today

• Compile profile, project, error, and knowledge-gap notes into structured Markdown memory.
• Generate a deterministic 15-file workspace scaffold containing:
  • AGENTS.md and metadata;
  • planner, executor, and reviewer role files;
  • pre-task and pre-merge checklists;
  • planning and review prompt templates;
  • candidate error-derived rules; and
  • principles, projects, errors, and knowledge-gap memory.
• Build an artifact manifest and deterministic ZIP bytes.
• Run deterministic evaluation helpers.
• Produce in-memory approval proposals and version records.
• Produce architecture-health records and Markdown.
• Expose preview operations through FastAPI.
• Review evidence through a static browser-openable frontend.

Quick start

Requirements

• Git
• Python 3.12 (the currently verified version)

Clone the repository and create a virtual environment:

git clone https://github.com/zhangzheng-debug/agentops-workbench.git
cd agentops-workbench

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r backend\requirements.txt

On macOS or Linux, activate the environment with:

source .venv/bin/activate

Validate the governed project pack:

python scripts\validate_pack.py

Preview a generated workspace without writing files:

python ai-worker\harness_generator\preview_workspace.py

The current preview reports 15 generated files.

Run the deterministic end-to-end demo in preview mode:

python scripts\d1_local_demo.py

Expected headline output:

status=ok
target_path=docs/source/generated/demo.md

The command does not write the generated document unless an explicit write option is supplied.

Run the API preview

python -m uvicorn backend.app.main:app --reload

Then open:

• API documentation: http://127.0.0.1:8000/docs
• Health check: http://127.0.0.1:8000/health

The static review UI can be opened directly from frontend/index.html. It is not a production UI and does not provide complete live API integration.

Verify

The repository uses Python's standard unittest runner:

python -m unittest discover -s ai-worker\tests -p "test_*.py"
python -m unittest discover -s frontend\tests -p "test_*.py"
python scripts\run_backend_ci.py
python scripts\validate_pack.py

The backend suite is large and can take several minutes. The CI runner executes the complete suite while quarantining an explicit set of legacy handoff/state assertions. Each entry must reproduce its recorded exception type, assertion site, and failure-reason fragment. New failures, changed failure reasons, missing entries, and unexpected successes fail the command. The exact list is visible in scripts/run_backend_ci.py.

Safety model

Generated rules, prompts, and other governance artifacts are candidates only. Publication requires evaluation, explicit human approval, a version record, and a rollback path.

The repository deliberately separates:

candidate generation
  -> deterministic checks
  -> human review and approval
  -> versioned publication
  -> rollback-ready evidence

Current limitations

• Experimental local MVP; not production ready.
• Default API runtime state is in memory.
• SQLite repository adapters exist, but they are not a production durability claim.
• No authentication, authorization, or production hardening.
• Static frontend with incomplete live API wiring.
• No built-in LLM or RAG execution.
• No autonomous self-modification or automatic rule promotion.
• No external MCP integrations or deployment automation.
• Product-generated Skills and Hooks remain roadmap work.

Repository map

• ai-worker/: deterministic compilation, generation, evaluation, and export.
• backend/: FastAPI preview routes and governance/repository primitives.
• frontend/: static reviewer-facing preview and Python-based source checks.
• scripts/: local demos, smoke runners, and validation helpers.
• generated_workspace_template/: template material for generated workspaces.
• docs/: architecture, API, demo, and maintainer documentation.

The many D*, I*, and POST_I6* records preserve narrow implementation and review evidence. They are maintainer history, not separate product features.

Current Local App Demo Entry

The public quick start above is the recommended entrypoint. The following compact index preserves exact maintainer evidence required by the repository's deterministic documentation checks.

Maintainer evidence index

The current product-facing reviewer package acceptance layer is the D47 local reviewer export package product-facing reviewer package acceptance helper. Its implementation is scripts/d47_local_reviewer_export_package_product_facing_reviewer_package_acceptance.py. The preceding D46 product-facing reviewer experience documentation sync remains supporting evidence.

The current product-facing reviewer acceptance walkthrough is the D41 local reviewer export package product-facing reviewer acceptance walkthrough.

D41 reviewer acceptance walkthrough source of truth:

• scripts/d41_local_reviewer_export_package_product_facing_reviewer_acceptance_walkthrough.py
• backend/tests/test_d41_local_reviewer_export_package_product_facing_reviewer_acceptance_walkthrough.py
• D41_LOCAL_REVIEWER_EXPORT_PACKAGE_PRODUCT_FACING_REVIEWER_ACCEPTANCE_WALKTHROUGH_COMPLETION_REVIEW.md

Its compact evidence includes:

version=0.2-no-write-evaluator
walkthrough_ready=True
reviewer_commands_executed=False
backend_server_started=False
browser_opened=False
approval_claimed=False
production_readiness_claimed=False

The current product-facing reviewer entrypoint is the D39 local reviewer export package product-facing entrypoint:

• scripts/d39_local_reviewer_export_package_product_facing_entrypoint.py
• backend/tests/test_d39_local_reviewer_export_package_product_facing_entrypoint.py
• D39_LOCAL_REVIEWER_EXPORT_PACKAGE_PRODUCT_FACING_SLICE_COMPLETION_REVIEW.md

The D43 review surface consolidation and D44 review surface documentation consolidation retain:

• scripts/d43_local_reviewer_export_package_product_facing_review_surface_consolidation.py
• scripts/d41_local_reviewer_export_package_product_facing_reviewer_acceptance_walkthrough.py
• scripts/d39_local_reviewer_export_package_product_facing_entrypoint.py

These helpers are deterministic, in-memory, and no-write. They are not a filesystem writer, not a source reader, and not an archive writer. They are not approval, not merge permission, not PR permission, and not production readiness. This helper does not execute reviewer commands, does not start backend, does not start frontend, and does not open browsers. It does not grant source-read permission or write permission. No approval bypass is allowed.

files_written=False
archives_created=False
source_files_read=False

Protected manual seed and mojibake rejection examples remain documented in docs/demo-scripts.md. This reviewer path does not write docs/source, and the repository retains the explicit boundary: No docs/source writes.

Project status and roadmap

The implemented MVP centers on deterministic memory compilation, workspace generation, export, evaluation, approval primitives, and architecture-health reporting. See:

• Project brief
• Architecture map
• MVP roadmap
• Post-MVP integration plan

Future work includes clearer package boundaries, reproducible dependency management, complete API/frontend integration, stronger authentication and authorization, and carefully gated optional model integrations.

Contributing and security

• Read CONTRIBUTING.md before proposing a change.
• Report vulnerabilities through the process in SECURITY.md.
• Community participation follows CODE_OF_CONDUCT.md.
• Repository-specific engineering constraints are defined in AGENTS.md.

License

Licensed under the Apache License 2.0.