docs: add comprehensive API and SDK reference document #158#583
docs: add comprehensive API and SDK reference document #158#583NeuralImprint wants to merge 1 commit into
Conversation
|
@NeuralImprint is attempting to deploy a commit to the sreerevanth's projects Team on Vercel. A member of the Team first needs to authorize it. |
📝 WalkthroughWalkthroughA new documentation file, docs/API-reference.md, was added describing the AgentWatch Python SDK entry points, authentication mechanisms, and REST/WebSocket API endpoints covering monitoring, telemetry, safety/guardrails, and multi-tenant administration. ChangesAPI Reference Documentation
Estimated code review effort: 1 (Trivial) | ~5 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/API-reference.md`:
- Around line 11-13: The fenced Python example in the API reference is left
open, causing the following section to render as code. Update the markdown
snippet near the import example so the code fence is properly closed after the
agentwatch import, keeping the “Core SDK Methods” heading outside the block.
- Around line 125-130: Use brace placeholders for all path parameters in the API
reference section: update the session, tenant, and key path templates so the
documented endpoints use {session_id}, {tenant_id}, and {key_id} instead of
literal segment names. Apply the same placeholder format consistently to the
other endpoint templates in this section, especially the entries around the
sessions-related paths and any similar API docs nearby, so readers copy valid
URLs.
- Around line 217-220: The WebSocket auth guidance in the API reference still
advertises the `api_key` query-string fallback, which should be removed. Update
the authentication description for the WebSocket endpoint to keep `X-Api-Key`
header-based authentication only, and delete any mention of `api_key` in the
URL; if needed, note a short-lived token flow instead.
- Around line 84-97: The `/health` documentation example is missing fields that
the real health payload includes, so update the sample to match the response
shape produced by the health handler. In the API reference, adjust the example
under the Service Health Check section to include the additional `traces`,
`event_bus`, `safety`, and `cost` properties alongside the existing `status`,
`version`, `timestamp`, and `database_connected` fields. Keep the example
aligned with the actual handler output so clients can rely on the documented
response structure.
- Around line 208-211: Update the API-reference section for Manage Active Safety
Policies to explicitly document the required permissions for each endpoint: note
that GET /api/v1/safety/policy requires policy:read and PUT
/api/v1/safety/policy requires policy:write. Keep the wording aligned with the
existing endpoint descriptions, and make sure the permissions are clearly
attached to the corresponding API operations in docs/API-reference.md.
- Around line 7-33: The Python SDK reference only lists symbols and is missing
the actual contract for the public API. Update the documentation around
agentwatch.watch, agentwatch.detect_framework,
agentwatch.detect_framework_label, and agentwatch.GenericAdapter to include
their parameters, return values, and a short usage example for each, so the
section serves as the single source of truth for how to use the SDK.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4f83e66d-6edd-4d1f-a0f7-9191e8b5c382
📒 Files selected for processing (1)
docs/API-reference.md
| ## Python SDK Reference | ||
|
|
||
| To start monitoring your AI agents, make sure you import the core functions from the root package: | ||
|
|
||
| ```python | ||
| import agentwatch | ||
| Core SDK Methods | ||
| 1. agentwatch.watch | ||
| What it does: This is the primary entry point for AgentWatch. It dynamically attaches monitoring hooks to an active AI agent instance. Once wrapped, it automatically streams execution trace events, logs performance metrics, and applies active safety policies in real time. | ||
|
|
||
| Returns: The monitored, safety wrapped agent instance. | ||
|
|
||
| 2. agentwatch.detect_framework | ||
| What it does: Inspects a live runtime object to identify which orchestration library is being used behind the scenes. | ||
|
|
||
| Supported Frameworks: Dynamically identifies popular frameworks like LangChain, CrewAI, and AutoGen. | ||
|
|
||
| Returns: A reference to the identified framework adapter class. | ||
|
|
||
| 3. agentwatch.detect_framework_label | ||
| What it does: A helper method that identifies the running agent framework and returns a clean, human readable string such as langchain, crewai, or autogen. | ||
|
|
||
| Returns: A string label. | ||
|
|
||
| 4. agentwatch.GenericAdapter | ||
| What it does: The base adapter class used for parsing agent lifecycles. If you are running an in house or unsupported agent framework, you can subclass GenericAdapter to build custom telemetry parsing logic. | ||
|
|
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Document the SDK contract, not just the symbols.
This section names the public entrypoints, but it does not document parameters, return values, or usage examples, so it misses the stated “single source of truth” goal.
Suggested structure
-1. agentwatch.watch
-What it does: ...
+### `agentwatch.watch(...)`
+Parameters:
+ - ...
+Returns:
+ - ...
+Example:
+ ```python
+ ...
+ ```🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/API-reference.md` around lines 7 - 33, The Python SDK reference only
lists symbols and is missing the actual contract for the public API. Update the
documentation around agentwatch.watch, agentwatch.detect_framework,
agentwatch.detect_framework_label, and agentwatch.GenericAdapter to include
their parameters, return values, and a short usage example for each, so the
section serves as the single source of truth for how to use the SDK.
| ```python | ||
| import agentwatch | ||
| Core SDK Methods |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Close the fenced code example.
The opening ```python block never closes, so the rest of the reference renders as code and becomes unreadable.
Fix
```python
import agentwatch
+```
Core SDK Methods🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/API-reference.md` around lines 11 - 13, The fenced Python example in the
API reference is left open, causing the following section to render as code.
Update the markdown snippet near the import example so the code fence is
properly closed after the agentwatch import, keeping the “Core SDK Methods”
heading outside the block.
| Service Health Check | ||
| Endpoint: GET /health | ||
|
|
||
| Description: A quick, publicly accessible health check used by load balancers and uptime monitors to ensure the server, database connections, and telemetry loops are operating smoothly. | ||
|
|
||
| Example Response: | ||
|
|
||
| JSON | ||
| { | ||
| "status": "ok", | ||
| "version": "0.2.0", | ||
| "timestamp": "2026-07-09T11:15:00Z", | ||
| "database_connected": true | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Match the /health example to the real payload.
The handler returns traces, event_bus, safety, and cost too, so the sample currently misleads clients about the response shape.
Fix
{
"status": "ok",
"version": "0.2.0",
"timestamp": "2026-07-09T11:15:00Z",
- "database_connected": true
+ "database_connected": true,
+ "traces": {...},
+ "event_bus": {...},
+ "safety": {...},
+ "cost": {...}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Service Health Check | |
| Endpoint: GET /health | |
| Description: A quick, publicly accessible health check used by load balancers and uptime monitors to ensure the server, database connections, and telemetry loops are operating smoothly. | |
| Example Response: | |
| JSON | |
| { | |
| "status": "ok", | |
| "version": "0.2.0", | |
| "timestamp": "2026-07-09T11:15:00Z", | |
| "database_connected": true | |
| } | |
| Service Health Check | |
| Endpoint: GET /health | |
| Description: A quick, publicly accessible health check used by load balancers and uptime monitors to ensure the server, database connections, and telemetry loops are operating smoothly. | |
| Example Response: | |
| JSON | |
| { | |
| "status": "ok", | |
| "version": "0.2.0", | |
| "timestamp": "2026-07-09T11:15:00Z", | |
| "database_connected": true, | |
| "traces": {...}, | |
| "event_bus": {...}, | |
| "safety": {...}, | |
| "cost": {...} | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/API-reference.md` around lines 84 - 97, The `/health` documentation
example is missing fields that the real health payload includes, so update the
sample to match the response shape produced by the health handler. In the API
reference, adjust the example under the Service Health Check section to include
the additional `traces`, `event_bus`, `safety`, and `cost` properties alongside
the existing `status`, `version`, `timestamp`, and `database_connected` fields.
Keep the example aligned with the actual handler output so clients can rely on
the documented response structure.
| Endpoint: GET /api/v1/sessions/session_id | ||
|
|
||
| Description: Grabs complete high level summary logs and metadata for a single target session. | ||
|
|
||
| Retrieve Trace Events | ||
| Endpoint: GET /api/v1/sessions/session_id/events |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Use brace placeholders for path params.
session_id, tenant_id, and key_id are written as literal path segments here; readers will copy invalid URLs unless these are documented as {session_id}, {tenant_id}, and {key_id}.
Fix
-Endpoint: GET /api/v1/sessions/session_id
+Endpoint: GET /api/v1/sessions/{session_id}Apply the same pattern to the other path templates in this section.
Also applies to: 163-199, 224-234
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/API-reference.md` around lines 125 - 130, Use brace placeholders for all
path parameters in the API reference section: update the session, tenant, and
key path templates so the documented endpoints use {session_id}, {tenant_id},
and {key_id} instead of literal segment names. Apply the same placeholder format
consistently to the other endpoint templates in this section, especially the
entries around the sessions-related paths and any similar API docs nearby, so
readers copy valid URLs.
| Manage Active Safety Policies | ||
| GET /api/v1/safety/policy — Returns currently active compliance rules. | ||
|
|
||
| PUT /api/v1/safety/policy — Modifies parameters and updates global enforcement behaviors. |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Document the policy permissions on these endpoints.
The server enforces policy:read for GET /api/v1/safety/policy and policy:write for PUT /api/v1/safety/policy, so the current wording understates the access-control contract. citeturn0search0turn0search1
Fix
-GET /api/v1/safety/policy — Returns currently active compliance rules.
-PUT /api/v1/safety/policy — Modifies parameters and updates global enforcement behaviors.
+GET /api/v1/safety/policy — Requires `policy:read`; returns the active policy.
+PUT /api/v1/safety/policy — Requires `policy:write`; updates the active policy.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Manage Active Safety Policies | |
| GET /api/v1/safety/policy — Returns currently active compliance rules. | |
| PUT /api/v1/safety/policy — Modifies parameters and updates global enforcement behaviors. | |
| Manage Active Safety Policies | |
| GET /api/v1/safety/policy — Requires `policy:read`; returns the active policy. | |
| PUT /api/v1/safety/policy — Requires `policy:write`; updates the active policy. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/API-reference.md` around lines 208 - 211, Update the API-reference
section for Manage Active Safety Policies to explicitly document the required
permissions for each endpoint: note that GET /api/v1/safety/policy requires
policy:read and PUT /api/v1/safety/policy requires policy:write. Keep the
wording aligned with the existing endpoint descriptions, and make sure the
permissions are clearly attached to the corresponding API operations in
docs/API-reference.md.
| Description: Establishes a persistent, low latency WebSocket connection used to push raw, real time agent framework activities directly to attached browser visualization dashboards. | ||
|
|
||
| How to Authenticate: Pass your API key via the standard X-Api-Key header during the initial handshake, or add it directly as a query element named api_key. | ||
|
|
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Remove the query-string API key fallback.
Putting api_key in the URL leaks credentials into logs, browser history, and proxies; keep WebSocket auth header-based or move to a short-lived token flow.
Fix
-How to Authenticate: Pass your API key via the standard X-Api-Key header during the initial handshake, or add it directly as a query element named api_key.
+How to Authenticate: Pass a short-lived token or API key via the `X-Api-Key` header during the initial handshake.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Description: Establishes a persistent, low latency WebSocket connection used to push raw, real time agent framework activities directly to attached browser visualization dashboards. | |
| How to Authenticate: Pass your API key via the standard X-Api-Key header during the initial handshake, or add it directly as a query element named api_key. | |
| Description: Establishes a persistent, low latency WebSocket connection used to push raw, real time agent framework activities directly to attached browser visualization dashboards. | |
| How to Authenticate: Pass a short-lived token or API key via the `X-Api-Key` header during the initial handshake. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/API-reference.md` around lines 217 - 220, The WebSocket auth guidance in
the API reference still advertises the `api_key` query-string fallback, which
should be removed. Update the authentication description for the WebSocket
endpoint to keep `X-Api-Key` header-based authentication only, and delete any
mention of `api_key` in the URL; if needed, note a short-lived token flow
instead.
sreerevanth
left a comment
sreerevanth
left a comment
There was a problem hiding this comment.
✅ Require a persistent AGENTWATCH_ERASURE_SECRET instead of generating a random or all-zero signing key.
✅ Fix the tenant/user scope mapping so tenant_id doesn't incorrectly route through agent_id.
✅ Surface erasure failures instead of always returning HTTP 200 with a signed receipt.
✅ Fix the reasoning fingerprint reason logic (distance == 0 ≠ no planner signal).
✅ Update the HMAC docstring (or include timestamps in the signature).
|
@NeuralImprint please check the pr |
🧪 PR Test Results
Python 3.12 · commit ae5c126 |
2 participants
Public API and SDK Reference Documentation
This pull request adds a comprehensive, single-source-of-truth reference document for the entire AgentWatch platform ecosystem. It details all backend routing structures, code-level SDK interfaces, security settings, and licensing tier parameters.
Documentation Highlights
Python SDK Interface: Documents core methods like watch, detect_framework, and detect_framework_label, alongside system runtime exceptions like AgentWatchBlockedError.
REST API Endpoints: Complete overview of system status checks, health tracking endpoints, session data logging routes, evaluation metrics pipelines, and multi-tenant administrative boundaries.
Security & Authentication Layers: Outlines how the system handles standard API Keys, SAML single sign-on tokens, and operational access privileges across user tiers.
Premium Feature Entitlements: Explains how client requests are checked against license files and entitlement verification keys to guarantee premium access compliance.
Why this is needed
Previously, the system's public facing functions and communication endpoints were spread across multiple code modules like server.py, auth.py, and entitlement.py. This new resource unifies all that knowledge into a single location, making it much easier for developers to build on top of AgentWatch securely.
Closes #158
Summary by CodeRabbit