Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 13 additions & 18 deletions python/llamaindex-tool-policy/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,24 @@
# llamaindex-tool-policy

Demonstrates how to add [Agent Assembly](https://github.com/ai-agent-assembly/agent-assembly-examples) governance to [LlamaIndex](https://docs.llamaindex.ai/) tool calls when no native adapter is available.
Demonstrates [Agent Assembly](https://github.com/ai-agent-assembly/agent-assembly-examples) governance for [LlamaIndex](https://docs.llamaindex.ai/) tool calls using the **native LlamaIndex adapter**.

Because LlamaIndex does not yet have a native Agent Assembly adapter, this example shows the **manual wrapper pattern** — wrapping each `FunctionTool` with `GovernedToolRunner` so governance is enforced before every tool invocation. This pattern works for any Python callable.
LlamaIndex has a native Agent Assembly adapter (`agent_assembly.adapters.llamaindex`). It monkey-patches the concrete `FunctionTool.call` / `acall` execution methods, so once the adapter's hooks are registered, **every** LlamaIndex tool call is governed automatically — the exact method a `FunctionAgent` / `ReActAgent` invokes to run a tool. A denied tool's body never executes; the adapter returns a `ToolOutput` flagged `is_error=True` carrying a `[BLOCKED by governance policy]` message so the agent loop can react instead of crashing.

## What this example demonstrates

- Initializing Agent Assembly with `init_assembly()`.
- Applying governance to `FunctionTool` calls using `GovernedToolRunner`.
- Running an **allowed** tool call (`query_index`).
- Registering the native `LlamaIndexAdapter` against a local policy interceptor.
- Running an **allowed** tool call (`query_index`) through the patched `FunctionTool.call`.
- Running another **allowed** tool call (`summarize_docs`).
- Running a **denied** tool call (`execute_sql` — blocked by `deny_arbitrary_execution`).
- How to add governance to any framework that lacks a native adapter.
- Running a **denied** tool call (`execute_sql` — blocked by `deny_arbitrary_execution`); its body never runs.

## Prerequisites

| Requirement | Version |
|---|---|
| Python | >= 3.12 |
| [uv](https://github.com/astral-sh/uv) | latest |
| Agent Assembly Python SDK | >= 0.0.1b2 |
| Agent Assembly Python SDK | >= 0.0.1b2 (with the LlamaIndex adapter) |

No API key or running gateway is required for the offline demo.

Expand Down Expand Up @@ -52,8 +51,8 @@ Policy rules (local simulation of gateway policy):
DENY — execute_sql, run_shell_command (arbitrary execution)
ALLOW — everything else

Wrapping LlamaIndex tools with GovernedToolRunner...
Tools wrapped: query_index, summarize_docs, execute_sql
Registering the native LlamaIndex governance adapter...
FunctionTool.call / acall are now governed by Agent Assembly.

Running governed tool calls:
--------------------------------------------
Expand All @@ -64,7 +63,7 @@ Running governed tool calls:
✅ ALLOWED — 📝 Summary for 'policy enforcement': Agent Assembly provides governance...

→ execute_sql({'sql': 'DROP TABLE users; --'})
❌ BLOCKED — Tool 'execute_sql' is blocked by policy rule 'deny_arbitrary_execution'.
❌ BLOCKED — [BLOCKED by governance policy] Tool 'execute_sql' is blocked by policy rule 'deny_arbitrary_execution'. ...
```

## Run tests
Expand All @@ -77,21 +76,17 @@ uv run pytest tests/ -v

1. Start an Agent Assembly gateway or use your SaaS workspace URL.
2. Copy `.env.example` to `.env` and fill in credentials.
3. Replace `LocalPolicyEngine` with the gateway-backed `GatewayClient`:

```python
with init_assembly(gateway_url="http://localhost:8080", agent_id="my-agent") as ctx:
runner = GovernedToolRunner("query_index", query_fn, ctx.client)
result = runner.run(query="...")
```
3. Let `init_assembly()` auto-detect and register the LlamaIndex adapter against
the live gateway interceptor instead of the local `LocalPolicyEngine` — the
tool-call code does not change; only the policy source does.

## Troubleshooting

| Problem | Fix |
|---|---|
| `ModuleNotFoundError: agent_assembly` | Run `uv sync` first |
| `ModuleNotFoundError: llama_index` | Run `uv sync` — `llama-index-core` is a required dependency |
| `ToolExecutionBlockedError` in tests | Expected — the deny policy rule for `execute_sql` is intentional |
| `ImportError: cannot import name 'LlamaIndexAdapter'` | Your installed SDK predates the LlamaIndex adapter — upgrade `agent-assembly` |

## Links

Expand Down
12 changes: 10 additions & 2 deletions python/llamaindex-tool-policy/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
[project]
name = "llamaindex-tool-policy"
version = "0.1.0"
description = "Agent Assembly governance demo with LlamaIndex tool wrapping"
description = "Agent Assembly governance demo with the native LlamaIndex adapter"
requires-python = ">=3.12"
dependencies = [
"agent-assembly>=0.0.1b2",
# Floor bumped to the release that ships the native LlamaIndex adapter
# (agent_assembly.adapters.llamaindex, AAASM-3536). Earlier SDK releases
# lack it, so this example's CI is release-gated on that publish.
"agent-assembly>=0.0.1b5",
"llama-index-core>=0.14.22",
]

Expand All @@ -17,3 +20,8 @@ dev = [
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]

[tool.uv.sources]
# Track the SDK's git master so the example validates against the merged
# adapter until 0.0.1b5 publishes it to PyPI. Revert to the PyPI pin then.
agent-assembly = { git = "https://github.com/ai-agent-assembly/python-sdk.git", branch = "master" }
103 changes: 60 additions & 43 deletions python/llamaindex-tool-policy/src/main.py
Original file line number Diff line number Diff line change
@@ -1,45 +1,57 @@
"""
llamaindex-tool-policy: Agent Assembly governance demo with LlamaIndex.

LlamaIndex does not yet have a native Agent Assembly adapter, so this example
shows how to add governance by wrapping each FunctionTool with GovernedToolRunner.
This pattern works for ANY Python callable, not just LlamaIndex.

Run (offline mode, no gateway or API key required):
"""llamaindex-tool-policy: Agent Assembly governance demo with LlamaIndex.

LlamaIndex has a native Agent Assembly adapter
(``agent_assembly.adapters.llamaindex``): it monkey-patches the concrete
``FunctionTool.call`` / ``acall`` execution methods, so once the adapter's hooks
are registered, **every** LlamaIndex tool call is governed automatically — the
exact method a ``FunctionAgent`` / ``ReActAgent`` invokes to run a tool. A denied
tool's body never executes; the adapter returns a ``ToolOutput`` flagged
``is_error=True`` carrying a ``[BLOCKED by governance policy]`` message so the
agent loop can react instead of crashing.

This is an **offline** demo: ``LocalPolicyEngine`` simulates the gateway's
allow/deny verdict in-process, so no gateway or API key is required. The tools
and their ``FunctionTool.call`` invocations are real — only the policy decision
is local.

Run:
uv run python src/main.py
"""

from __future__ import annotations

import os
import sys
from typing import Any

sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))

from agent_assembly import init_assembly
from agent_assembly.exceptions import ToolExecutionBlockedError
from agent_assembly.adapters.llamaindex import LlamaIndexAdapter, LlamaIndexPatch

from src.policy import GovernedToolRunner, LocalPolicyEngine
from src.tools import _execute_sql_fn, _query_index_fn, _summarize_docs_fn
from src.policy import LocalPolicyEngine
from src.tools import execute_sql, query_index, summarize_docs

_DEMO_CALLS: list[tuple[str, Any, dict]] = [
("query_index", _query_index_fn, {"query": "what is Agent Assembly?"}),
("summarize_docs", _summarize_docs_fn, {"topic": "policy enforcement"}),
("execute_sql", _execute_sql_fn, {"sql": "DROP TABLE users; --"}),
#: The adapter's deny short-circuit marker. The LlamaIndex tool patch returns a
#: ToolOutput carrying this string on a deny rather than raising.
_BLOCKED_MARKER = "[BLOCKED by governance policy]"

_DEMO_CALLS = [
(query_index, {"query": "what is Agent Assembly?"}),
(summarize_docs, {"topic": "policy enforcement"}),
(execute_sql, {"sql": "DROP TABLE users; --"}),
]


def _run_governed_call(
runner: GovernedToolRunner,
label: str,
kwargs: dict,
) -> None:
print(f" → {label}({kwargs})")
try:
result = runner.run(**kwargs)
print(f" ✅ ALLOWED — {result}")
except ToolExecutionBlockedError as exc:
print(f" ❌ BLOCKED — {exc}")
def _run_governed_call(tool: object, kwargs: dict[str, str]) -> None:
name = tool.metadata.get_name() # type: ignore[attr-defined]
print(f" → {name}({kwargs})")
# The adapter has patched FunctionTool.call, so this single call is governed.
output = tool.call(**kwargs) # type: ignore[attr-defined]
content = str(getattr(output, "content", output))
if _BLOCKED_MARKER in content:
print(f" ❌ BLOCKED — {content}")
else:
print(f" ✅ ALLOWED — {content}")
print()


Expand All @@ -65,31 +77,36 @@ def main() -> None:
print(f" Mode: {ctx.network_mode} (offline demo)")
print()

policy = LocalPolicyEngine()

print("Policy rules (local simulation of gateway policy):")
print(" DENY — execute_sql, run_shell_command (arbitrary execution)")
print(" ALLOW — everything else")
print()

print("Wrapping LlamaIndex tools with GovernedToolRunner...")
runners = {
name: GovernedToolRunner(name, fn, policy)
for name, fn, _ in _DEMO_CALLS
}
print(" Tools wrapped: query_index, summarize_docs, execute_sql")
# Register the native LlamaIndex adapter against the local policy engine.
# This patches FunctionTool.call so every tool call below is governed
# automatically — no per-tool wrapper needed.
#
# init_assembly() in sdk-only mode already auto-detected LlamaIndex and
# patched FunctionTool.call against a no-op interceptor (there is no
# gateway offline). Revert that first so this example's LocalPolicyEngine
# is the live interceptor; in production init_assembly wires the adapter
# to the gateway and this manual step is unnecessary.
print("Registering the native LlamaIndex governance adapter...")
LlamaIndexPatch(callback_handler=None).revert()
adapter = LlamaIndexAdapter()
adapter.register_hooks(LocalPolicyEngine())
print(" FunctionTool.call / acall are now governed by Agent Assembly.")
print()

print("Running governed tool calls:")
print("-" * 44)
for tool_name, fn, kwargs in _DEMO_CALLS:
_run_governed_call(runners[tool_name], tool_name, kwargs)
try:
print("Running governed tool calls:")
print("-" * 44)
for tool, kwargs in _DEMO_CALLS:
_run_governed_call(tool, kwargs)
finally:
adapter.unregister_hooks()

print("Assembly context shut down.")
print()
print("Note: When a native LlamaIndex adapter is available,")
print(" GovernedToolRunner will no longer be needed — governance")
print(" will be applied automatically by init_assembly().")


if __name__ == "__main__":
Expand Down
91 changes: 32 additions & 59 deletions python/llamaindex-tool-policy/src/policy.py
Original file line number Diff line number Diff line change
@@ -1,77 +1,50 @@
"""Local offline policy engine for the llamaindex-tool-policy example.

LlamaIndex does not yet have a native Agent Assembly adapter, so governance
is applied by explicitly calling GovernedToolRunner.run() before each
LlamaIndex FunctionTool invocation.

This module defines:
- LocalPolicyEngine — simulates gateway policy (allow / deny)
- GovernedToolRunner — wraps any callable with AssemblyCallbackHandler
so every tool invocation passes through governance before execution
LlamaIndex now has a native Agent Assembly adapter
(``agent_assembly.adapters.llamaindex``). It monkey-patches the concrete
``FunctionTool.call`` / ``acall`` execution methods so every tool invocation
passes through governance automatically — no per-call wrapper is needed.

The adapter calls ``interceptor.check_tool_start(...)`` before a tool body runs
and blocks on a ``deny`` (returning a ``[BLOCKED by governance policy]``
``ToolOutput`` rather than raising, so an agent loop can react). This module
provides ``LocalPolicyEngine`` — the interceptor that simulates gateway policy
offline (allow / deny) — which is what ``main.py`` hands to the adapter.

The ``_enforce = True`` flag puts the interceptor in fail-closed posture: an
unknown / malformed verdict denies rather than silently allowing (AAASM-3107).
"""

from __future__ import annotations

from typing import Any
from uuid import uuid4

from agent_assembly.adapters.langchain import AssemblyCallbackHandler
from agent_assembly.exceptions import ToolExecutionBlockedError

DENIED_TOOLS: frozenset[str] = frozenset({
"execute_sql",
"run_shell_command",
})
DENIED_TOOLS: frozenset[str] = frozenset(
{
"execute_sql",
"run_shell_command",
}
)


class LocalPolicyEngine:
"""Simulates Agent Assembly gateway policy enforcement in offline mode."""
"""Simulates Agent Assembly gateway policy enforcement in offline mode.

Implements the ``check_tool_start`` interceptor contract the LlamaIndex
adapter calls before a governed ``FunctionTool`` runs. Returns an
``{"status": "allow"}`` / ``{"status": "deny", "reason": ...}`` verdict.
"""

#: Fail-closed posture: the adapter denies an unknown verdict under enforce.
_enforce = True

def check_tool_start(
self,
serialized: dict[str, Any],
**kwargs: Any,
) -> dict[str, str]:
tool_name = serialized.get("name", "")
def check_tool_start(self, **kwargs: Any) -> dict[str, str]:
tool_name = str(kwargs.get("tool_name") or "")
if tool_name in DENIED_TOOLS:
return {
"status": "deny",
"reason": (
f"Tool '{tool_name}' is blocked by policy rule "
"'deny_arbitrary_execution'."
f"Tool '{tool_name}' is blocked by policy rule 'deny_arbitrary_execution'."
),
}
return {"status": "allow"}


class GovernedToolRunner:
"""Wraps a callable with Agent Assembly governance.

Use this as a bridge when a framework does not have a native adapter.
Call ``run()`` instead of invoking the tool directly; governance runs first.

Example::

runner = GovernedToolRunner("query_index", query_index_fn, policy)
result = runner.run(query="What is Agent Assembly?")
"""

def __init__(
self,
tool_name: str,
fn: Any,
policy: LocalPolicyEngine,
) -> None:
self._tool_name = tool_name
self._fn = fn
self._handler = AssemblyCallbackHandler(interceptor=policy)

def run(self, **kwargs: Any) -> Any:
import json

input_str = json.dumps(kwargs)
self._handler.on_tool_start(
serialized={"name": self._tool_name, "type": "tool"},
input_str=input_str,
run_id=uuid4(),
)
return self._fn(**kwargs)
1 change: 1 addition & 0 deletions python/llamaindex-tool-policy/src/tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
- summarize_docs — safe summarisation tool (ALLOWED by policy)
- execute_sql — arbitrary SQL execution (DENIED by policy)
"""

from __future__ import annotations

from llama_index.core.tools import FunctionTool
Expand Down
Loading
Loading