Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/verify-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,3 +184,22 @@ jobs:

- name: Run smoke tests
run: uv run pytest tests/ -v

haystack-tool-policy:
name: haystack-tool-policy
runs-on: ubuntu-latest
defaults:
run:
working-directory: python/haystack-tool-policy
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0

- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
version: "latest"

- name: Install dependencies
run: uv sync --extra dev

- name: Run smoke tests
run: uv run pytest tests/ -v
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ below is the shared cross-language view.
| [`python/langgraph/`](./python/langgraph/) | LangGraph | Node-level governance on a compiled `StateGraph`, blocking a destructive tool mid-graph |
| [`python/pydantic-ai/`](./python/pydantic-ai/) | Pydantic AI | Tool-call governance driven offline by `TestModel` (allow / deny / pending) |
| [`python/google-adk/`](./python/google-adk/) | Google ADK | Scripted offline tool trajectory governing `BaseTool.run_async` (no cloud creds) |
| [`python/haystack-tool-policy/`](./python/haystack-tool-policy/) | Haystack | Govern a real Haystack agent via the native adapter — real `Tool.invoke` allow/deny through a `ToolInvoker` |

### Node.js / TypeScript

Expand Down
1 change: 1 addition & 0 deletions python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This directory contains runnable Python examples showing how to integrate Agent
| `langgraph/` | LangGraph | Node-level governance on a compiled `StateGraph`, blocking a destructive tool mid-graph |
| `pydantic-ai/` | Pydantic AI | Tool-call governance driven offline by `TestModel` (allow / deny / pending) |
| `google-adk/` | Google ADK | Scripted offline tool trajectory governing `BaseTool.run_async` (no cloud creds) |
| `haystack-tool-policy/` | Haystack | Govern a real Haystack agent via the native adapter — real `Tool.invoke` allow/deny through a `ToolInvoker` |

All examples use the `agent-assembly` Python package (available on PyPI).

Expand Down
8 changes: 8 additions & 0 deletions python/haystack-tool-policy/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Agent Assembly gateway connection (required for production mode)
# Leave unset to run in offline / sdk-only demo mode
# AGENT_ASSEMBLY_GATEWAY_URL=http://localhost:8080
# AGENT_ASSEMBLY_API_KEY=your-api-key-here

# Haystack LLM provider (optional — this example drives tools through a real
# ToolInvoker with a hand-built ToolCall, so no model/API key is needed).
# OPENAI_API_KEY=your-openai-key-here
106 changes: 106 additions & 0 deletions python/haystack-tool-policy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# haystack-tool-policy

Demonstrates [Agent Assembly](https://github.com/ai-agent-assembly/agent-assembly-examples) governance over a real [Haystack](https://haystack.deepset.ai/) agent using the SDK's **native Haystack adapter**.

Haystack has a native adapter: `HaystackPatch` hooks `haystack.tools.Tool.invoke` — the single execution chokepoint Haystack 2.x uses for every tool, including the agentic `Agent` → `ToolInvoker` tool-call loop. This example governs three real `haystack.tools.Tool` instances by driving them through a genuine `ToolInvoker` (the component a Haystack `Agent` uses to execute a model-chosen tool call), so a denied tool is short-circuited **before its body runs** — real governance, not a no-op.

## What this example demonstrates

- Initializing Agent Assembly with `init_assembly()` (which auto-detects Haystack).
- Installing the native Haystack adapter (`HaystackPatch`) against a local policy.
- Running real `haystack.tools.Tool` calls through a real `ToolInvoker`.
- An **allowed** tool call (`query_index`) — the tool body executes.
- Another **allowed** tool call (`summarize_docs`).
- A **denied** tool call (`execute_sql` — blocked by `deny_arbitrary_execution`); its underlying function never runs.

## Prerequisites

| Requirement | Version |
|---|---|
| Python | >= 3.12 |
| [uv](https://github.com/astral-sh/uv) | latest |
| Agent Assembly Python SDK | >= 0.0.1b2 |
| Haystack | >= 2.0.0, < 3.0 |

No API key or running gateway is required for the offline demo — the tools are driven through a `ToolInvoker` with a hand-built `ToolCall`, so no LLM is involved.

## Setup

```bash
cd python/haystack-tool-policy
uv sync --extra dev
```

## Run

```bash
uv run python src/main.py
```

### Expected output

```
==============================================================
Agent Assembly — Haystack Tool Policy Demo
==============================================================

Initializing Agent Assembly (gateway: http://localhost:8080, sdk-only mode)...
Agent: haystack-demo-agent
Gateway: http://localhost:8080
Mode: sdk-only (offline demo)

Policy rules (local simulation of gateway policy):
DENY — execute_sql, run_shell_command (arbitrary execution)
ALLOW — everything else

Installing the native Haystack adapter against the demo policy...
Adapter installed: True

Running real Haystack tools through a ToolInvoker:
--------------------------------------------
→ query_index({'query': 'what is Agent Assembly?'})
✅ ALLOWED — Index results for 'what is Agent Assembly?': [chunk-12, chunk-44, chunk-07] (mock)

→ summarize_docs({'topic': 'policy enforcement'})
✅ ALLOWED — Summary for 'policy enforcement': Agent Assembly provides governance... (mock)

→ execute_sql({'sql': 'DROP TABLE users; --'})
❌ BLOCKED — [BLOCKED by governance policy] Tool 'execute_sql' is blocked by policy rule 'deny_arbitrary_execution'...

Tool bodies that actually executed: ['query_index', 'summarize_docs']
```

`execute_sql` is absent from the executed list — the deny short-circuited it before the tool ran.

## Run tests

```bash
uv run pytest tests/ -v
```

## How the offline demo wires the policy

`init_assembly()` auto-detects Haystack and patches `Tool.invoke` for you. In offline `sdk-only` mode it wires a no-op interceptor (there is no live gateway to answer policy), so this demo reverts that and re-installs the same native adapter against a `LocalPolicyEngine` to make a real allow/deny visible without a gateway.

In production you point `init_assembly()` at a gateway and let its auto-detected adapter enforce real policy — no manual re-install needed:

```python
with init_assembly(gateway_url="https://your-workspace", api_key="...", agent_id="my-agent") as ctx:
# Haystack tools are governed automatically from here.
...
```

## Troubleshooting

| Problem | Fix |
|---|---|
| `ModuleNotFoundError: agent_assembly` | Run `uv sync` first |
| `ModuleNotFoundError: haystack` | Run `uv sync` — `haystack-ai` is a required dependency |
| `execute_sql` shows ALLOWED | Make sure the demo policy is installed *after* `init_assembly()` (it reverts the auto-applied no-op patch); see `src/main.py` |

## Links

- [Agent Assembly Python SDK](https://github.com/ai-agent-assembly/python-sdk)
- [Haystack docs](https://haystack.deepset.ai/)
- [Agent Assembly Examples](../../README.md)
- [Python Examples](../README.md)
27 changes: 27 additions & 0 deletions python/haystack-tool-policy/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[project]
name = "haystack-tool-policy"
version = "0.1.0"
description = "Agent Assembly governance demo: a real Haystack agent governed via the native adapter"
requires-python = ">=3.12"
dependencies = [
"agent-assembly>=0.0.1b2",
# Haystack 2.x ships the `Tool` / `ToolInvoker` API the SDK's native adapter
# hooks (haystack.tools.Tool.invoke). 1.x predates it and is out of scope.
"haystack-ai>=2.0.0,<3.0",
]

[project.optional-dependencies]
dev = [
"pytest>=8.0.0",
"pytest-mock>=3.14.0",
]

[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]

[tool.uv.sources]
# Track the SDK's git master so the example validates against the latest merged
# adapter before 0.0.1b5 publishes it to PyPI. Switch back to the PyPI release
# pin once 0.0.1b5 is out.
agent-assembly = { git = "https://github.com/ai-agent-assembly/python-sdk.git", branch = "master" }
Empty file.
135 changes: 135 additions & 0 deletions python/haystack-tool-policy/src/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
"""
haystack-tool-policy: an Agent Assembly governance demo with a real Haystack agent.

Haystack has a **native Agent Assembly adapter** (``HaystackPatch``) that hooks
``haystack.tools.Tool.invoke`` — the single execution chokepoint Haystack 2.x uses
for every tool, including the agentic ``Agent`` -> ``ToolInvoker`` tool-call loop.
This example governs three real ``haystack.tools.Tool`` instances by driving them
through a genuine ``ToolInvoker`` (the component a Haystack ``Agent`` uses to run a
model-chosen tool call) with the native adapter installed against a local policy:

- ALLOW query_index, summarize_docs (safe, read-only)
- DENY execute_sql (arbitrary execution)

A denied tool's body never runs — the adapter short-circuits ``Tool.invoke`` and
returns a ``[BLOCKED by governance policy]`` message the agent can react to.

Run (offline mode, no gateway or API key required):
uv run python src/main.py
"""

from __future__ import annotations

import os
import sys

sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))

from agent_assembly import init_assembly
from agent_assembly.adapters.haystack import HaystackPatch
from haystack.tools import Tool

from src.policy import LocalPolicyEngine
from src.tools import EXECUTED, build_tools

#: The tool calls the demo drives, as (tool_name, arguments). ``execute_sql`` is
#: the one the policy denies.
_DEMO_CALLS: list[tuple[str, dict[str, str]]] = [
("query_index", {"query": "what is Agent Assembly?"}),
("summarize_docs", {"topic": "policy enforcement"}),
("execute_sql", {"sql": "DROP TABLE users; --"}),
]

_BLOCKED_MARKER = "[BLOCKED by governance policy]"


def _invoke_through_tool_invoker(
tools: list[Tool], tool_name: str, arguments: dict[str, str]
) -> str:
"""Run *tool_name* via a real Haystack ``ToolInvoker`` and return its result.

Feeds the invoker a hand-built ``ToolCall`` (the shape a chat model would emit)
so the governed ``Tool.invoke`` is exercised on the genuine agent tool-dispatch
path, not in isolation — no LLM needed.
"""
from haystack.components.tools import ToolInvoker
from haystack.dataclasses import ChatMessage, ToolCall

invoker = ToolInvoker(tools=tools)
invoker.warm_up()
message = ChatMessage.from_assistant(
tool_calls=[ToolCall(tool_name=tool_name, arguments=arguments)]
)
output = invoker.run(messages=[message])
return str(output["tool_messages"][0].tool_call_results[0].result)


def _print_call(tool_name: str, arguments: dict[str, str], result: str) -> None:
print(f" → {tool_name}({arguments})")
if _BLOCKED_MARKER in result:
print(f" ❌ BLOCKED — {result}")
else:
print(f" ✅ ALLOWED — {result}")
print()


def main() -> None:
print("=" * 62)
print(" Agent Assembly — Haystack Tool Policy Demo")
print("=" * 62)
print()

gateway_url = os.environ.get("AGENT_ASSEMBLY_GATEWAY_URL", "http://localhost:8080")
api_key = os.environ.get("AGENT_ASSEMBLY_API_KEY")

print(f"Initializing Agent Assembly (gateway: {gateway_url}, sdk-only mode)...")

with init_assembly(
gateway_url=gateway_url,
api_key=api_key,
agent_id="haystack-demo-agent",
mode="sdk-only",
) as ctx:
print(f" Agent: {ctx.client.agent_id}")
print(f" Gateway: {ctx.client.gateway_url}")
print(f" Mode: {ctx.network_mode} (offline demo)")
print()

print("Policy rules (local simulation of gateway policy):")
print(" DENY — execute_sql, run_shell_command (arbitrary execution)")
print(" ALLOW — everything else")
print()

# init_assembly() has already auto-detected Haystack and patched
# Tool.invoke — but in offline sdk-only mode it wires a no-op interceptor
# (there is no live gateway/runtime to answer policy). For this *offline*
# demo we revert that and re-install the same native adapter against a
# LocalPolicyEngine so a real allow/deny is visible without a gateway. In
# production you would instead point init_assembly() at a gateway and let
# its auto-detected adapter enforce real policy — no manual re-install.
print("Installing the native Haystack adapter against the demo policy...")
HaystackPatch(LocalPolicyEngine()).revert() # drop the auto-applied no-op patch
patch = HaystackPatch(LocalPolicyEngine())
installed = patch.apply()
print(f" Adapter installed: {installed}")
print()

try:
tools = build_tools()
print("Running real Haystack tools through a ToolInvoker:")
print("-" * 44)
for tool_name, arguments in _DEMO_CALLS:
result = _invoke_through_tool_invoker(tools, tool_name, arguments)
_print_call(tool_name, arguments, result)
finally:
patch.revert()

print("Assembly context shut down.")
print()
print(f"Tool bodies that actually executed: {EXECUTED}")
print(" (note: 'execute_sql' is absent — the deny short-circuited it before")
print(" the tool ran, proving real governance rather than a no-op.)")


if __name__ == "__main__":
main()
49 changes: 49 additions & 0 deletions python/haystack-tool-policy/src/policy.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
"""Local offline policy engine for the haystack-tool-policy example.

Unlike a framework that needs a hand-written tool wrapper, Haystack has a
**native Agent Assembly adapter**: ``HaystackPatch`` monkey-patches
``haystack.tools.Tool.invoke`` so every governed tool consults the policy before
its body runs. This module provides the policy *interceptor* the adapter calls —
the same ``check_tool_start`` contract the SDK hands its adapters in production —
so the example governs real Haystack tools end-to-end without a live gateway.

- ``LocalPolicyEngine`` — simulates gateway policy (allow / deny) and carries
the ``_enforce`` flag so the adapter is in fail-closed ``enforce`` posture.
"""

from __future__ import annotations

from typing import Any

#: Tool names this demo policy denies (arbitrary-execution surface).
DENIED_TOOLS: frozenset[str] = frozenset({"execute_sql", "run_shell_command"})


class LocalPolicyEngine:
"""Simulates Agent Assembly gateway policy enforcement in offline mode.

Implements the ``check_tool_start`` interceptor contract the Haystack adapter
invokes before a tool runs. ``_enforce = True`` puts the adapter in the
fail-closed ``enforce`` posture so a deny actually blocks the tool (and an
unknown verdict would deny rather than silently allow).
"""

#: Fail-closed posture marker the adapter reads (AAASM-3106 / AAASM-3107).
_enforce = True

def check_tool_start(
self, *, tool_name: str = "", **_kwargs: Any
) -> dict[str, str]:
"""Return an allow/deny verdict for *tool_name*.

The adapter passes the tool name as a keyword; deny the
arbitrary-execution tools, allow everything else.
"""
if tool_name in DENIED_TOOLS:
return {
"status": "deny",
"reason": (
f"Tool '{tool_name}' is blocked by policy rule 'deny_arbitrary_execution'."
),
}
return {"status": "allow"}
Loading