Skip to content

Commit f75920d

Browse files
authored
docs: update condition tree controls (#28)
* docs: update condition tree controls * build: fix docs validation script * docs: clarify condition submodels * docs: fix regex escaping in curl example
1 parent 4e6408a commit f75920d

12 files changed

Lines changed: 220 additions & 102 deletions

File tree

components/models.mdx

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,14 @@ step = Step(
5252
### Control Models
5353

5454
```python
55-
from agent_control_models import ControlDefinition, ControlScope, ControlAction
55+
from agent_control_models import (
56+
ConditionNode,
57+
ControlAction,
58+
ControlDefinition,
59+
ControlScope,
60+
ControlSelector,
61+
EvaluatorSpec,
62+
)
5663

5764
control = ControlDefinition(
5865
description="Block toxic user messages",
@@ -62,8 +69,10 @@ control = ControlDefinition(
6269
step_types=["llm"],
6370
stages=["pre"],
6471
),
65-
selector={"path": "input"},
66-
evaluator={"name": "regex", "config": {"pattern": "toxic"}},
72+
condition=ConditionNode(
73+
selector=ControlSelector(path="input"),
74+
evaluator=EvaluatorSpec(name="regex", config={"pattern": "toxic"}),
75+
),
6776
action=ControlAction(decision="deny"),
6877
)
6978
```
@@ -135,11 +144,32 @@ Complete control specification.
135144
- `enabled` (bool): Whether control is active
136145
- `execution` (str): Execution mode (`server` or `sdk`)
137146
- `scope` (ControlScope): When to apply the control
138-
- `selector` (ControlSelector): What data to evaluate
139-
- `evaluator` (EvaluatorSpec): How to evaluate
147+
- `condition` (ConditionNode): Recursive condition tree; leaf nodes contain `selector` + `evaluator`
140148
- `action` (ControlAction): What to do on match
141149
- `tags` (List[str]): Tags for categorization
142150

151+
### ConditionNode
152+
153+
Recursive condition tree used by `ControlDefinition.condition`.
154+
155+
- Leaf nodes contain both `selector` and `evaluator`
156+
- Composite nodes contain exactly one of `and`, `or`, or `not`
157+
- Maximum nesting depth is 6
158+
159+
### ControlSelector
160+
161+
Selector used inside a leaf condition.
162+
163+
- `path` (Optional[str]): Dot-path for the value to evaluate; defaults to `"*"`
164+
- Common paths: `input`, `output`, `input.query`, `context.user_id`, `name`, `*`
165+
166+
### EvaluatorSpec
167+
168+
Evaluator used inside a leaf condition.
169+
170+
- `name` (str): Evaluator name, such as `regex`, `list`, `sql`, or `galileo.luna2`
171+
- `config` (Dict[str, Any]): Evaluator-specific configuration payload
172+
143173
### EvaluationRequest
144174

145175
Request for evaluating controls.

concepts/controls.mdx

Lines changed: 104 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ description: Understand how controls work and how to define them.
77

88
A Control is a protection rule that evaluates agent interactions (inputs/outputs) and takes action based on configured criteria. It defines when to check, what to check, how to evaluate it, and what to do with the results.
99

10-
Control formula: **Control = Scope (When) + Selector (What) + Evaluator (How) + Action (Decision)**
10+
Control formula: **Control = Scope (When) + Condition (What + How) + Action (Decision)**
1111

1212
## 1. Scope (When to Check)
1313

@@ -75,9 +75,66 @@ Fields:
7575

7676
---
7777

78-
## 2. Selector (What to Check)
78+
## 2. Condition (What and How to Check)
7979

80-
The **Selector** specifies which portion of the step's data to extract and pass to the evaluator for analysis. It uses a path specification to navigate the step object.
80+
The **Condition** is a recursive boolean tree. Leaf conditions pair a `selector` with an `evaluator`, and composite conditions can combine child conditions with `and`, `or`, and `not`.
81+
82+
### Example 1: Leaf condition that checks tool output for PII
83+
84+
```json
85+
{
86+
"condition": {
87+
"selector": {
88+
"path": "output"
89+
},
90+
"evaluator": {
91+
"name": "regex",
92+
"config": {
93+
"pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b"
94+
}
95+
}
96+
}
97+
}
98+
```
99+
100+
### Example 2: Composite condition with `and` and `not`
101+
102+
```json
103+
{
104+
"condition": {
105+
"and": [
106+
{
107+
"selector": {
108+
"path": "context.risk_level"
109+
},
110+
"evaluator": {
111+
"name": "list",
112+
"config": {
113+
"values": ["high", "critical"]
114+
}
115+
}
116+
},
117+
{
118+
"not": {
119+
"selector": {
120+
"path": "context.user_role"
121+
},
122+
"evaluator": {
123+
"name": "list",
124+
"config": {
125+
"values": ["admin", "security"]
126+
}
127+
}
128+
}
129+
}
130+
]
131+
}
132+
}
133+
```
134+
135+
### 2.1 Selector (What to Check Inside a Leaf)
136+
137+
Inside a leaf condition, the **Selector** specifies which portion of the step's data to extract and pass to the evaluator for analysis. It uses a path specification to navigate the step object.
81138

82139
Field:
83140

@@ -141,9 +198,9 @@ Common Paths:
141198

142199
---
143200

144-
## 3. Evaluator (How to Check)
201+
### 2.2 Evaluator (How to Check Inside a Leaf)
145202

146-
The **Evaluator** receives the data extracted by the selector and evaluates it against configured rules, returning whether the data matches specified criteria.
203+
Inside a leaf condition, the **Evaluator** receives the data extracted by the selector and evaluates it against configured rules, returning whether the data matches specified criteria.
147204

148205
Components:
149206

@@ -217,7 +274,7 @@ Agent Control supports custom evaluators for domain-specific requirements. See [
217274

218275
---
219276

220-
## 4. Action (What to Do)
277+
## 3. Action (What to Do)
221278

222279
The **Action** defines what happens when the evaluator matches/detects an issue.
223280

@@ -313,13 +370,15 @@ Putting it all together - a control that blocks Social Security Numbers in tool
313370
"step_types": ["tool"],
314371
"stages": ["post"]
315372
},
316-
"selector": {
317-
"path": "output"
318-
},
319-
"evaluator": {
320-
"name": "regex",
321-
"config": {
322-
"pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b"
373+
"condition": {
374+
"selector": {
375+
"path": "output"
376+
},
377+
"evaluator": {
378+
"name": "regex",
379+
"config": {
380+
"pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b"
381+
}
323382
}
324383
},
325384
"action": {
@@ -354,10 +413,12 @@ async with AgentControlClient() as client:
354413
"enabled": True,
355414
"execution": "server",
356415
"scope": {"step_names": ["generate_response"], "stages": ["post"]},
357-
"selector": {"path": "output"},
358-
"evaluator": {
359-
"name": "regex",
360-
"config": {"pattern": r"\\b\\d{3}-\\d{2}-\\d{4}\\b"}
416+
"condition": {
417+
"selector": {"path": "output"},
418+
"evaluator": {
419+
"name": "regex",
420+
"config": {"pattern": r"\b\d{3}-\d{2}-\d{4}\b"}
421+
},
361422
},
362423
"action": {"decision": "deny"}
363424
}
@@ -384,16 +445,20 @@ curl -X PUT "http://localhost:8000/api/v1/controls/$CONTROL_ID/data" \
384445
"enabled": true,
385446
"execution": "server",
386447
"scope": {"step_names": ["generate_response"], "stages": ["post"]},
387-
"selector": {"path": "output"},
388-
"evaluator": {
389-
"name": "regex",
390-
"config": {"pattern": "\\\\b\\\\d{3}-\\\\d{2}-\\\\d{4}\\\\b"}
448+
"condition": {
449+
"selector": {"path": "output"},
450+
"evaluator": {
451+
"name": "regex",
452+
"config": {"pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b"}
453+
}
391454
},
392455
"action": {"decision": "deny"}
393456
}
394457
}'
395458
```
396459

460+
Regex pattern note: the pattern itself is `\b\d{3}-\d{2}-\d{4}\b`. Python raw strings render that as `r"\b\d{3}-\d{2}-\d{4}\b"`, while JSON payloads must escape backslashes as `"\\b\\d{3}-\\d{2}-\\d{4}\\b"`.
461+
397462
### Example: Block Toxic Input (Luna-2 AI)
398463

399464
To use this evaluator, install the package and restart the server.
@@ -413,13 +478,15 @@ await controls.create_control(
413478
"enabled": True,
414479
"execution": "server",
415480
"scope": {"step_names": ["process_user_message"], "step_types": ["llm"], "stages": ["pre"]},
416-
"selector": {"path": "input"},
417-
"evaluator": {
418-
"name": "galileo.luna2",
419-
"config": {
420-
"metric": "input_toxicity",
421-
"operator": "gt",
422-
"target_value": 0.5
481+
"condition": {
482+
"selector": {"path": "input"},
483+
"evaluator": {
484+
"name": "galileo.luna2",
485+
"config": {
486+
"metric": "input_toxicity",
487+
"operator": "gt",
488+
"target_value": 0.5
489+
}
423490
}
424491
},
425492
"action": {"decision": "deny"}
@@ -445,13 +512,15 @@ curl -X PUT "http://localhost:8000/api/v1/controls/$CONTROL_ID/data" \
445512
"enabled": true,
446513
"execution": "server",
447514
"scope": {"step_names": ["process_user_message"], "step_types": ["llm"], "stages": ["pre"]},
448-
"selector": {"path": "input"},
449-
"evaluator": {
450-
"name": "galileo.luna2",
451-
"config": {
452-
"metric": "input_toxicity",
453-
"operator": "gt",
454-
"target_value": 0.5
515+
"condition": {
516+
"selector": {"path": "input"},
517+
"evaluator": {
518+
"name": "galileo.luna2",
519+
"config": {
520+
"metric": "input_toxicity",
521+
"operator": "gt",
522+
"target_value": 0.5
523+
}
455524
}
456525
},
457526
"action": {"decision": "deny"}
@@ -460,4 +529,3 @@ curl -X PUT "http://localhost:8000/api/v1/controls/$CONTROL_ID/data" \
460529
```
461530

462531
> **Note**: For the Luna-2 evaluator, set the `GALILEO_API_KEY` environment variable. See the [Evaluators](/concepts/evaluators/overview) for all available evaluators.
463-

concepts/overview.mdx

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Overview
3-
description: Core concepts behind Agent Control — policies, controls, selectors, evaluators, and actions.
3+
description: Core concepts behind Agent Control — policies, controls, conditions, selectors, evaluators, and actions.
44
---
55

66
Understanding these core concepts will help you get the most out of Agent Control. Start with the high-level [architecture](/concepts/architecture) to see how components fit together, then dive into [evaluators](/concepts/evaluators/overview) to understand how checks are implemented.
@@ -10,7 +10,7 @@ Understanding these core concepts will help you get the most out of Agent Contro
1010
A **[Control](/concepts/controls)** is a single rule that defines what to check and what to do when a condition is met.
1111

1212
```text
13-
Control = Scope + Selector + Evaluator + Action
13+
Control = Scope + Condition + Action
1414
```
1515

1616
Example: "If the output contains an SSN pattern, block the response."
@@ -20,15 +20,19 @@ Example: "If the output contains an SSN pattern, block the response."
2020
"name": "block-ssn-in-output",
2121
"execution": "server",
2222
"scope": { "step_types": ["llm"], "stages": ["post"] },
23-
"selector": { "path": "output" },
24-
"evaluator": {
25-
"name": "regex",
26-
"config": { "pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b" }
23+
"condition": {
24+
"selector": { "path": "output" },
25+
"evaluator": {
26+
"name": "regex",
27+
"config": { "pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b" }
28+
}
2729
},
2830
"action": { "decision": "deny" }
2931
}
3032
```
3133

34+
Leaf conditions pair a `selector` with an `evaluator`. Composite conditions can use `and`, `or`, and `not` to combine multiple leaf checks.
35+
3236
## Scope
3337

3438
**Scope** defines when a control runs by filtering which steps are evaluated.

core/quickstart.mdx

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -160,10 +160,12 @@ async def setup():
160160
"enabled": True,
161161
"execution": "server",
162162
"scope": {"stages": ["post"]},
163-
"selector": {"path": "output"},
164-
"evaluator": {
165-
"name": "regex",
166-
"config": {"pattern": r"\b\d{3}-\d{2}-\d{4}\b"},
163+
"condition": {
164+
"selector": {"path": "output"},
165+
"evaluator": {
166+
"name": "regex",
167+
"config": {"pattern": r"\b\d{3}-\d{2}-\d{4}\b"},
168+
},
167169
},
168170
"action": {"decision": "deny"},
169171
},
@@ -182,6 +184,8 @@ async def setup():
182184
asyncio.run(setup())
183185
```
184186

187+
Controls store leaf `selector` and `evaluator` definitions under `condition`, which also enables composite `and`, `or`, and `not` trees.
188+
185189
Now, run your agent code.
186190

187191
**🎉 Done!** Your agent now blocks SSN patterns automatically.

core/reference.mdx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ uv run mypy . # Type-check
3737
A **Control** is a single rule that defines what to check and what to do when a condition is met.
3838

3939
```text
40-
Control = Scope + Selector + Evaluator + Action
40+
Control = Scope + Condition + Action
4141
```
4242

4343
**Control associations** are direct links between controls and agents.
@@ -286,4 +286,3 @@ make alembic-upgrade
286286
- [Models](/components/models)
287287
- [Evaluators](/components/evaluators)
288288
- [UI Dashboard](/components/ui)
289-

examples/agent-control-demo.mdx

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -103,8 +103,7 @@ Key insight: controls are enforced server-side, so you can update rules in real
103103
Controls are defined on the server with:
104104

105105
- Scope: When to check (step types, stages: pre or post)
106-
- Selector: What to check (input, output, specific fields)
107-
- Evaluator: How to check (regex patterns, list matching, AI-based)
106+
- Condition: What and how to check
108107
- Action: What to do (allow, deny, steer, warn, log)
109108

110109
Example from `setup_controls.py`:
@@ -116,10 +115,12 @@ control_data = ControlDefinition(
116115
enabled=True,
117116
execution="server",
118117
scope=ControlScope(step_types=["tool"], stages=["post"]),
119-
selector=ControlSelector(path="output"),
120-
evaluator=EvaluatorConfig(
121-
name="regex",
122-
config={"pattern": r"\b\d{3}-\d{2}-\d{4}\b"},
118+
condition=ConditionNode(
119+
selector=ControlSelector(path="output"),
120+
evaluator=EvaluatorSpec(
121+
name="regex",
122+
config={"pattern": r"\b\d{3}-\d{2}-\d{4}\b"},
123+
),
123124
),
124125
action=ControlAction(decision="deny"),
125126
)

0 commit comments

Comments
 (0)