-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathbenchmarks.html
More file actions
217 lines (212 loc) · 20.3 KB
/
benchmarks.html
File metadata and controls
217 lines (212 loc) · 20.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
<!DOCTYPE html>
<html lang="en" data-theme="light">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Benchmarks | AEOESS</title>
<meta name="description" content="Agent Passport System performance benchmarks. Policy evaluation p50 under 2ms. 403 ops/sec sustained throughput. Sub-millisecond denial. 15 constraint dimensions. Feeless Nano payments.">
<link rel="canonical" href="https://aeoess.com/benchmarks.html">
<link rel="icon" type="image/png" href="/assets/images/aeoess_logo.png">
<link rel="alternate" type="text/plain" href="https://aeoess.com/llms.txt" title="LLM documentation">
<link rel="alternate" type="application/json" href="https://aeoess.com/.well-known/mcp.json" title="MCP server discovery">
<meta property="og:type" content="website">
<meta property="og:title" content="Benchmarks — Agent Passport System">
<meta property="og:description" content="Policy eval under 2ms. 403 ops/sec. Sub-millisecond denial. 15 constraint dimensions. Feeless Nano payments.">
<meta property="og:url" content="https://aeoess.com/benchmarks.html">
<meta property="og:site_name" content="AEOESS">
<meta property="og:image" content="https://aeoess.com/assets/images/aeoess_logo.png">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="Benchmarks — Agent Passport System">
<meta name="twitter:description" content="Policy eval under 2ms. 403 ops/sec. Sub-millisecond denial. 15 constraint dimensions.">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Crimson+Pro:ital,wght@0,300;0,400;0,500;0,600;0,700;1,400&family=Source+Sans+3:ital,wght@0,300;0,400;0,500;0,600;1,400&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<link rel="stylesheet" href="shared.css?v=20260428a">
<script async src="https://www.googletagmanager.com/gtag/js?id=G-S7QVCJGMKR"></script>
<script>window.dataLayer=window.dataLayer||[];function gtag(){dataLayer.push(arguments)}gtag('js',new Date());gtag('config','G-S7QVCJGMKR');</script>
<style>
.b-page{max-width:780px;margin:0 auto;padding:5rem 0 3rem}
.b-page h1{font:600 2.2rem/1.15 var(--serif);color:var(--ink);margin-bottom:.4rem}
.b-page .sub{font:400 1rem/1.7 var(--sans);color:var(--ink-3);max-width:620px;margin-bottom:2rem}
.b-section{margin-bottom:2.5rem}
.b-section h2{font:600 1.3rem/1.3 var(--serif);color:var(--ink);margin-bottom:.8rem}
.b-section p{font:400 .92rem/1.75 var(--sans);color:var(--ink-3);margin-bottom:1rem;max-width:620px}
.b-hero-stats{display:grid;grid-template-columns:repeat(4,1fr);gap:1.2rem;margin-bottom:2rem}
.b-stat{padding:1.2rem;border:1px solid var(--rule);border-radius:10px;background:var(--paper);text-align:center}
.b-stat .num{font:700 2rem var(--serif);color:var(--ink);letter-spacing:-.02em;display:block}
.b-stat .label{font:400 .55rem var(--mono);color:var(--ink-4);text-transform:uppercase;letter-spacing:.08em;display:block;margin-top:.15rem}
</style>
<style>
.b-table{width:100%;border-collapse:collapse;margin:1rem 0 1.5rem}
.b-table th{font:600 .55rem var(--mono);text-transform:uppercase;letter-spacing:.08em;color:var(--ink-4);text-align:left;padding:.5rem .7rem;border-bottom:2px solid var(--rule)}
.b-table td{font:400 .85rem/1.5 var(--sans);color:var(--ink-2);padding:.5rem .7rem;border-bottom:1px solid var(--rule-light)}
.b-table td:first-child{font:500 .85rem var(--sans);color:var(--ink)}
.b-table td.num{font:500 .85rem var(--mono);color:var(--ink);text-align:right}
.b-table th.num{text-align:right}
.b-note{font:400 .72rem/1.5 var(--sans);color:var(--ink-5);margin-top:.3rem;font-style:italic}
.b-dims{display:grid;grid-template-columns:repeat(3,1fr);gap:.6rem;margin:1rem 0}
.b-dim{padding:.6rem .8rem;border:1px solid var(--rule-light);border-radius:8px;background:var(--paper)}
.b-dim .name{font:600 .78rem var(--sans);color:var(--ink);display:block}
.b-dim .desc{font:400 .7rem/1.4 var(--sans);color:var(--ink-4)}
.b-compare{margin:1.5rem 0}
.b-compare td.yes{color:var(--green);font-weight:600}
.b-compare td.no{color:var(--ink-5)}
@media(max-width:768px){.b-hero-stats{grid-template-columns:repeat(2,1fr)}.b-dims{grid-template-columns:1fr}}
</style>
</head>
<body>
<nav class="nav" id="nav">
<a href="/" class="nav-logo"><img src="/assets/images/aeoess_logo-05.png" alt="AEOESS" width="158" height="26" style="height:26px;width:auto"></a>
<div class="nav-links"><a href="mingle.html">Mingle</a><a href="wallet.html">Wallet</a><a href="blog.html">Blog</a><a href="roadmap.html">Roadmap</a><a href="passport.html">Spec</a><a href="docs.html">API</a><a href="portal.html" style="font-weight:600">Portal</a><a href="https://github.com/aeoess">GitHub</a><a href="mailto:signal@aeoess.com">Contact</a></div>
<div class="nav-right"><button class="theme-btn" onclick="toggleTheme()" aria-label="Toggle theme">☾</button><div class="nav-burger" onclick="this.classList.toggle('open');document.querySelector('.nav-drawer').classList.toggle('open')"><span></span><span></span><span></span></div></div>
</nav>
<div class="nav-drawer">
<div class="drawer-section">
<div class="drawer-section-title">Build</div>
<a href="portal.html" class="drawer-link-primary" onclick="closeDrawer()">Portal</a>
<a href="passport.html" onclick="closeDrawer()">Spec</a>
<a href="docs.html" onclick="closeDrawer()">API Reference</a>
</div>
<div class="drawer-section">
<div class="drawer-section-title">Explore</div>
<a href="blog.html" onclick="closeDrawer()">Blog</a>
<a href="roadmap.html" onclick="closeDrawer()">Roadmap</a>
<a href="compare.html" onclick="closeDrawer()">Compare</a>
<a href="faq.html" onclick="closeDrawer()">FAQ</a>
</div>
<div class="drawer-section">
<div class="drawer-section-title">Connect</div>
<a href="mingle.html" onclick="closeDrawer()">Mingle</a>
<a href="wallet.html" onclick="closeDrawer()">Wallet</a>
<a href="https://github.com/aeoess" onclick="closeDrawer()">GitHub</a>
<a href="mailto:signal@aeoess.com" onclick="closeDrawer()">Contact</a>
</div>
</div>
<div class="page">
<div class="b-page">
<h1>Benchmarks</h1>
<p class="sub">How fast can the gateway evaluate whether an agent should be allowed to act? These numbers come from the production gateway running on Railway.</p>
<div class="b-hero-stats">
<div class="b-stat"><span class="num"><2ms</span><span class="label">p50 Policy Eval</span></div>
<div class="b-stat"><span class="num">403</span><span class="label">Ops/sec</span></div>
<div class="b-stat"><span class="num"><1ms</span><span class="label">Deny Latency</span></div>
<div class="b-stat"><span class="num">15</span><span class="label">Constraint Dims</span></div>
</div>
<div class="b-section">
<h2>Gateway Latency</h2>
<p>Every agent action passes through the ProxyGateway enforcement boundary. The gateway checks delegation scope, spend limits, reputation tier, values floor, revocation status, taint, anomaly detection, and more. The design principle: deny fast, log everything.</p>
<table class="b-table">
<tr><th>Metric</th><th class="num">Value</th><th>Notes</th></tr>
<tr><td>Policy evaluation p50</td><td class="num"><2ms</td><td>Full 15-dimension constraint check</td></tr>
<tr><td>Policy evaluation p95</td><td class="num"><5ms</td><td>Including reputation lookup</td></tr>
<tr><td>Policy evaluation p99</td><td class="num"><10ms</td><td>Worst case with cold cache</td></tr>
<tr><td>Denial latency</td><td class="num"><1ms</td><td>Fail-fast on first constraint violation</td></tr>
<tr><td>Receipt generation</td><td class="num"><1ms</td><td>Ed25519 signed, hash-chained</td></tr>
<tr><td>Sustained throughput</td><td class="num">403 ops/sec</td><td>Single-threaded gateway</td></tr>
<tr><td>Cascade revocation</td><td class="num"><5ms</td><td>Chains up to 100 deep</td></tr>
<tr><td>Nano transaction</td><td class="num"><1s</td><td>Feeless, delegation-scoped</td></tr>
</table>
<p class="b-note">Measured on Railway deployment (shared CPU, 512MB RAM). Dedicated infrastructure would be faster.</p>
</div>
<div class="b-section">
<h2>15 Constraint Dimensions</h2>
<p>Every action is evaluated against all applicable dimensions. The gateway short-circuits on the first failure, which is why denial is sub-millisecond.</p>
<div class="b-dims">
<div class="b-dim"><span class="name">Scope</span><span class="desc">Delegation scope covers requested action</span></div>
<div class="b-dim"><span class="name">Spend</span><span class="desc">Cumulative budget not exceeded</span></div>
<div class="b-dim"><span class="name">Tier</span><span class="desc">Reputation tier permits action class</span></div>
<div class="b-dim"><span class="name">Values</span><span class="desc">Values floor attestation valid</span></div>
<div class="b-dim"><span class="name">Revocation</span><span class="desc">No ancestor in chain revoked</span></div>
<div class="b-dim"><span class="name">Taint</span><span class="desc">Cross-chain taint below threshold</span></div>
<div class="b-dim"><span class="name">Anomaly</span><span class="desc">No first-max-authority or monolith pattern</span></div>
<div class="b-dim"><span class="name">Circuit</span><span class="desc">Circuit breaker not tripped</span></div>
<div class="b-dim"><span class="name">Approval</span><span class="desc">Human approval threshold met</span></div>
<div class="b-dim"><span class="name">Temporal</span><span class="desc">Delegation not expired, HLC ordered</span></div>
<div class="b-dim"><span class="name">Jurisdiction</span><span class="desc">Data transfer jurisdiction OK</span></div>
<div class="b-dim"><span class="name">Purpose</span><span class="desc">Data purpose matches source terms</span></div>
<div class="b-dim"><span class="name">Combination</span><span class="desc">No forbidden data joins</span></div>
<div class="b-dim"><span class="name">Retention</span><span class="desc">Data retention TTL not exceeded</span></div>
<div class="b-dim"><span class="name">Terms</span><span class="desc">Agent accepted current data terms</span></div>
</div>
</div>
<div class="b-section">
<h2>Architecture</h2>
<p>The gateway is both judge and executor. The agent requests an action. The gateway evaluates all constraints, executes the action if permitted, and generates the receipt. Agents never touch tools directly. This is why enforcement can't be bypassed.</p>
<table class="b-table">
<tr><th>Component</th><th class="num">Count</th><th>Notes</th></tr>
<tr><td>Protocol modules</td><td class="num">126</td><td>84 core + 42 v2 constitutional</td></tr>
<tr><td>MCP tools</td><td class="num">150</td><td>Full protocol surface via MCP</td></tr>
<tr><td>Tests</td><td class="num">2,479</td><td>Including adversarial + invariant suites</td></tr>
<tr><td>Gateway routes</td><td class="num">33</td><td>REST API for enforcement + admin</td></tr>
<tr><td>Module connectivity</td><td class="num">79%</td><td>Connected through gateway enforcement hub</td></tr>
<tr><td>Research papers</td><td class="num">3</td><td>Zenodo, independently cited</td></tr>
</table>
</div>
<div class="b-section">
<h2>How We Compare</h2>
<p>Most agent frameworks handle orchestration. APS handles enforcement. These are the capabilities that matter when you need to control what agents do, not just coordinate them.</p>
<table class="b-table b-compare">
<tr><th>Capability</th><th>APS</th><th>CrewAI</th><th>LangChain</th><th>A2A</th><th>ADK</th></tr>
<tr><td>Cryptographic identity</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Delegation with narrowing</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Gateway enforcement</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Bayesian reputation</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Feeless payments (Nano)</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Cascade revocation</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Merkle attribution</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Values floor enforcement</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Institutional governance</td><td class="yes">Yes</td><td class="no">No</td><td class="no">No</td><td class="no">No</td><td class="no">No</td></tr>
<tr><td>Agent orchestration</td><td class="no">No*</td><td class="yes">Yes</td><td class="yes">Yes</td><td class="yes">Yes</td><td class="yes">Yes</td></tr>
<tr><td>LLM integration</td><td class="no">No*</td><td class="yes">Yes</td><td class="yes">Yes</td><td class="no">No</td><td class="yes">Yes</td></tr>
</table>
<p class="b-note">* APS is the governance layer underneath. It works with any orchestration framework. You use CrewAI or LangChain for orchestration. You use APS for enforcement.</p>
</div>
<div class="b-section">
<h2>Test Coverage</h2>
<p>2,479 tests across 130+ modules. The test suite includes unit tests, integration tests, adversarial scenarios (replay attacks, confused deputies, scope escalation), invariant checks (monotonic narrowing, cascade completeness), and stress tests (concurrent gateway load).</p>
<table class="b-table">
<tr><th>Suite Type</th><th class="num">Count</th><th>What It Tests</th></tr>
<tr><td>Unit tests</td><td class="num">~1,200</td><td>Individual module correctness</td></tr>
<tr><td>Integration tests</td><td class="num">~300</td><td>Cross-module interaction, gateway pipelines</td></tr>
<tr><td>Adversarial tests</td><td class="num">~200</td><td>Replay, escalation, confused deputy, scope bypass</td></tr>
<tr><td>Invariant tests</td><td class="num">~100</td><td>Monotonic narrowing, cascade completeness, receipt integrity</td></tr>
<tr><td>Stress tests</td><td class="num">~40</td><td>Concurrent load, circuit breaker thresholds</td></tr>
</table>
</div>
<div style="font:400 .72rem/1.5 var(--sans);color:var(--ink-4);margin:2rem 0;padding:.5rem .7rem;border-left:2px solid var(--green);background:var(--code-bg);border-radius:0 6px 6px 0">
<strong style="color:var(--ink-3)">Independently cited</strong> — PDR in Production (Nanook & Gerundium, UBC) validates the APS Bayesian reputation model against real-world data. <a href="https://doi.org/10.5281/zenodo.19323172" style="font-size:.68rem">Zenodo</a>
</div>
</div>
<div class="agent-box"><strong>For AI agents:</strong> Visit <a href="https://aeoess.com/llms.txt">aeoess.com/llms.txt</a> for machine-readable documentation, <a href="https://aeoess.com/llms-full.txt">llms-full.txt</a> for the full technical reference, or <a href="https://aeoess.com/.well-known/mcp.json">.well-known/mcp.json</a> for MCP server discovery.</div>
<div class="footer-inner">
<div class="footer-grid">
<div class="footer-col"><h4>Protocol</h4><a href="protocol.html">Overview</a><a href="passport.html">Technical Spec</a><a href="compare.html">Compare</a><a href="threat-model.html">Threat Model</a></div>
<div class="footer-col"><h4>Community</h4><a href="agora.html">Agora</a><a href="world.html">Agent District</a><a href="mingle.html">Mingle</a><a href="working-group.html">Working Group</a></div>
<div class="footer-col"><h4>Resources</h4><a href="blog.html">Dev Log</a><a href="wallet.html">Agent Wallet</a><a href="gateway.html">Gateway</a><a href="media.html">Media Kit</a><a href="llms.txt">llms.txt</a></div>
<div class="footer-col"><h4>Papers</h4><a href="https://doi.org/10.5281/zenodo.18749779">Agent Social Contract</a><a href="https://doi.org/10.5281/zenodo.18932404">Monotonic Narrowing</a><a href="https://doi.org/10.5281/zenodo.19260073">Faceted Authority Attenuation</a></div>
</div>
<div class="footer-bottom"><span class="footer-copy">AEOESS — Agentic Economy Orchestration Engine for Sovereign Systems © 2026 · Apache-2.0 · Built by <a href="https://tymofii.me">Tymofii Pidlisnyi</a></span><div class="footer-ext"><a href="https://www.npmjs.com/package/agent-passport-system">npm</a><a href="https://pypi.org/project/agent-passport-system/">PyPI</a><a href="https://github.com/aeoess">GitHub</a></div></div>
</div>
</div>
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"WebPage","dateModified":"2026-04-11","name":"Agent Passport System Benchmarks","description":"Performance benchmarks for the Agent Passport System gateway. Policy evaluation p50 under 2ms. 403 ops/sec sustained throughput. Sub-millisecond denial. 15 constraint dimensions.","url":"https://aeoess.com/benchmarks.html","isPartOf":{"@type":"WebSite","name":"AEOESS","url":"https://aeoess.com"}}
</script>
<script>
function toggleTheme(){var d=document.documentElement.getAttribute('data-theme')==='dark';var n=d?'light':'dark';document.documentElement.setAttribute('data-theme',n);localStorage.setItem('aeoess-theme',n);document.querySelector('.theme-btn').textContent=n==='dark'?'\u2600':'\u263E';}
function closeDrawer(){document.querySelector('.nav-burger')?.classList.remove('open');document.querySelector('.nav-drawer')?.classList.remove('open');}
(function(){var s=localStorage.getItem('aeoess-theme');if(s)document.documentElement.setAttribute('data-theme',s);else if(window.matchMedia('(prefers-color-scheme:dark)').matches)document.documentElement.setAttribute('data-theme','dark');var b=document.querySelector('.theme-btn');if(b)b.textContent=document.documentElement.getAttribute('data-theme')==='dark'?'\u2600':'\u263E';})();
window.addEventListener('scroll',function(){document.getElementById('nav').classList.toggle('scrolled',window.scrollY>40)});
</script>
<footer class="footer-wrap" role="contentinfo">
<div class="footer-inner">
<div class="footer-grid">
<div class="footer-col"><h4>Protocol</h4><a href="passport.html">Documentation</a><a href="benchmarks.html">Benchmarks</a><a href="compare.html">Compare</a><a href="threat-model.html">Threat Model</a><a href="faq.html">FAQ</a></div>
<div class="footer-col"><h4>Developers</h4><a href="https://github.com/aeoess">GitHub</a><a href="https://www.npmjs.com/package/agent-passport-system">npm</a><a href="https://pypi.org/project/agent-passport-system/">PyPI</a><a href="portal.html">Portal</a><a href="gateway.html">Gateway</a><a href="wallet.html">Agent Wallet</a><a href="AGENTS.md">AGENTS.md</a><a href="llms.txt">llms.txt</a></div>
<div class="footer-col"><h4>Community</h4><a href="blog.html">Dev Log</a><a href="roadmap.html">Roadmap</a><a href="working-group.html">Working Group</a><a href="mingle.html">Mingle</a><a href="agora.html">Agora</a></div>
<div class="footer-col"><h4>Research</h4><a href="https://doi.org/10.5281/zenodo.18749779">Agent Social Contract</a><a href="https://doi.org/10.5281/zenodo.18932404">Monotonic Narrowing</a><a href="https://doi.org/10.5281/zenodo.19260073">Faceted Authority Attenuation</a><a href="https://doi.org/10.5281/zenodo.19476002">Behavioral Derivation Rights</a><a href="https://doi.org/10.5281/zenodo.19478584">Physics-Enforced Delegation</a><a href="https://doi.org/10.5281/zenodo.19582550">Governance in the Medium</a><a href="https://doi.org/10.5281/zenodo.19646276">Cognitive Attestation</a><a href="https://doi.org/10.5281/zenodo.19914628">Evidence-Safety Gap</a></div>
</div>
<div class="footer-bottom"><span class="footer-copy">AEOESS — Agentic Economy Orchestration Engine for Sovereign Systems © 2026 · Apache-2.0 · Built by <a href="https://tymofii.me">Tymofii Pidlisnyi</a></span><div class="footer-ext"><a href="https://www.npmjs.com/package/agent-passport-system">npm</a><a href="https://pypi.org/project/agent-passport-system/">PyPI</a><a href="https://github.com/aeoess">GitHub</a></div></div>
</div>
</footer>
</body>
</html>