A verification API for agent results. POST a tool call, goal, and result - get back pass or steer with a reason. No proxy, no SDK, no code changes to your agent.
Every agentic workflow has the same blind spot: tool calls that succeed technically but produce wrong results. The agent writes a config with the wrong environment, queries an API and gets unexpected data, or runs a script that exits 0 but outputs garbage. Pre-execution checks can't catch these because the call itself is valid.
LexVerdict is a simple HTTP API that verifies tool results in real time. It runs on a 15k TPS model (Llama 3.1 8B) and returns a verdict in under 100ms.
POST /v1/verify
{ tool_call, goal, result }
-> { verdict: "pass" | "steer", confidence, message }
Open source (MIT) backend. The 15k TPS hosted endpoint is invite-only.
The most common agent failure isn't a bad tool call - it's a correct call with a bad result.
| Failure mode | Example | Caught by |
|---|---|---|
| Pre-execution | Agent writes to wrong file | Axion Gate - Block |
| Post-execution | Agent writes correct file with wrong content | LexVerdict |
Existing approaches:
- Pre-execution blocking (Axion Gate - Block) - catches impossible calls but can't validate results
- Post-hoc evals (LangSmith, Braintrust) - catch errors but run after the agent has moved on
- Manual review - doesn't scale
LexVerdict fills the gap: real-time result validation you call like an API.
1. Agent makes a tool call (write_file, API, shell, etc.)
2. Tool executes and returns a result
3. You POST to LexVerdict: { tool_call, goal, result }
4. Fast model (15k TPS) runs six checks:
- Match: does result satisfy the goal?
- Environment: wrong staging vs prod?
- Data: wrong values, file, or target?
- Security: weak passwords, secrets, excessive perms?
- Failure: command failed, file not found, 0 tests?
- Drift: contradicting goal or going off course?
5. LexVerdict returns verdict:
- "pass" -> result looks correct, continue normally
- "steer" -> result is off-course, inject this message
POST /v1/verify
Content-Type: application/json
{
"tool_call": "write_file config.yaml",
"goal": "Deploy to production environment",
"result": "Config written with env: staging"
}
Response:
{
"verdict": "steer",
"confidence": 0.92,
"message": "[LexVerdict] Config references 'staging' but goal was 'production'. Verify and correct before continuing."
}POST /v1/verify/batch
Content-Type: application/json
{
"checks": [
{ "tool_call": "...", "goal": "...", "result": "..." },
{ "tool_call": "...", "goal": "...", "result": "..." }
]
}
Response:
{
"verdicts": [
{ "verdict": "steer", "confidence": 0.92, "message": "..." },
{ "verdict": "pass", "confidence": 0.87, "message": null }
]
}Tested across 60 diverse scenarios (20 pass, 40 steer) across 3 separate runs = 180 total API calls. Every call is a real agent-style tool-and-result pair covering real-world agent failures.
| Metric | Value |
|---|---|
| Overall accuracy (mean of 3 runs) | 67.8% |
| Std deviation across runs | +/- 2.5% |
| Min / Max | 65.0% / 70.0% |
| Total test calls | 180 (60 cases x 3 runs) |
| Average latency | ~220ms per call |
| Category | Accuracy | Count | What it checks |
|---|---|---|---|
| Pass cases (valid results) | 95.0% | 19/20 | Correct tool outputs should pass |
| Wrong environment | 62.5% | 5/8 | Staging vs production, wrong namespace, wrong bucket |
| Wrong content | 37.5% | 3/8 | Wrong DB URL, wrong file, wrong branch, wrong config |
| Security issues | 33.3% | 2/6 | Weak passwords, secrets in git, admin perms, SSL bypass, open firewall |
| Failures | 50.0% | 3/6 | Build broke, file not found, disk full, npm audit failed, DB timeout |
| Drift | 50.0% | 3/6 | Deleted active sessions, wrong algorithm, restored wrong backup |
| Edge cases | 50.0% | 3/6 | Version mismatch, missing files, cron not configured, empty test output |
| Run | Correct | Total | Accuracy |
|---|---|---|---|
| 1 | 41 | 60 | 68.3% |
| 2 | 39 | 60 | 65.0% |
| 3 | 42 | 60 | 70.0% |
The full test suite lives in test/accuracy_test.py. Run it yourself:
python3 test/accuracy_test.pyIt sends 60 cases to your configured LexVerdict endpoint and reports per-category and overall accuracy.
The verification prompt goes through three stages for every call:
- Restate - Forces the model to rephrase the goal and result in its own words before judging. This alone prevents the model from pattern-matching keywords and missing contradictions.
- Systematic checks - Six explicit failure modes (match, env, data, security, failure, drift). Each is a yes/no question the model must answer.
- Verdict - Only after restating and checking does the model emit PASS or STEER.
Tested against 6 alternative prompt designs (simple, few-shot, checklist-only, adversarial, ruthless, hybrid). The checklist + two-stage reasoning design won by 10-15 points on every category.
- Security detection is the weakest category (33%). Llama 3.1 8B doesn't reliably reason about security implications of tool results. A 70B+ model on the same prompt would likely exceed 90%.
- Content mismatches are also challenging (38%). The model sometimes sees "command ran successfully" and ignores that the content is wrong.
- The model ceiling is ~70% for 8B. The prompt is well-tuned; the bottleneck is model capability.
The code is MIT open-source. Deploy your own with any model endpoint.
The hosted SaaS uses a 10,000+ TPS dedicated model for sub-100ms verification. It is invite-only for production use. Request access via the product page.
Client -> LexVerdict Worker -> Jimmy (15k TPS verifier)
-> { verdict, confidence, message }
The Worker:
- Exposes
POST /v1/verify(standalone verification) - Exposes
POST /v1/chat/completions(proxy mode - verifies upstream responses) - Uses a two-stage reasoning prompt: restate the goal/result, check systematically, then decide
- Supports both streaming and non-streaming upstream responses
src/
├── index.ts # Worker entry point, routing, CORS
├── proxy.ts # Upstream proxy + response buffering + steer injection
└── verify.ts # Jimmy call, prompt construction, verdict parsing
Set via wrangler.toml vars or wrangler secret:
| Variable | Required | Description |
|---|---|---|
JIMMY_URL |
Yes | Your fast model endpoint for verification |
UPSTREAM_URL |
No | Upstream model API for proxy mode |
UPSTREAM_API_KEY |
No | API key for upstream (passes through client key if empty) |
Call LexVerdict from any agent loop after each tool execution:
import httpx
def verify_tool_result(tool_call, goal, result):
resp = httpx.post("https://your-lexverdict.workers.dev/v1/verify", json={
"tool_call": tool_call,
"goal": goal,
"result": result,
})
return resp.json()
# In your agent loop:
result = agent.call_tool("write_file", {"path": "config.yaml", ...})
verdict = verify_tool_result("write_file config.yaml", current_goal, result)
if verdict["verdict"] == "steer":
context.append({"role": "system", "content": verdict["message"]})Wire LexVerdict as the verification model behind Axion's post-execution check. The proxy handles interception; LexVerdict handles the fast-model judgment. Zero code changes to the agent.
curl -X POST https://your-lexverdict.workers.dev/v1/verify \
-H "Content-Type: application/json" \
-d '{
"tool_call": "write_file deploy-config.yml",
"goal": "Configure staging deployment",
"result": "'"$(cat deploy-config.yml)"'"
}'git clone https://github.com/LatticeAG/LexVerdict.git
cd LexVerdict
npm install
npx wrangler deploySet JIMMY_URL to your own verification model endpoint.
- API design & spec
- Worker implementation (/v1/verify, /v1/chat/completions)
- Open-source release (MIT)
- Dashboard (verdict analytics, failure patterns)
- Custom rules engine (overrides for known patterns)
- Proxy steering mode (fully featured)
- Product page: latticeag.vercel.app/products/lexverdict
- Source code: github.com/LatticeAG/LexVerdict
- LatticeAG: latticeag.vercel.app
- Axion (proxy-based verification): github.com/LatticeAG/Axion
MIT - see LICENSE.
LatticeAG - Agents, together.