Python: Information-flow control based prompt injection defense

[shrutitople]

Motivation and Context

LLM agents are vulnerable to prompt injection attacks — malicious instructions in external content (tool results, API responses) that cause data exfiltration or unauthorized actions.
This PR introduces FIDES, a deterministic defense based on information flow control (IFC). Instead of detecting injections, it tracks content provenance via labels and enforces policies — untrusted content can't influence trusted operations, private data can't leak to public channels.

Description

Security Primitives, Middleware & Tools — _security.py (single consolidated module)

• Labels: IntegrityLabel (trusted/untrusted) × ConfidentialityLabel (public/private/user_identity)
• Lattice combination: most-restrictive-wins via combine_labels()
• Variable indirection: ContentVariableStore replaces untrusted content with opaque VariableReferenceContent placeholders — the LLM never sees raw untrusted data
• LabelTrackingFunctionMiddleware — 3-tier automatic label propagation:
  1. Per-item embedded labels (additional_properties.security_label)
  2. Tool-level source_integrity declaration
  3. Join of input argument labels (fallback)
• PolicyEnforcementFunctionMiddleware — blocks or requests approval when context confidentiality exceeds a tool's max_allowed_confidentiality
• SecureAgentConfig — one-line setup wiring middleware, tools, and instructions
• quarantined_llm — isolated LLM call (no tools) for safe summarization of untrusted content
• inspect_variable — controlled access to hidden variables with label awareness
• All results use list[Content] (aligned with upstream FunctionTool.invoke())

Framework Integration — _tools.py, DevUI

• FunctionApprovalRequest content type for human-in-the-loop policy enforcement
• DevUI maps approval requests to interactive approve/reject UI

Tests — test_security.py

• 115 unit tests covering label propagation, variable indirection, policy enforcement, quarantine, 3-tier labeling, and edge cases

Samples — python/samples/02-agents/security/

Sample	Demonstrates
email_security_example.py	Integrity-based defense against injection in email content
repo_confidentiality_example.py	Confidentiality-based data exfiltration prevention
github_mcp_labels_example.py	Integration with GitHub MCP server labels

Documentation

• FIDES_DEVELOPER_GUIDE.md (in python/samples/02-agents/security/), python/samples/02-agents/security/README.md, docs/features/FIDES_IMPLEMENTATION_SUMMARY.md

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible (115 new tests)
• Is this a breaking change? No — all changes are additive; security middleware is opt-in via SecureAgentConfig

[Copilot]

Pull request overview

Introduces FIDES, an information-flow control (IFC) security layer for the agent framework to deterministically mitigate prompt injection and data exfiltration via integrity/confidentiality labels, variable indirection, and policy enforcement.

Changes:

• Add core security primitives (labels, variable store, lineage) plus security middleware for label propagation and policy enforcement.
• Add security tools (quarantined_llm, inspect_variable) and DevUI support for displaying/handling policy-violation approval requests.
• Add new security samples and extensive documentation/ADRs describing FIDES usage and design.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
python/samples/getting_started/security/repo_confidentiality_example.py	Sample demonstrating confidentiality-based exfiltration prevention.
python/samples/getting_started/security/github_mcp_labels_example.py	Sample demonstrating parsing GitHub MCP label metadata and enforcing policies.
python/samples/getting_started/security/email_security_example.py	Sample demonstrating integrity-based prompt injection defense + quarantine processing.
python/samples/getting_started/security/init.py	Marks security samples as a package.
python/packages/devui/agent_framework_devui/_mapper.py	Adds policy-violation details to approval request events sent to the UI.
python/packages/devui/agent_framework_devui/_executor.py	Propagates policy-violation metadata through approval responses.
python/packages/core/agent_framework/_tools.py	Adds policy-approval plumbing and placeholder replacement around tool approval flows.
python/packages/core/agent_framework/_security_tools.py	Implements quarantine/inspection tools and tool-use instructions for hidden content.
python/packages/core/agent_framework/_security_middleware.py	Implements label tracking, variable hiding, and policy enforcement middleware.
python/packages/core/agent_framework/_security.py	Adds label types, label combination, variable store, and lineage/message labeling primitives.
python/packages/core/agent_framework/init.py	Exposes security APIs and adds ai_function alias.
docs/decisions/0011-prompt-injection-defense.md	ADR describing the FIDES design and rationale.
QUICK_START_FIDES.md	Quick-start guide for configuring and using FIDES.
FIDES_IMPLEMENTATION_SUMMARY.md	High-level implementation summary of FIDES components and deliverables.
FIDES_DEVELOPER_GUIDE.md	Full developer guide for FIDES concepts, APIs, best practices, and examples.

[Eduard van Valkenburg (eavanvalkenburg)]

the code overall is looking good, I would like to see all _security*.py files folded into the _security.py file, that is more in line with the rest of the repo.

[Evan Mattson (moonbox3)]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework
_security.py	701	193	72%	420–421, 439, 442–443, 518–519, 521, 570, 581, 596, 606, 673–674, 677, 681, 688–689, 691, 698–701, 703, 706–713, 715, 718–721, 723–725, 727–728, 731–732, 734–735, 738–739, 741–745, 748–752, 754, 1012, 1016–1017, 1020–1023, 1029–1030, 1060–1061, 1087–1095, 1116, 1219, 1223, 1254–1255, 1281–1282, 1286–1287, 1321–1324, 1406–1407, 1412–1417, 1422–1424, 1455, 1669, 1674–1675, 1679, 1683–1685, 1727, 1731–1732, 1734, 1738, 1742, 1745, 1748, 1754, 1761, 1771–1772, 1787, 1821, 1825–1827, 1830, 1834, 1837, 1840, 1846, 1851, 1861–1862, 1901, 1942–1943, 1968, 2064, 2092–2094, 2129–2131, 2139, 2147, 2370, 2402–2403, 2405, 2407–2409, 2412, 2418, 2427, 2433–2434, 2436, 2467, 2471–2474, 2476, 2621–2624, 2627–2628, 2632, 2634, 2636, 2639–2643, 2650, 2655, 2666–2667, 2669, 2671–2673, 2743, 2753–2754
_tools.py	980	93	90%	190–191, 364, 366, 379, 404–406, 414, 432, 446, 453, 460, 483, 485, 492, 500, 539, 583, 587, 619–621, 623, 629, 674–676, 678, 701, 727, 731, 769–771, 775, 797, 909–915, 951, 963, 965, 967, 970–973, 994, 998, 1002, 1016–1018, 1362, 1372, 1391, 1456, 1476, 1494–1500, 1629, 1633, 1679, 1740–1741, 1853, 1878–1879, 1900, 1920, 1922, 1978, 2041, 2213–2214, 2234, 2290–2291, 2429–2430, 2497, 2502, 2509
TOTAL	28414	3399	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
5708	20 💤	0 ❌	0 🔥	1m 29s ⏱️