Skip to content

Add PyRIT agent red-team pipeline - #310

Draft
liamcrumm wants to merge 5 commits into
responsibleai:mainfrom
liamcrumm:liamcrumm/redteam-pipeline
Draft

Add PyRIT agent red-team pipeline#310
liamcrumm wants to merge 5 commits into
responsibleai:mainfrom
liamcrumm:liamcrumm/redteam-pipeline

Conversation

@liamcrumm

@liamcrumm liamcrumm commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add pipeline.red_team, backed by exact-pinned pyrit==1.0.1, as an ASSERT-owned attack, execution, scoring, and viewer path.
  • Reuse existing model, callable, endpoint, and connector target sessions. Plans that declare outbound sinks require a traced callable or hosted model with a real tool-module backend.
  • Emit normal taxonomy.json, test_set.jsonl, inference_set.jsonl, scores.jsonl, node judgments, native result summaries, and viewer metadata.
  • Add a secretless tool-using agent example where PyRIT misses a tool-argument disclosure in the final response and ASSERT captures it from the trajectory.
  • Add a manual least-privilege workflow with SHA-pinned actions.
  • Make the dormant DSPy GEPA extra resolvable on current PyPI and declare its uv conflicts with the examples bundles.

Validation

  • 1146 passed, 74 skipped, 481 subtests passed
  • npm run check --prefix viewer
  • npm run build --prefix viewer
  • uv lock --check
  • python -m build and twine check dist/*
  • Built-wheel smoke without the redteam extra
  • Secretless run with only the redteam extra
  • Two real PyRIT runs in one Python process
  • Provider input refusal, auth failure, partial failure, and target error repros
  • Rebuild from the run-local copied config and attack-data snapshot
  • End-to-end example result: PyRIT final-response score false, ASSERT trajectory policy_violation: true, evidence post_diagnostic.message

Current scope

  • Supports PyRIT PromptSendingAttack with Foundry-compatible Baseline and risk-category labels.
  • Uses one impermissible risk category, deterministic substring scoring, and explicit outbound sink declarations.
  • Adaptive multi-turn attacks, semantic scoring, and live hosted or managed-agent validation remain follow-up work.

Add an ASSERT-owned red-team stage that runs PyRIT attacks through existing target sessions, emits native findings and viewer metadata, and ships a secretless on-demand workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e7836021-2bfc-4b51-a539-f26ccdc02830
Fail closed on incomplete tool evidence, preserve typed target failures, mark overrefusal not applicable, and complete viewer and changelog integration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e7836021-2bfc-4b51-a539-f26ccdc02830
Keep proven sink findings when later tool calls differ, fail incomplete evidence symmetrically, short-circuit auth failures, and make copied run configs self-contained.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e7836021-2bfc-4b51-a539-f26ccdc02830
Serialize the first attack path, preserve source config identity, snapshot attack data after preflight, and emit side-effect-free viewer rebuild commands.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e7836021-2bfc-4b51-a539-f26ccdc02830
Preserve pinned run configs across failed preflight checks and ignore inert concurrency when deciding whether cached attacks must rerun.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e7836021-2bfc-4b51-a539-f26ccdc02830
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant