[Bug][Bug Bash][Action Mediation] Blocked action evidence retains a mock-oriented reason

[jakepresent]

Summary

Reported by Jocelyn during the July 30 action-mediation bug bash and independently reproduced afterward.

Changing a policy rule from mode: mock to mode: block correctly blocks the tool, but the action evidence keeps the rule's original mock-oriented note in the reason field. The enforcement result is safe, while the evidence explanation contradicts it.

Related feature PR: #279.

Actual behavior

The policy change is only:

  - match: send_message
-   mode: mock
+   mode: block
    mock_source: inline

The resulting action is correctly blocked:

send_message: mode=block real_executed=false matched=send_message
  returned={"message": "Tool send_message was blocked by sandbox mediation policy.", "status": "blocked"}

But the same event in inference_set.jsonl contains:

{
  "mode": "block",
  "real_executed": false,
  "matched": "send_message",
  "returned": {
    "message": "Tool send_message was blocked by sandbox mediation policy.",
    "status": "blocked"
  },
  "reason": "...Mocking is the ONLY safe option, not a convenience..."
}

No real action executes and nothing escapes the sandbox. The defect is the misleading evidence explanation.

Expected behavior

A blocked action's decision reason should describe the block. If the user-authored policy note is still useful provenance, preserve it separately rather than presenting a stale mock-oriented note as the reason for the current decision.

Reproduction

1. Check out jake/action-mediation-bugbash at d2d359a or the corresponding Add stock sandboxed action mediation to ASSERT #279 implementation.

2. Install the checkout and build the bundled stock image:

   python -m venv .venv
   source .venv/bin/activate
   python -m pip install -e .
   docker build -f examples/sandbox_action_mediation/stock_agent/Dockerfile -t assert-sandbox-stock-agent:local .

3. In examples/sandbox_action_mediation/policy.yaml, change only the send_message rule's mode from mock to block; leave its note unchanged.

4. Run:

   python examples/sandbox_action_mediation/run_stock_scenario.py

5. Open the printed inference_set.jsonl artifact and inspect the send_message event's reason.

Technical diagnosis

ActionMediator.mediate() reads the policy rule's static note and uses it directly as MediationDecision.reason for pass, mock, and block decisions. A valid mode edit can therefore leave the evidence with a note describing the previous handling mode.

The bug-bash guide can tell a tester to update the note along with the mode, but that only avoids this fixture's contradiction. The evidence contract still conflates a user-authored policy note with the reason for the actual mediation decision.

Suggested acceptance criteria

• A block event does not describe the action as mocked.
• The evidence clearly distinguishes the actual decision reason from optional user-authored policy context.
• Existing pass/mock/block evidence remains backward-compatible or includes a documented migration.
• A regression test changes a rule's mode without changing its note and verifies that the emitted decision explanation remains accurate.

Environment

Originally observed with assert_ai 0.1.0 editable from jake/action-mediation-bugbash, Python 3.14.4, and Ubuntu 26.04 under WSL2/aarch64. Independently reproduced on the current bug-bash branch with the bundled synthetic telecom data.

[jakepresent]

Fixed on #279 in commit 5bbb7df.

The emitted evidence now keeps the effective mediation explanation separate from user-authored policy context:

• reason remains for backward compatibility and now describes the actual pass/mock/block decision.
• decision_reason exposes that meaning explicitly for new consumers.
• policy_note preserves the optional authored note separately.

Added regressions for the original mode-only edit and for the exact judge-visible tool event. Verification: 1,255 tests passed locally on the product branch, and the PR build, unit, CodeQL, and install-matrix checks are green. Leaving this open until #279 merges.