Skip to content

claude-code-chat-browser: Adversarial predicate-overlap fixtures for tool dispatch classification #116

Description

@clean6378-max-it

Calendar Day

Wednesday, July 8, 2026 (PR 2 of 2)

Planned Effort

3 story points (Medium–High) — sprint item #4

Companion PR: Wednesday PR 1 (renderer tests #3) — independent files.

Blocks: Thursday PR 2 (dispatch simplification #6) — adversarial fixtures must land before refactor.

Problem

utils/tool_dispatch.py classifies tool results through _parse_tool_result using _TOOL_RESULT_DISPATCH — a 15-entry ordered (predicate, builder) tuple where first matching predicate wins and order is load-bearing.

tests/test_tool_dispatch_ordering.py (#95 / PR #98) only checks ordering structurally via _predicate_index for 4 declared ORDERING_INVARIANTS. It does not construct JSON blobs that satisfy multiple predicates simultaneously and assert the actual classified winner.

Week 1 PR #104 added KNOWN_TOOL_TYPES and test_tool_dispatch_sync.py but not behavioral overlap fixtures. A new upstream tool shape matching a broad predicate before a narrower handler would misclassify silently while existing tests still pass.

Relationship to prior work (not a duplicate)

Prior Status What it covers What it does not cover
#95 Predicate ordering structural enforcement Closed Declarative ORDERING_INVARIANTS; index(A) < index(B) in test_tool_dispatch_ordering.py Adversarial JSON; _parse_tool_result winner; result_type assertions
PR #98 test: enforce tool_dispatch predicate ordering invariants Merged Implementation of #95 — structural tuple-position guards Behavioral overlap fixtures
PR #104 KNOWN_TOOL_TYPES registry + sync test Merged Tool-name set parity across backend/frontend (test_tool_dispatch_sync.py) Predicate overlap classification

This issue is the behavioral half that planning explicitly marked as residual after #95/#98 and #104. Keep test_tool_dispatch_ordering.py unchanged; add test_tool_dispatch_adversarial.py (or a behavioral section) alongside it.

Goal

One merged PR that adds adversarial toolUseResult fixtures asserting documented classification winners via _parse_tool_result, complementing (not replacing) the structural ordering tests.

Scope

Touch points

  • tests/test_tool_dispatch_adversarial.py (new) — or extend tests/test_tool_dispatch_ordering.py with a behavioral section.
  • Build adversarial JSON shapes satisfying multiple predicates; assert result_type via _parse_tool_result.
  • Cover every pair in ORDERING_INVARIANTS:
    • is_plan_tool_result vs is_file_write_tool_result (filePath + content).
    • is_task_message_tool_result vs narrower is_task_retrieval_tool_result / is_task_completed_tool_result / is_task_async_tool_result.
  • Regression case: demonstrate reordering a predicate flips classification and fails the new test.
  • Optional: hypothesis-based generation (pattern from tests/test_parser_fuzz.py).

Out of scope

Acceptance Criteria

  • Adversarial fixtures for every ORDERING_INVARIANTS pair in test_tool_dispatch_ordering.py.
  • Behavioral classification via _parse_tool_result, not tuple index comparison.
  • Boundary shapes distinguish broad is_task_message_tool_result from narrower task predicates.
  • At least one regression case proves reordering a predicate causes test failure.
  • Existing structural tests in test_tool_dispatch_ordering.py preserved.
  • Full pytest, mypy --strict, and ruff pass.
  • PR approved by at least 1 reviewer.

Verification

cd C:\Users\Jasen\CppAliance\claude-code-chat-browser
.\.venv\Scripts\Activate.ps1
pytest tests/test_tool_dispatch_adversarial.py tests/test_tool_dispatch_ordering.py -v
pytest tests/test_tool_dispatch_sync.py tests/test_real_session_fixtures.py -q
pytest -q
mypy .
ruff check .

Manual: temporarily swap two predicates in _TOOL_RESULT_DISPATCH — confirm adversarial test fails with a clear message, then revert.

References

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions