Skip to content

agent classifier: stale tool registry flags current Claude Code tools (Glob/TaskOutput) as unknown/suspicious #85

Description

@vaderyang

Summary

heron's agent classifier flags current Claude Code tools (e.g. Glob, TaskOutput) as SuspiciousSignal::UnknownToolName, and when every tool in a call is unrecognized the call's tool_surface becomes Unknown. The cause is a stale, tiny hardcoded tool registry — not anything actually suspicious in the captured traffic.

In the console this shows up as legitimate agent activity tagged "suspicious / unknown_tool_name". It also drives AgentClassifierUnknownCount — the same Unknown-surface path that triggered the prod worker panic in #81.

Root cause

server/h-llm/src/agent_classifier.rs:18-44ClassifierConfig::default() seeds only:

cli_tool_allowlist:   ["bash", "shell", "exec", "run_command", "ShellExec"]
known_tool_registry:  ["Read", "Edit", "Write", "Grep", "Bash", "Task", "WebFetch", "WebSearch"]

Eight tools. Current Claude Code (and Codex CLI) expose far more, so anything outside that list falls through to:

} else {
    suspicious.push(SuspiciousSignal::UnknownToolName { name: name.clone() });  // agent_classifier.rs:91
}

Missing common Claude Code tools include at least: Glob, TodoWrite, MultiEdit, NotebookEdit, BashOutput, KillShell, ExitPlanMode, SlashCommand, and the task-management family (TaskOutput, TaskCreate, TaskGet, TaskList, TaskStop, TaskUpdate). (prod runs the default seed — its config.toml has no [classifier] override.)

Proposed fix

  1. Expand known_tool_registry (and cli_tool_allowlist where appropriate) in the Default seed to the current Claude Code + Codex CLI tool inventory. Keep the list alphabetized and commented with the source CLI so it's maintainable.
  2. (Consider) whether a genuinely-unknown tool name should be Suspicious at all, or just Unknown — a new/unrecognized tool isn't inherently suspicious. At minimum, recognized-but-new tools shouldn't trip it. Out of scope if contentious; the registry expansion is the core fix.

Acceptance criteria

  • A call using Glob (and TaskOutput, TodoWrite, etc.) classifies with a non-Unknown tool_surface and produces no UnknownToolName signal.
  • A deterministic test exercises the current Claude Code tool set and asserts none are flagged unknown.
  • cargo test --workspace green.

Severity

Low/medium — not a crash (post-#82), but it pollutes the console with false "suspicious" signals and inflates the Unknown classification rate for normal agent traffic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions