Skip to content

fix(sensor): extract custom_tool_call tool usage from Codex sessions - #34

Merged
pengyuzhang merged 1 commit into
mainfrom
fix/codex-custom-tool-call
Aug 10, 2026
Merged

fix(sensor): extract custom_tool_call tool usage from Codex sessions#34
pengyuzhang merged 1 commit into
mainfrom
fix/codex-custom-tool-call

Conversation

@pengyuzhang

@pengyuzhang pengyuzhang commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Problem

Codex Desktop emits agent tool invocations as custom_tool_call / custom_tool_call_output response items. CodexParser only recognised function_call / function_call_output, so those records were dropped.

The failure was silent rather than loud: affected sessions parsed successfully, has_meaningful_content() returned True, and the event exported with its conversation text intact — while every tool call in it was discarded. A session that ran five exec calls was indistinguishable from a plain chat.

For a sensor whose detections key on tool arguments (credential exfiltration, data movement), losing the command string loses the entire signal.

Root cause

Three shape differences that have to be handled together — fixing only the first yields tools with empty arguments, which is arguably worse than dropping them because it looks like coverage:

function_call custom_tool_call
Record type function_call custom_tool_call
Arguments JSON string under arguments raw, frequently non-JSON string under input
Output plain string list of {type, text} content items

The list-shaped output also bypassed the isinstance(output, str) truncation guard and placed a list into ToolUsage.result, which is typed Optional[str].

Changes

  • Match custom_tool_call / custom_tool_call_output alongside the existing types
  • New _parse_tool_arguments() — coerces the argument payload to a dict, falling back to {"raw": ...} so a non-JSON command string is preserved rather than discarded
  • New _normalize_tool_output() — flattens a list of content items to text before truncating, so list outputs are capped like every other result
  • Take tool status from the record instead of hardcoding pendingsuccess
  • Record the response-item type as tool_type so downstream rules can distinguish agent tool calls from classic function calls

No schema change. No new source. function_call behaviour is unchanged and pinned by a regression test.

Verification

Real dataadr-sensor --source codex end-to-end over 16 local sessions:

before after
Tool usages extracted 211 246

+35 recovered (custom_tool_call: 35, function_call: 211); 243 of 246 carry non-empty arguments. The 3 without are legitimately argument-less (list_mcp_resources, list_mcp_resource_templates, list_available_plugins_to_install).

Unit — 9 new tests in TestCodexParser, fixtures synthetic per CONTRIBUTING but every field shape copied from real session records:

  • real shape (raw input, list output) → name, args, joined result, status
  • JSON input parses to a dict rather than {"raw"}
  • large list output is normalised and truncated
  • classic function_call unchanged (regression guard)
  • both record shapes in one session
  • orphaned call (no output) keeps args, stays pending
  • stray output with unknown call_id invents no tool
  • output list containing non-dict / text-less items degrades gracefully
  • event_msg records skipped cleanly (currently unparsed — see below)

Suite: 114 → 123 passing. ruff check clean on the changed module.

Not addressed here

Found while investigating, left for separate PRs:

  • event_msg records are unparsed — they carry web_search_end (query + call_id, an egress signal), token_count (total_token_usage, model_context_window), and user_message.images
  • CodexParser is the only parser that ignores max_age_days, so it globs every session ever written and --all-history is a no-op for it
  • ~/.codex/archived_sessions/ is a sibling of sessions/ and falls outside the glob

Codex Desktop emits agent tool invocations as `custom_tool_call` /
`custom_tool_call_output` response items, which CodexParser did not
recognise. Affected sessions parsed successfully and exported as healthy
events while every tool call in them was silently discarded, so a session
that ran five `exec` calls was indistinguishable from a plain chat.

Three shape differences had to be handled together:

- record type: `custom_tool_call` is now matched alongside `function_call`
- arguments: carried as a raw, frequently non-JSON string under `input`
  rather than a JSON string under `arguments`. Matching only on the type
  would have yielded tools with empty arguments, which for a shell
  execution tool discards the entire signal
- output: a list of `{type, text}` content items rather than a plain
  string, so it bypassed truncation and placed a list in a str field

Tool status is now taken from the record instead of being hardcoded, and
the response-item type is recorded as `tool_type` so downstream rules can
distinguish agent tool calls from classic function calls.

Verified against a real Codex Desktop session: 5 tools recovered with
arguments and results intact, previously 0.
@pengyuzhang
pengyuzhang force-pushed the fix/codex-custom-tool-call branch from 78b175d to a064784 Compare August 9, 2026 20:43

@barisozbas barisozbas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pengyuzhang
pengyuzhang merged commit d0e62f9 into main Aug 10, 2026
8 checks passed
@soltan2000hn-web

Copy link
Copy Markdown

thenk you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants