feat(codex): implement F103 NDJSON output parity for Codex provider#364
Merged
Merged
Conversation
b88543f to
238087f
Compare
- `.zpm/kb/pr_feature_f103_codex_provider_jsonl_output_parity/journal.wal`: Add ZPM PR tracking journal - `.zpm/kb/pr_feature_f103_codex_provider_jsonl_output_parity/knowledge.pl`: Add ZPM PR knowledge base schema - `CHANGELOG.md`: Document F103 presence-aware output extraction and Response population - `README.md`: Update output formatting feature description to include Response auto-population - `docs/reference/interpolation.md`: Expand Response section with agent step examples and JSON/Response comparison - `docs/user-guide/agent-steps.md`: Update Output/Response field descriptions and output_format table for F103 - `internal/infrastructure/agents/codex_provider.go`: Add extractCodexAssistantText/extractCodexTextContent helpers; wire extractTextContent hook; implement presence-aware Output overwrite and Response population in Execute and ExecuteConversation - `internal/infrastructure/agents/codex_provider_delegation_test.go`: Remove stub-era conditionals; add OutputExtraction, HookWired, PlainTextFallback, NDJSONOutput delegation tests - `internal/infrastructure/agents/codex_provider_test_helpers_test.go`: Add ndjsonLine test helper - `internal/infrastructure/agents/codex_provider_unit_test.go`: Expand unit test coverage for F103 extraction paths; remove duplicated ProviderName/TimestampOrdering tests - `internal/infrastructure/agents/helpers.go`: Remove stale comment from appendCodexOptions Closes #363
238087f to
e6de4ad
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
state.OutputandConversationResult.Outputnow contain clean aggregated assistant text extracted from NDJSON streams instead of raw JSON envelopes, regardless ofoutput_formatvalue — matching the behavior Claude, Gemini, and OpenCode already had since 0.7.1.assistant_messageevent (hadText=true, text="") is correctly distinguished from non-NDJSON plain-text fallback (hadText=false), preventing inflated token estimates and stale turn Content when the model responds with empty output.state.Responsefor Codex viatryParseJSONResponseon the extracted assistant text, giving Codex the same automatic structured-output heuristic already present in other providers and enabling{{.states.step.Response.field}}in downstream workflow steps.state.Outputstores extracted text (not raw NDJSON) for all CLI providers and thatstate.Responseis now populated heuristically across all six providers.Changes
Infrastructure — Codex Provider
internal/infrastructure/agents/codex_provider.go: WireextractCodexTextContenthook intocliProviderHooks; addextractCodexAssistantText(presence-aware NDJSON aggregator returning(string, bool)); addextractCodexTextContentadapter for the base hook signature; overrideExecuteandExecuteConversationwith F103 presence-aware overwrite ofOutput,Response, token counts, and trailing turnContent; simplifyNewCodexProviderto delegate toNewCodexProviderWithOptions; document NUL-byte preservation divergence from Claude provider.internal/infrastructure/agents/helpers.go: MigratefindFirstNDJSONEventandfindLastNDJSONEventfromstrings.Splittostrings.SplitSeq(allocation-free iterator).Tests
internal/infrastructure/agents/codex_provider_unit_test.go: Add ~450 lines of new unit tests covering all F103 paths: presence-aware overwrite,output_format-invariant extraction,Responsepopulation, emptyassistant_message, multi-event aggregation, NUL byte preservation, non-JSON plain text, and token recount; remove superseded timestamp/provider-name duplicate tests and stub-eraexpectedTokenshardcoding.internal/infrastructure/agents/codex_provider_delegation_test.go: Remove all stub-era conditionalif err != nil { return }guards; harden assertions withrequire.NoError; addTestCodexProvider_ExecuteConversation_OutputExtraction,HookWired,PlainTextFallback, andNDJSONOutputdelegation tests; asserterrors.Is(err, context.Canceled)on cancellation paths; remove no-opTestCodexProvider_Validate_BinaryNotFound.internal/infrastructure/agents/codex_provider_test_helpers_test.go: New file — sharedndjsonLinehelper that JSON-escapes arbitrary text into a valid Codexitem.completedNDJSON envelope.Documentation
docs/reference/interpolation.md: ExpandResponsesection to cover agent steps with examples showing{{.states.step.Response.field}}; clarifyJSONvsResponsesemantics and when each is populated.docs/user-guide/agent-steps.md: Updatestate.Outputdescription to reflect extracted-text semantics for CLI providers; add F103 Output Parity note to the display-filtering table; correct "raw NDJSON" references in silent-mode note.README.md: Update output-formatting feature bullet to mention automaticstate.Responsepopulation.CHANGELOG.md: Add [Unreleased] entry for F103 with full parity description.ZPM Knowledge Base
.zpm/kb/pr_feature_f103_codex_provider_jsonl_output_parity/journal.wal: PR tracking journal..zpm/kb/pr_feature_f103_codex_provider_jsonl_output_parity/knowledge.pl: PR Prolog schema for blocking-issue tracking.Test plan
make test— all unit and integration tests pass with zero failures and no data races (make test-race)make lint— zero violations; verifynolint:errcheckdirectives carry explanatory comments{{.states.codex_step.Output}}into a downstream step and confirm clean text (not raw NDJSON) is interpolated{{.states.codex_step.Response.field}}resolves correctly without requiringoutput_format: jsonCloses #363
Generated with awf commit workflow