fix(transport): pass oversized system prompts via --system-prompt-file#1101
Open
yayayouyou wants to merge 2 commits into
Open
fix(transport): pass oversized system prompts via --system-prompt-file#1101yayayouyou wants to merge 2 commits into
yayayouyou wants to merge 2 commits into
Conversation
A str `system_prompt` of >=131072 UTF-8 bytes was passed as a single
`--system-prompt` argv element, which exceeds Linux MAX_ARG_STRLEN and makes
execve() fail with E2BIG ("Argument list too long") before the CLI starts.
Above the per-argument limit, write the prompt to a temp file and use the
already-supported `--system-prompt-file` flag instead; smaller prompts are
passed inline unchanged. The temp file is removed in close(), including the
failed-connect() path so it cannot leak.
Fixes anthropics#1096.
Used AI assistance; reviewed and tested by me (full unit suite green on both
asyncio and trio).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses Linux execve() failures (E2BIG / “Argument list too long”) when a large system_prompt string is passed as a single --system-prompt argv element, by spilling oversized prompts to a temporary file and passing them via --system-prompt-file.
Changes:
- Add a size check in
SubprocessCLITransport._build_command()to route oversized string system prompts through a temp file. - Track and delete the temp file during
close()to avoid leaking it on failed startup paths. - Add unit tests covering the oversized spill behavior, boundary behavior, and temp-file cleanup.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
src/claude_agent_sdk/_internal/transport/subprocess_cli.py |
Adds temp-file spillover for oversized system_prompt strings and removes the temp file in close(). |
tests/test_transport.py |
Adds tests validating spillover behavior and cleanup for oversized system prompts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+299
to
+308
| fd, path = tempfile.mkstemp(prefix="claude-system-prompt-", suffix=".txt") | ||
| try: | ||
| with os.fdopen(fd, "w", encoding="utf-8") as f: | ||
| f.write(system_prompt) | ||
| except BaseException: | ||
| with suppress(OSError): | ||
| Path(path).unlink() | ||
| raise | ||
| self._system_prompt_tmp = Path(path) | ||
| return path |
| elif isinstance(self._options.system_prompt, str): | ||
| cmd.extend(["--system-prompt", self._options.system_prompt]) | ||
| system_prompt = self._options.system_prompt | ||
| if len(system_prompt.encode("utf-8")) >= _MAX_SYSTEM_PROMPT_ARG_BYTES: |
Comment on lines
+34
to
+38
| # Linux caps a single argv element (MAX_ARG_STRLEN) at 32 * PAGE_SIZE = 131072 | ||
| # bytes, including the trailing NUL. A ``--system-prompt`` value at or above this | ||
| # makes execve() fail with E2BIG ("Argument list too long") before the CLI ever | ||
| # starts, so oversized prompts are written to a temp file and passed via | ||
| # ``--system-prompt-file`` instead (see issue #1096). |
Comment on lines
+188
to
+189
| """A system prompt over the argv limit goes via file, not inline (#1096).""" | ||
| big = "x" * 200_000 # exceeds MAX_ARG_STRLEN (131072 bytes) |
Comment on lines
+208
to
+209
| """A prompt just under the limit is still passed inline (#1096).""" | ||
| prompt = "x" * (131072 - 1) # 131071 bytes: largest that fits in one arg |
Comment on lines
+201
to
+205
| path = cmd[cmd.index("--system-prompt-file") + 1] | ||
| assert transport._system_prompt_tmp is not None | ||
| assert Path(path).read_text(encoding="utf-8") == big | ||
|
|
||
| Path(path).unlink() # cleanup (close() would normally remove this) |
- Use NamedTemporaryFile so the fd cannot leak if opening or writing the spill file fails, and remove the previous temp file when _build_command() runs more than once. - Measure the argv threshold with os.fsencode() to match how subprocess encodes argv for execve(), rather than assuming UTF-8. - Qualify the MAX_ARG_STRLEN comment: 32 * PAGE_SIZE is 131072 bytes only on the common 4 KiB page size, and the fixed threshold is conservative. - Derive test sizes from _MAX_SYSTEM_PROMPT_ARG_BYTES instead of hardcoding 131072, make temp-file cleanup in tests try/finally-safe, and add a test that rebuilding the command replaces the spill file without leaking the previous one. Used AI assistance; reviewed and tested by me (ruff, mypy, and the full unit suite green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1096. A
strsystem_promptof ≥131072 UTF-8 bytes was passed as a single--system-promptargv element, which exceeds Linux MAX_ARG_STRLEN (32 × PAGE_SIZE) and makes execve() fail with E2BIG ("Argument list too long") before the CLI ever starts.Fix
Above the per-argument limit, write the prompt to a temp file and pass it via the already-supported
--system-prompt-fileflag; smaller prompts stay inline (no behavior change for the common case). The temp file is removed inclose(), including the failed-connect()path so it can't leak. Reuses the flag the{"type": "file"}system-prompt path already relies on — small and self-contained.Testing
tests/test_transport.py: oversized prompt →--system-prompt-file, prompt just under the limit stays inline, andclose()removes the temp file.ruffandmypyclean.Note on #1097
#1097 documents the limit; this PR is the actual workaround. Happy to add a doc note here too, or defer to the docs-only route if you'd prefer — just say the word.
Disclosure
Drafted with AI assistance; I've reviewed and tested the change myself.