feat: implement deterministic TELL v1 auditor - #1
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b532c48ffe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| func audit(ctx context.Context, options options, stdout, stderr io.Writer) int { | ||
| seed, err := readSeed(options.seedPath) |
There was a problem hiding this comment.
Make seed loading responsive to cancellation
When --seed names a FIFO, device, or another source whose open/read blocks, cancellation cannot reach this synchronous readSeed call. Because main uses signal.NotifyContext, SIGINT and SIGTERM have their default termination behavior suppressed, so the CLI can remain stuck indefinitely even after either signal and require SIGKILL. Seed opening and reading need to observe ctx or otherwise restore interruptibility during setup.
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| func operationalError(stderr io.Writer, err error) int { | ||
| _, _ = fmt.Fprintf(stderr, "tell: %v\n", err) |
There was a problem hiding this comment.
Escape control characters in operational diagnostics
If a user-controlled seed, report, or executable path contains a newline or terminal escape and the corresponding operation fails, wrapped errors such as *os.PathError retain those bytes and %v writes them directly to stderr. For example, a missing seed named missing\x1b[31m\nfile emits the escape sequence and an extra diagnostic line, violating the stated no-control-sequences behavior and allowing terminal/log injection; sanitize error text before printing, as is already done for summary paths.
Useful? React with 👍 / 👎.
Summary
Implements the complete bounded TELL v1 product: a Linux-first black-box failure-oracle auditor that sends a known-valid binary seed and the fixed
tell-default-v1mutation set to a literal target argv over stdin, then publishes deterministictell-report-v1JSON.This change adds the CLI, streaming mutation engine, exact observation equivalence classes, Linux process supervision, atomic private reports, actual Go-test-binary subprocess integration coverage, normative specification, user documentation, build tooling, and pinned CI. It also replaces the repository's original AGPL text with the requested MIT License.
Exact v1 contract
An audit passes if and only if:
0.(exit code, exact stdout bytes, exact stderr bytes)observation.Completed PASS/FAIL audits return
0/1. Usage, setup, invalid baseline, interruption, launch, capture, cleanup, report-write, unsupported-platform, and internal errors return2without publishing a new report. Mutations execute sequentially and continue after ordinary audit failures; infrastructure errors abort.Determinism and reporting
tell-default-v1order with independent candidates, unchanged filtering, collision-safe exact deduplication, fixed case IDs, and callback-scoped candidate storage.obs-<sha256>IDs over the specified domain-separated binary preimage and collision refusal.0600, syncs and closes them, checks interruption throughout publication, and renames only at the final atomic publication point.Process and security model
os/exec; no shell, command strings, expansion, interpolation, globbing, or merged streams.SIGKILL, oneWait, post-Waitgroup cleanup, and a fixed 250 ms inherited-pipe cleanup bound.137from naturalSIGKILL, treats valid-result stdinEPIPEas nonfatal, and handles SIGINT/SIGTERM as incomplete audits.TELL is not a sandbox. Targets run with the caller's privileges and may have filesystem or network side effects. A deliberately hostile descendant may escape process-group cleanup with
setsidorsetpgid. Reports may expose sensitive argv and output and must be reviewed before publication.Local validation
Go 1.26.5, Linux amd64:
Every command passed.
go list -m allprinted onlygithub.com/chasebryan/TELL;go mod tidycreated nogo.sum;/tmp/tell-a versionprintedtell 1.0.0. Both production binaries have SHA-2567619a090fe38a3abe1940dba0267c72f959c3a86bf29f44fa56445bee06f6481. Windows, Darwin, and FreeBSD cross-build checks also pass through the unsupported-platform implementation. Runner coverage was inspected; remaining uncovered branches are defensive pipe-creation/close/kill and unknown-wait-status error paths, while the live timeout, overflow, cancellation, EPIPE, post-Waitgroup kill, escaped-pipe, and cleanup paths use actual subprocesses.Limitations and nonclaims
v1 executes each input once. Target nondeterminism or environment-dependent output can create apparent differences.
TELL does not prove:
No random/grammar-aware/coverage-guided fuzzing, parallelism, timing analysis, minimization, corpus management, sandboxing, plugins, public SDK, telemetry, server mode, or additional report formats are included.