Skip to content

feat: implement deterministic TELL v1 auditor - #1

Merged
chasebryan merged 1 commit into
mainfrom
codex/tell-v1
Jul 26, 2026
Merged

feat: implement deterministic TELL v1 auditor#1
chasebryan merged 1 commit into
mainfrom
codex/tell-v1

Conversation

@chasebryan

Copy link
Copy Markdown
Owner

Summary

Implements the complete bounded TELL v1 product: a Linux-first black-box failure-oracle auditor that sends a known-valid binary seed and the fixed tell-default-v1 mutation set to a literal target argv over stdin, then publishes deterministic tell-report-v1 JSON.

This change adds the CLI, streaming mutation engine, exact observation equivalence classes, Linux process supervision, atomic private reports, actual Go-test-binary subprocess integration coverage, normative specification, user documentation, build tooling, and pinned CI. It also replaces the repository's original AGPL text with the requested MIT License.

Exact v1 contract

tell run --seed PATH --stdin [--timeout 2s] [--max-output-bytes 65536] [--report tell-report-v1.json] -- COMMAND [ARG...]
tell version
tell help
tell run --help

An audit passes if and only if:

  1. The baseline seed exits normally with code 0.
  2. At least one unique mutation different from the seed executes.
  3. Every mutation exits normally with a nonzero code.
  4. Every mutation has exactly the same (exit code, exact stdout bytes, exact stderr bytes) observation.

Completed PASS/FAIL audits return 0/1. Usage, setup, invalid baseline, interruption, launch, capture, cleanup, report-write, unsupported-platform, and internal errors return 2 without publishing a new report. Mutations execute sequentially and continue after ordinary audit failures; infrastructure errors abort.

Determinism and reporting

  • Implements the exact maximum-29-case tell-default-v1 order with independent candidates, unchanged filtering, collision-safe exact deduplication, fixed case IDs, and callback-scoped candidate storage.
  • Establishes rejection membership by exact bytes, with obs-<sha256> IDs over the specified domain-separated binary preimage and collision refusal.
  • Emits fixed-order, two-space-indented, HTML-unescaped JSON with required nulls, padded Base64, lowercase SHA-256, and exactly one final newline.
  • Writes same-directory temporary reports at 0600, syncs and closes them, checks interruption throughout publication, and renames only at the final atomic publication point.
  • Excludes input contents and machine/runtime metadata; includes the exact command vector and raw target output encodings.

Process and security model

  • Executes the supplied argv directly with os/exec; no shell, command strings, expansion, interpolation, globbing, or merged streams.
  • Uses a new Linux process group, explicit stdin/stdout/stderr pipes, concurrent bounded drains, exact cap+1 overflow detection, first-cause-wins termination, negative-PID SIGKILL, one Wait, post-Wait group cleanup, and a fixed 250 ms inherited-pipe cleanup bound.
  • Distinguishes normal exit 137 from natural SIGKILL, treats valid-result stdin EPIPE as nonfatal, and handles SIGINT/SIGTERM as incomplete audits.
  • Provides a compiling non-Linux implementation that returns a clear unsupported-platform error.

TELL is not a sandbox. Targets run with the caller's privileges and may have filesystem or network side effects. A deliberately hostile descendant may escape process-group cleanup with setsid or setpgid. Reports may expose sensitive argv and output and must be reviewed before publication.

Local validation

Go 1.26.5, Linux amd64:

test -z "$(gofmt -l .)"
go mod tidy
go vet ./...
CGO_ENABLED=0 go test -count=1 ./...
go test -race -count=1 ./...
go test -count=3 ./...
go list -m all
git diff --check

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
  go build -trimpath -buildvcs=false -ldflags="-buildid=" \
  -o /tmp/tell-a ./cmd/tell

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
  go build -trimpath -buildvcs=false -ldflags="-buildid=" \
  -o /tmp/tell-b ./cmd/tell

cmp /tmp/tell-a /tmp/tell-b
/tmp/tell-a version

Every command passed. go list -m all printed only github.com/chasebryan/TELL; go mod tidy created no go.sum; /tmp/tell-a version printed tell 1.0.0. Both production binaries have SHA-256 7619a090fe38a3abe1940dba0267c72f959c3a86bf29f44fa56445bee06f6481. Windows, Darwin, and FreeBSD cross-build checks also pass through the unsupported-platform implementation. Runner coverage was inspected; remaining uncovered branches are defensive pipe-creation/close/kill and unknown-wait-status error paths, while the live timeout, overflow, cancellation, EPIPE, post-Wait group kill, escaped-pipe, and cleanup paths use actual subprocesses.

Limitations and nonclaims

v1 executes each input once. Target nondeterminism or environment-dependent output can create apparent differences.

TELL does not prove:

  • that generated candidates are actually malformed;
  • that all malformed inputs are rejected;
  • constant-time behavior or absence of side channels;
  • exploitability or vulnerability severity;
  • memory safety;
  • cryptographic correctness;
  • complete input-space or fuzz coverage; or
  • general security.

No random/grammar-aware/coverage-guided fuzzing, parallelism, timing analysis, minimization, corpus management, sandboxing, plugins, public SDK, telemetry, server mode, or additional report formats are included.

@chasebryan
chasebryan marked this pull request as ready for review July 26, 2026 23:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b532c48ffe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/app/audit.go
}

func audit(ctx context.Context, options options, stdout, stderr io.Writer) int {
seed, err := readSeed(options.seedPath)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make seed loading responsive to cancellation

When --seed names a FIFO, device, or another source whose open/read blocks, cancellation cannot reach this synchronous readSeed call. Because main uses signal.NotifyContext, SIGINT and SIGTERM have their default termination behavior suppressed, so the CLI can remain stuck indefinitely even after either signal and require SIGKILL. Seed opening and reading need to observe ctx or otherwise restore interruptibility during setup.

Useful? React with 👍 / 👎.

Comment thread internal/app/audit.go
}

func operationalError(stderr io.Writer, err error) int {
_, _ = fmt.Fprintf(stderr, "tell: %v\n", err)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Escape control characters in operational diagnostics

If a user-controlled seed, report, or executable path contains a newline or terminal escape and the corresponding operation fails, wrapped errors such as *os.PathError retain those bytes and %v writes them directly to stderr. For example, a missing seed named missing\x1b[31m\nfile emits the escape sequence and an extra diagnostic line, violating the stated no-control-sequences behavior and allowing terminal/log injection; sanitize error text before printing, as is already done for summary paths.

Useful? React with 👍 / 👎.

@chasebryan
chasebryan merged commit 3ed7642 into main Jul 26, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant