v0.3.0 — YAML crusher, categorical drop summaries

Three features, all live-validated against a real Kubernetes triage agent (forge + gpt-5.4 / claude-sonnet-4-6).

YAML / describe crusher

New crush.YAMLCrusher + detect.YAMLLike content type: kubectl get -o yaml, kubectl describe, and config dumps now compress structurally instead of falling through to the log rule. Dependency-free indentation-tree parsing (tolerant of describe's near-YAML), deepest-first subtree folding — inner bulky blocks (env lists, managedFields, tolerations) fold before parents; a parent's residue only folds if still bulky — plus long-scalar-value folding (last-applied-configuration style). Each fold gets its own marker, so the model expands exactly the subtree it needs. The fidelity floor applies per subtree: a crashing container's section carries error vocabulary and survives whole, with zero k8s-specific knowledge.

Measured live: kubectl describe pod 75% (was 12–20% under the log rule), kubectl get -o yaml 49%.

Categorical drop summaries

Markers and sentinels now say what was dropped, not just how much:

• JSON records: <<ctxzip:HASH 142_records_offloaded>> [offloaded: 96 Running, 31 Succeeded, 15 other] — grouped by status-like fields (status/phase/reason/…, one nested probe deep).
• Text/log lines: [offloaded: 67 running ago), 21 running] — top line-signature groups.

The model can answer count/distribution questions from the marker alone instead of expanding, and knows when expansion is warranted.

crush.IsErrorLike export

The error-vocabulary floor is now callable by hosts, so integration layers (like forge's keep_patterns learning loop) can exclude floor-kept terms from pattern suggestions instead of duplicating the vocabulary.

Compatibility

No breaking API changes. Marker format, store contract, and deterministic-compression guarantees unchanged — prompt-cache prefix stability verified live (cache_creation: 0 re-writes across a 4-call anthropic invocation).

v0.2.0 — JSON object walker: whole-record compression for items[] shapes

compressEnvelope generalizes into a depth-limited JSON object walker: large string fields decode and recurse (runner envelopes whose stdout holds a JSON object), large array fields get whole-record row selection (kubectl-style {"items": [...]} — anomalous records survive intact, the rest offload as a retrievable array of complete records), large object fields recurse with a valid-JSON splice guard. Measured on a 251KB kubectl get pods -o json envelope: 62,754 → 1,498 tokens (97.6%) with the CrashLoopBackOff pod record kept whole.

v0.1.2 — truncated-envelope salvage

Salvages JSON envelopes cut mid-string by upstream size caps: the intact prefix is recovered, compressed via the normal routing path, and re-emitted as valid JSON with a _ctxzip_note telling the model the tail was destroyed upstream (not offloaded — re-run the tool). Ambiguous cuts (outside a string value) still fail open. Found live: a 108KB kubectl -o json envelope was hard-truncated by the runtime before compression, breaking the JSON and defeating the envelope path entirely.

v0.1.1 — JSON-envelope compression

Tool runners commonly wrap output in a single-line JSON envelope ({"stdout": "...", ...}) whose escaped newlines defeated every content detector — 28-37KB kubectl outputs passed through uncompressed. v0.1.1 compresses large string fields inside JSON-object envelopes via the normal routing path (valid JSON out, untouched fields byte-identical, deterministic), collapses digit-bearing identifier tokens in line dedup so infrastructure tables actually crush, and adds 'warn' to the never-drop error floor (kubectl TYPE=Warning events). Measured on the live shape: 7,261 → 95 tokens (98.7%) with the CrashLoopBackOff row kept verbatim.

v0.1.0 — reversible context compression engine

First release of ctxzip — reversible context compression for AI agents. Lossy on the wire, lossless end-to-end: every chunk dropped is hashed, stored locally, and replaced by an inline <<ctxzip:HASH ...>> marker the model can expand on demand.

Highlights

• Content-aware crushers — JSONCrusher (row selection + dedup), LogCrusher (line selection), TextCrusher (line-mode numeric-insensitive dedup for grep/log output; extractive BM25 sentence selection for prose — verbatim only, never paraphrased)
• CCR reversibility — ccr.Store interface with in-memory and durable bbolt backends; originals survive restarts; 12-hex content-addressed markers (short enough that LLMs transcribe them reliably); marker hash == store key by construction
• Fidelity floor — errors/anomalies/query anchors never dropped; built-in error vocabulary includes k8s state words (crash, backoff, oomkilled, evicted); callers extend it via Options.MustKeep (union semantics — can only add protection)
• Prompt-cache discipline — frozen prefix + protected recent turns forwarded byte-identical; fully deterministic output; Options.SkipNames exempts a host's expansion tool from re-compression
• Pure Go — single third-party dependency (bbolt); no cgo, no ML runtime
• cmd/ctxzip-demo — pipe any file/output through and watch the compress → store → retrieve round trip

Field-tested

Built and hardened against a live forge agent: 5.6 KB grep output compressed 1,397 → 51 tokens (96%) with the one crashing pod kept verbatim, full original retrievable by marker hash. Several v0.1.0 behaviors (line-mode splitting, stop-term filtering, 12-hex hashes, SkipNames) exist because live testing found the failure modes.

Use

go get github.com/initializ/ctxzip@v0.1.0

See the README for the full options reference and the three-seam runtime embedding guide.

Compatibility: v0.x — the API may change between minor versions while live-driven hardening continues.