feat(transform): adaptive chars-per-token for the profitability gate (opt-in)#128
Open
dhruvraajeev wants to merge 1 commit into
Open
feat(transform): adaptive chars-per-token for the profitability gate (opt-in)#128dhruvraajeev wants to merge 1 commit into
dhruvraajeev wants to merge 1 commit into
Conversation
…(opt-in)
The gate weighs image tokens (exact, from the 28px patch grid) against text
tokens (chars / CHARS_PER_TOKEN). That CPT constant is hand-tuned: 4 for
reminders/tool_results, 2.0 for slab/history. docs/ADAPTIVE_CPT_PLAN.md measured
real density near ~1.5 on dense content, so the gate over-estimates how cheap
text is and declines compressions that would have won.
This learns CPT from telemetry pxpipe already writes. bucket_chars,
baseline_tokens and image_pixels have been logged for a while and nothing read
them; a 6-bucket least-squares fit over those rows solves a marginal token rate
per bucket (CPT = 1/slope). No new dependencies.
src/core/cpt-fit.ts the fit. Pure math, no fs, safe in the Workers build.
src/cpt-store.ts Node-only. Reads events.jsonl, writes cpt-state.jsonl,
hands the gate a resolver.
Fails closed everywhere, which is the part worth reviewing. Min 20 samples, min
8 bucket appearances, plausible band 0.8-6.0, condition-number check,
non-positive slope, throwing resolver — each rejects that one bucket back to its
baked constant and records why in cpt-state.jsonl. A wrong learned rate costs
money; a missing one is just today's behavior. Precedence is explicit host
charsPerToken -> learned -> constant. Workers keeps the constants outright.
Image cost is subtracted on the patch grid from anthropic-vision.ts, not the
retired pixels/750 figure. Pages are 1568x728 and both edges are multiples of
28, so pixels/784 reproduces the patch count exactly (56x26 = 1456).
Off unless PXPIPE_ADAPTIVE_CPT=1; refits are throttled and never block a
request. Verified: the fit recovers CPTs it was generated from within 5%
including under noise, and a learned rate flips a real gate verdict both ways.
Savings on production traffic are NOT measured — that needs a shadow-mode A/B,
which is why this is opt-in. Reproduce: npx tsx eval/adaptive-cpt/demo-cpt.mts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gate weighs image tokens (exact, from the 28px patch grid) against text tokens (chars / CHARS_PER_TOKEN). That CPT constant is hand-tuned: 4 for reminders/tool_results, 2.0 for slab/history. docs/ADAPTIVE_CPT_PLAN.md measured real density near ~1.5 on dense content, so the gate over-estimates how cheap text is and declines compressions that would have won.
This learns CPT from telemetry pxpipe already writes. bucket_chars, baseline_tokens and image_pixels have been logged for a while and nothing read them; a 6-bucket least-squares fit over those rows solves a marginal token rate per bucket (CPT = 1/slope). No new dependencies.
src/core/cpt-fit.ts the fit. Pure math, no fs, safe in the Workers build.
src/cpt-store.ts Node-only. Reads events.jsonl, writes cpt-state.jsonl,
hands the gate a resolver.
Fails closed everywhere, which is the part worth reviewing. Min 20 samples, min 8 bucket appearances, plausible band 0.8-6.0, condition-number check, non-positive slope, throwing resolver — each rejects that one bucket back to its baked constant and records why in cpt-state.jsonl. A wrong learned rate costs money; a missing one is just today's behavior. Precedence is explicit host charsPerToken -> learned -> constant. Workers keeps the constants outright.
Image cost is subtracted on the patch grid from anthropic-vision.ts, not the retired pixels/750 figure. Pages are 1568x728 and both edges are multiples of 28, so pixels/784 reproduces the patch count exactly (56x26 = 1456).
Off unless PXPIPE_ADAPTIVE_CPT=1; refits are throttled and never block a request. Verified: the fit recovers CPTs it was generated from within 5% including under noise, and a learned rate flips a real gate verdict both ways. Savings on production traffic are NOT measured — that needs a shadow-mode A/B, which is why this is opt-in. Reproduce: npx tsx eval/adaptive-cpt/demo-cpt.mts