Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c97f953
ops: record the checkout-disabled v7.0.0 production deployment
devswha Aug 2, 2026
5cc1514
Merge pull request #671 from devswha/bot/polar-golive-evidence
devswha Aug 2, 2026
1b207d4
ops: Gate B readiness ledger — satisfied items and the four owner blo…
devswha Aug 2, 2026
54f47a1
Merge pull request #672 from devswha/bot/gate-b-ledger
devswha Aug 2, 2026
ee3f716
ops: remeasure deepseek flash after the 0731 re-post-training
devswha Aug 2, 2026
1bf7f40
Merge pull request #673 from devswha/bot/deepseek-0731-measurement
devswha Aug 2, 2026
3ac94b2
ops: root-cause the deepseek-0731 fidelity failures
devswha Aug 2, 2026
04ae8af
Merge pull request #674 from devswha/bot/deepseek-0731-rootcause
devswha Aug 2, 2026
41fddc3
ops: retract the deepseek-0731 disqualification — the failures were t…
devswha Aug 2, 2026
fb96618
Merge pull request #675 from devswha/bot/deepseek-0731-correction
devswha Aug 2, 2026
d5eef0e
ops: measure the deepseek-0731 reasoning dial and cache behavior
devswha Aug 3, 2026
cd5a232
Merge pull request #676 from devswha/bot/deepseek-0731-effort-low
devswha Aug 3, 2026
c51c946
feat(web): deepseek reasoning controls for the server-paid free tier
devswha Aug 3, 2026
47b5807
fix: restore the injectable-clock jsdoc dropped by the extraBody edit
devswha Aug 3, 2026
ac09fd2
fix: a temperature-rejecting model cannot recover the field from extr…
devswha Aug 3, 2026
c295414
Merge pull request #677 from devswha/bot/free-tier-deepseek
devswha Aug 3, 2026
ab83c99
chore: retrigger preview for the deepseek free-tier smoke
devswha Aug 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,14 @@
# PATINA_FREE_API_KEY=your-provider-key
# PATINA_FREE_PROVIDER=gemini
# PATINA_FREE_MODEL=gemini-3.6-flash
#
# Free-tier alternative (measured 2026-08-03, see
# docs/operations/serving-engine-deepseek-0731-correction-20260803.md):
# deepseek/deepseek-v4-flash matches the gate at ~1/8 the cost but ~44s per
# rewrite at the low-reasoning point. When the free provider is deepseek the
# server cuts rewrite reasoning to 'low' automatically;
# PATINA_FREE_REWRITE_REASONING=low|medium|high|off overrides that cut.
# BYOK and Pro rewrites never receive it.
# PATINA_QUOTA_HMAC_SECRET=your-long-random-secret

# --- Pro tier ($9.99/mo USD, license-gated; production provider: Polar) -----
Expand Down
46 changes: 46 additions & 0 deletions docs/operations/dep-prod-disabled-20260803.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# DEP_PROD_DISABLED — production deployed with checkout disabled (2026-08-03)

> Exit-evidence record for the `DEP_PROD_DISABLED` blocker in
> [`v6.4-preflight-hold.json`](v6.4-preflight-hold.json). Immutable: append
> corrections as new dated sections.

## Deployment

| Fact | Value |
|---|---|
| Release merge | PR #670, dev → main (merge commit), v7.0.0 |
| Production deployment | `https://patina-klpr5q2jo-devshwas-projects.vercel.app`, status Ready, 2026-08-03 |
| Stable alias | `https://patina.vibetip.help` |
| Deployed version | 7.0.0 (`origin/main` package.json) |
| Binding table on board | Polar production tuple only (`PAY-B-20260729-POLAR-ea8385dc-4c9c3f17`) |

## Disabled launch shape (fetched from the alias post-deploy)

`/launch-config.js` served exactly the six-field disabled artifact:
`{schemaVersion: 1, channel: "disabled", enabled: false, checkoutOrigin: null,
checkoutPath: null, evidence: null}` — no checkout button is exposed.

## Gate probes (UTC 2026-08-03, against the alias)

| Probe | Result | Meaning |
|---|---|---|
| pro tier, no Authorization | **401** `license required` | fail closed |
| pro tier, unknown license | **403** `license not entitled` | Polar gate answering on 7.0.0 |
| free tier rewrite | 200 stream, `terminal_failed` | see the incident below |

## Fail-closed regression caught during the rollout

The first preview build of this change **failed by design**: the Preview
environment still carried `PATINA_PRO_CHECKOUT_ENABLED=true` with the retired
Lemon Squeezy URL, and `generate-launch-config.mjs` refused it
(`must exactly match a source-controlled checkout evidence binding`). The
preview flag was reset to `false` and the build went green — live proof that
environment values alone cannot resurrect a dead checkout route.

## Incident (open, blocks Gate-B health evidence)

The free-tier smoke returned `terminal_failed`: the server-side Gemini key is
rejected with **HTTP 429 "project has exceeded its monthly spending cap"**.
This predates and is independent of this deployment (same key served 6.3.4).
Owner action: raise/clear the spend cap in AI Studio, then re-run the free and
pro smokes before recording Gate-B health evidence.
39 changes: 39 additions & 0 deletions docs/operations/gate-b-readiness-20260803.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Gate B readiness — what exists, what blocks (2026-08-03)

> Working ledger for the `GATE_B` blocker in
> [`v6.4-preflight-hold.json`](v6.4-preflight-hold.json). Gate B requires:
> completed production source-binding integration, hosted identity, usage,
> dedicated runtime, content-valid PAY-B-COST evidence, and real-path OBS
> evidence — approved by the Payment Runtime Owner + maintainer.

## Satisfied

| Requirement | Evidence |
|---|---|
| Production source-binding integration | PR #668 (`db909c2`), shipped to main via PR #670; binding table carries exactly `PAY-B-20260729-POLAR-ea8385dc-4c9c3f17` |
| Hosted identity | `https://patina.vibetip.help` serving v7.0.0; deployment `patina-klpr5q2jo…` ([`dep-prod-disabled-20260803.md`](dep-prod-disabled-20260803.md)) |
| PAY-B-COST evidence | [`pay-b-cost-v1.md`](pay-b-cost-v1.md) + `pay-b-cost-20260724*.json.bundle.json`; margin decision [`pro-margin-decision-20260729.md`](pro-margin-decision-20260729.md) (~55% at 100 rewrites/mo on gemini-3.6-flash) |
| Rollback procedures | [`rollback-drills.md`](rollback-drills.md) — measured 2026-07-23 (sale-close within the 10-minute bound); owner sign-off outstanding |
| Approval + payout + KYC | [`polar-approval-20260803.md`](polar-approval-20260803.md) |
| Secret presence | [`secret-manager-record-20260803.md`](secret-manager-record-20260803.md) |

## Blocking — owner actions, in order

1. **Gemini spend cap (incident).** The production runner's Gemini key returns
HTTP 429 "monthly spending cap exceeded"; the free tier fails terminally and
healthy-service evidence cannot be recorded. Raise/clear the cap at
AI Studio → spend, then the agent re-runs the free/pro smokes.
2. **`PATINA_SYNTHETIC_PRO_LICENSE`.** The pro-monitor synthetic probe needs a
real license; the prior verification license was shredded. Issue one via the
bounded forever-100% verification code (a zero-amount checkout), hand only
the license key to the secret manager — never paste it into the repo or chat
logs that persist to disk.
3. **Real-path OBS evidence.** After 1 and 2: the `/api/pro-monitor` cron cycle
must produce an ACKed healthy `OBS-ALERT-v1` receipt with `realPath: true`
per [`pro-launch.md`](pro-launch.md) / the pro-launch-v1 dashboard spec.
4. **Gate-B approval.** Payment Runtime Owner + maintainer (both hats: owner)
record the approval naming this ledger's evidence.

Then Gate D, rollback sign-off, `PAY_OPEN`, and the live-open env flip
(`PATINA_PRO_GATE_EVIDENCE_ID=PAY-B-20260729-POLAR-ea8385dc-4c9c3f17`,
`PATINA_PRO_CHECKOUT_ENABLED=true`, regenerate, redeploy).
100 changes: 100 additions & 0 deletions docs/operations/serving-engine-deepseek-0731-20260803.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# deepseek-v4-flash-0731 remeasured — meaning-gutting fixed, fidelity now the blocker (2026-08-03)

> Same apparatus as the definitive 2026-07-27 rerun in
> [`serving-engine-cost-20260725.md`](serving-engine-cost-20260725.md): all 22
> live-quality fixtures, fixed judge `gpt-5.5` via the codex-cli subscription
> seat, candidate over the DeepSeek API with thinking disabled. Not a Gate-B
> artifact and not a provider-default change; the v6.4 hold keeps defaults
> frozen.

## Why remeasured

DeepSeek re-post-trained the flash line and replaced it in place on 2026-07-31
(`deepseek-v4-flash` now serves V4-Flash-0731, public beta, pricing unchanged:
$0.14/M in, $0.28/M out, ~$0.003 per patina rewrite — 10x under the shipped
gemini-3.6-flash at $0.030).

## Result: 15 pass / 1 warn / 6 error (gemini-3.6-flash baseline: 20/22)

The July disqualifier is **gone**: `ko-news-01`, gutted to MPS 24 in July, now
scores **MPS 100**. Worst-case MPS across all 22 fixtures is 50 (`en-blog-01`);
20 of 22 sit at MPS >= 80. The model no longer deletes meaning wholesale.

The new failure mode is **fidelity** — omitted claims/anchors:

| fixture | mps | fidelity | note |
|---|---:|---:|---|
| ko-blog-01 | 70 | **41.7** | fidelity<70 |
| ko-howto-01 | 100 | **58.3** | fidelity<70 |
| ko-news-01 | 100 | **58.3** | fidelity<70 |
| ko-social-01 | 100 | **66.7** | fidelity<70 |
| en-howto-01 | 100 | **66.7** | fidelity<70, ai_after 33.3, ai_not_improved |
| en-blog-01 | **50** | 83.3 | mps<70 |
| ko-public-docs-01 | 100 | 100 | warn: ai_not_improved (50.0 → 50.0) |

Reading: it now preserves the gist (MPS high) but drops individual claims —
four of five fidelity failures are Korean. This is measured on the fixed
post-register-failure rubric that already exempts packaging removal, so these
are real omissions, not rubric artifacts.

## Verdict

- **Not a Pro-tier swap candidate today.** 15/22 vs 20/22 with five fidelity
floor failures loses to the shipped engine on the column that matters for a
paid meaning-preserving product.
- **Trajectory is real.** One post-training pass removed the meaning-gutting
failure entirely. Re-measure on the next flash update; if fidelity clears the
floor at comparable pass counts, the 10x cost cut (~55% → ~85%+ margin at 100
rewrites/mo) justifies the frozen-default process.
- **Possible near-term use: the free tier.** The free tier burns the server's
Gemini budget (currently over its monthly spend cap) on non-paying traffic.
Serving free-tier rewrites on deepseek-v4-flash at 1/10 cost — while Pro
stays on gemini-3.6-flash — would cut the burn and decouple the free tier
from the Gemini cap. Separate decision: needs the env-driven free-runner path
checked and an owner call; not part of this measurement.

Raw run: 2026-08-03, `quality:live`, 22 fixtures, judge codex-cli/gpt-5.5,
candidate `deepseek-v4-flash` with `{"thinking":{"type":"disabled"}}`.

## Root cause (2026-08-03 addendum): why fidelity fails

Four failing fixtures were regenerated with the identical prompt path and the
raw deliveries inspected. Two distinct causes, neither of which is "the model
writes worse prose":

### 1. Output-contract violations (3 of 4 inspected failures)

The rewrite prompt requires a `[BODY]` / `[SELF_AUDIT]` structure; the engine
must return them so the delivery layer can strip the audit and hand back only
the body. gemini-3.6-flash and claude-sonnet-5 follow the contract; 0731 does
not, inconsistently per run:

- `en-howto-01`, `ko-news-01`: the whole `[SELF_AUDIT]` bullet block survived
into the delivered text — the customer would receive the model's self-review
appended to their document. The judge correctly charges the garbage.
- `ko-howto-01`: an orphan duplicate `[BODY]` tag at the end of the delivery.
- `ko-blog-01`: no tags at all.

The re-post-training that improved "agentic" benchmarks appears to have made
the model editorialize about its own work instead of following the output
schema. A patina-side stripper hardening could salvage some of this (tolerate
malformed/duplicated tags), but a serving engine that only sometimes honors
the response contract is a per-request coin flip.

### 2. Fabrication under naturalness pressure (ko-blog-01, fidelity 41.7)

Original: 통근 시간 절감이 생산성 향상에 기여한다 (plain claim).
Delivered: "생산성이 올라간다는 **연구 결과도 나온다**" — the model invented a
supporting research finding that the original never made. It fabricates
evidence to make prose sound more human. This is the one failure patina can
never engineer around: the product's core promise is that the claim set does
not change.

### Reading

The July failure (wholesale meaning deletion) is genuinely fixed; the August
failures are contract compliance and claim fabrication. Cause 1 is partially
mitigable on our side and worth re-testing on the next model update; cause 2
is disqualifying for the paid tier as long as it reproduces. The free-tier
option stands, but with the stripper hardening as a prerequisite so scaffold
leakage never reaches a visitor.
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Correction: the deepseek-0731 failures were a measurement artifact (2026-08-03)

> Corrects the verdict in
> [`serving-engine-deepseek-0731-20260803.md`](serving-engine-deepseek-0731-20260803.md)
> and its root-cause addendum. Those documents stand as history; this one
> supersedes their conclusions. Pattern note: this is the same failure class as
> the July register-failure saga — the apparatus, not the engine.

## What was wrong with the first run

The 15/22 run disabled DeepSeek reasoning (`thinking: {type: "disabled"}`) to
match the July cost point. The shipped gemini-3.6-flash baseline runs with its
default (low) reasoning. That asymmetry caused the failures: with thinking
off, 0731 drifts on the `[BODY]`/`[SELF_AUDIT]` output contract (audit
leakage, orphan tags) and, in one case, fabricated a supporting claim. With
thinking at its default, all four inspected deliveries came back clean — no
tag leakage, no fabrication, dropped claims restored.

## Thinking-on rerun: 20 pass / 2 warn / 0 error

Same 22 fixtures, same fixed judge (`gpt-5.5` via codex-cli), same prompt
path; only the thinking override removed.

| | gemini-3.6-flash (2026-07-27) | deepseek-v4-flash-0731, thinking on |
|---|---:|---:|
| pass | 20/22 (2 fail) | **20/22 (2 warn, 0 error)** |
| worst MPS | 80 | 80 |
| worst fidelity | — | 91.7 |
| ai_not_improved | 2 | 2 (`en-social-01` −1.8, `ko-email-01` −0.1; both near-clean sources) |
| approx cost / rewrite | $0.030 | ~$0.004 (reasoning tokens included; still ~8x cheaper) |
| latency / rewrite | ~8s | **~60–90s** (reasoning-dominated) |

Both warns are the borderline "source already scores low, rewrite does not
improve it" class — the same evasion-adjacent shape the July analysis treated
as tolerable at 2/22 for the shipped engine.

## Corrected verdict

- Quality: with default thinking, 0731 **matches the shipped engine** on this
gate — the July meaning-gutting and the August contract/fabrication findings
are both apparatus-resolved or model-resolved.
- The real remaining tradeoff is **latency**: ~60–90s per rewrite versus ~8s.
For the streaming playground UX this is user-visible waiting, and the
free-tier hourly burst window compounds it. Cost favors 0731 ~8x.
- Caveats before any provider decision: this is **n=1 per fixture** and the
harness itself documents ±20 MPS swing between identical runs — a swap
candidate needs `--repeat` validation; DeepSeek bills output 2x at announced
peak hours (policy announced, date TBA), which moves the cost figure; and
the v6.4 hold freezes provider defaults, so any change goes through the
frozen-default process.

## Options this opens (owner decisions, not taken here)

1. **Free tier on 0731 (thinking on)**: cuts free-tier burn ~8x and decouples
it from the Gemini spend cap; latency is more tolerable for a free tier.
2. **Pro on 0731**: only after repeat-validated quality and a latency call —
a paying user waiting 60–90s is a product regression even if quality ties.
3. **Status quo** pending the Gemini cap fix, re-measuring on DeepSeek's next
update or the peak-pricing activation.

## Addendum (2026-08-03): the reasoning dial, measured

Latency anatomy on `ko-news-01` (prompt ~22.8k tokens, 99.9% cache-hit): the
API decodes at ~125 tok/s; the time goes to reasoning volume, not transport.
`budget_tokens` is ignored by the API; `reasoning_effort` works.

| thinking | reasoning tokens | latency/rewrite | 22-fixture result |
|---|---:|---:|---|
| disabled | 0 | ~4s | 15 pass — contract violations + fabrication (retracted run) |
| `reasoning_effort: low` | ~5.3k | ~44s | **20 pass / 0 warn / 2 error** (both `mps<70`: en-marketing 66.7, ko-marketing 50) |
| default | ~11.5k | ~60–94s | 20 pass / 2 warn / 0 error |
| gemini-3.6-flash (shipped) | its default | ~8s | 20 pass / 2 fail |

Production context: the shipped gemini rewrite call also runs full reasoning —
`scoringExtraBody` cuts reasoning only on the MPS/fidelity judges, and the
rewrite call is excluded because reduced thinking measurably amputated
content. The effort-low DeepSeek result rhymes with that lesson in a milder
form: both failures are meaning drops, concentrated in the marketing register,
one (66.7) within the harness's documented ±20 MPS single-run swing.

Cache behavior is a strength, not a risk: the patina prompt's fixed 22k-token
pattern/persona prefix hits DeepSeek's automatic prefix cache at ~99.9%
($0.0028/M on hits; a cold miss costs ~$0.003 once). Deploys that change
pattern files reset the prefix, which is expected and cheap.

Standing options update: the free-tier candidate settings are effort-low
(~44s) or default (~60–94s); either needs `--repeat` validation before a
swap, per the harness's own noise bound.
2 changes: 1 addition & 1 deletion docs/operations/v6.4-preflight-hold.json
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@
}
},
"sourceHashes": {
".env.example": "e7d1762137e29232217712bdc7c699f6abd24595271f31dc2528fcb92c2cb00d",
".env.example": "29b1cdc48b5c14ed0c84716ad9ca42ba23001b9f40f99efb78c2c368180e7e5a",
"src/model-defaults.js": "c568977fcac8ea44d5387a8a8745b062675ec94d73d39ce528c492ad35f87176",
"src/providers.js": "92415eacaf87da2d0f2aed7db97feeb98b8b087adfe584a02c4478353b807d90",
"src/web-rewrite-contract.js": "be433e8260e452028eb53b5c1aed0935a4150cd669ec2d5e826302219df22b9a",
Expand Down
Loading