feat: sync-views, and a scheduled action that runs it for production - #184
Merged
Conversation
`views --sync` made the consequential act look like an option on the harmless one. `views` reads a 90-day window; the sync rescues that window before it ages out permanently, and since 0.33.0 that is the thing keeping history alive. It also did not read correctly in a crontab, which now matters. `sync-views` is symmetric with `sync-access` — an operator command that reconciles against an external system, run periodically, distinct from the read commands. The old form keeps working and says where it moved. Cutting it would have broken docs, muscle memory and possibly a crontab — and a scheduled sync that starts failing silently is the exact failure ADR-023 decision 9 exists to prevent. Production had nowhere the sync could correctly run. Its Cloudflare credential is deliberately not on a laptop (#38), and resolveWriteTarget refuses a sync from a checkout whose marker names a different deployment, because that would read one account's analytics and write them to another — storing a summary whose ids match nothing and reporting a measured zero for every document. CI is where prod's credentials already live, so that is where its schedule belongs. Verified the workflow's assumptions before writing it: CI has no config.json and no registry, only a restored marker and env vars. Ran sync-views in exactly that shape against the test deployment and it resolved the target, queried Analytics Engine and posted the summary. The workflow has no health step, deliberately. `health` asserts the deployed build matches the checkout's, and deployment is decoupled from releases (ADR-010) — production legitimately runs a commit main has moved past. A daily job failing on that would cry wolf, and a scheduled job that cries wolf gets muted, which is the failure this workflow exists to prevent. The make/CLI parity gap stays open in #166. It depended on this naming decision and is now unblocked rather than resolved.
The rename changed the refusal message and the e2e suite still asserted the old one. It passed locally because e2e needs a live wrangler dev and is cancelled without one — so the local run was green on a suite that had not executed. Ran it properly this time. Asserts both spellings rather than just the new one: the old form is kept working on purpose, and this is the test that would catch it quietly ceasing to reach the real command.
A scheduled job that silently stops running is invisible — the same shape of failure, one level up, as the silent data loss the sync exists to prevent. So it gets the same treatment: say something on every successful run and let the silence be the alarm. Failures go to the channel that interrupts someone, and say what a failure costs rather than that a job went red: history is quietly not being saved, and each failed run shortens the window in which that is still fixable. The message names the deployment, taken from the restored intent file — the host only, never the account id. That is not secret but it is not something to post into a chat channel either. A plain fetch rather than a marketplace Slack action. This job can read production's Cloudflare token, its deployment bearer and its intent file; every third-party action added here is a supply-chain surface with access to all three, in exchange for building a JSON body that takes ten lines. The CLI ships zero runtime dependencies because it is the thing that asks for your Cloudflare token, and the job holding those secrets should not be held to a looser standard. Both webhooks are optional and read from environment secrets, so a fork without them skips the steps and nothing else changes. That keeps the workflow useful as the worked example of the operator-side schedule ADR-023 asks for, which is easy to recommend and less obvious to wire up. Notifications never fail the job, and that had to cover a thrown fetch as well as a non-2xx response — testing found the first version exited 1 on an unreachable host, which would have turned a run whose sync SUCCEEDED red. A Slack outage would then have looked exactly like lost data, inverting the point of alerting. Nothing is logged but HTTP status. A response can echo the request and the request URL is the credential.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: sync-views, and a scheduled action that runs it for production
Refs: #166 · unblocks the prod half of #167
Overview
Two things, and the second is the one that matters today: production has been capturing view data
since this morning's deploy, and nothing could promote it into the durable summary. Analytics
Engine drops it after ~90 days. This adds the schedule that keeps it.
views --sync→pagevault sync-viewsviewsreads a 90-day window; the sync rescues that window before it ages out permanently. As aflag, the consequential act looked like an option on the harmless one — and it did not read
correctly in a crontab, which now matters because a schedule is what keeps history alive.
sync-viewsis symmetric withsync-access: an operator command that reconciles against anexternal system, run periodically, distinct from the read commands.
views --syncstill works and prints a note pointing at the new name:Kept rather than cut because it is in the docs, in muscle memory, and quite possibly in a crontab —
and a scheduled sync that starts failing silently is the exact failure ADR-023 §9 exists to prevent.
The note goes to stderr, so a
--jsonpipe is unaffected.The scheduled sync, and why it has to live in CI
Production's Cloudflare credential is deliberately not on a laptop (#38).
resolveWriteTargetalsorefuses a sync run from a checkout whose marker names a different deployment — it would read one
account's analytics and write them to another, storing a summary whose ids match nothing and
reporting a measured zero for every document. Both are correct, and together they meant prod had
nowhere the sync could run at all.
CI is where prod's credentials already live, so that is where its schedule belongs. Daily at 03:17
UTC — the odd minute is deliberate, GitHub's scheduler is busiest on the hour and drops jobs under
load, and a missed run here is invisible.
Verified the workflow's assumptions before writing it. CI has no
config.jsonand no registry,only a restored marker and env vars. I ran
sync-viewsin exactly that shape against the testdeployment — it resolved the target, queried Analytics Engine and posted the summary. The guard
passes because the restored marker and the target name the same deployment.
One thing I built and then removed
I had a
healthstep at the end to report remaining risk. It would have been wrong:healthasserts the deployed build matches this checkout's build, and deployment is decoupled from
releases (ADR-010), so production legitimately runs a commit
mainhas moved past. The step wouldhave failed the job daily on version drift that isn't a fault.
A scheduled job that cries wolf gets muted — which is precisely the failure this workflow exists to
prevent. Removed, with the reasoning left in the file so the next person doesn't re-add it. The sync
step already reports what it captured, and a real failure fails the job.
PR Prep Summary
health's fix hintsync-viewsmake viewsparity gap stays in #166 — it depended on this naming decision and is now unblocked, not resolvedTesting
New tests:
sync-viewsdispatches and refuses a filtered sync before touching the network · the oldform still reaches the real command and says where it moved · both are reachable from help and
viewsno longer advertises the flag.Not covered: the workflow itself has never run. It cannot be exercised without production's
secrets, so its first real run is on merge — I'd dispatch it manually once rather than wait for
03:17.
Files Changed
New
.github/workflows/sync-views-prod.yml— daily + manual,environment: productionModified
cli/bin/pagevault.mjs—sync-viewsdispatch, deprecation note, usage summarycli/lib/help.mjs— split intoviews(read-only) andsync-viewscli/lib/ops/health.mjs— the fix hint names the new commandcli/pagevault.test.mjs·CHANGELOG.md· three docsDeployment Instructions
No Worker change. After merge, dispatch
Sync production view metricsonce by hand to confirmthe secrets resolve — the first scheduled run is otherwise 03:17 UTC and a silent failure there
looks identical to no data.
CLOUDFLARE_API_TOKENin theproductionenvironment needsAccount · Account Analytics · Read.It is the same secret
deploy-prod.ymluses; if that permission was never added, the run fails witha 403 and the CLI's error names the exact dashboard path.
Next Steps
0.35.0. Rolls up the held#166wording fix too.make viewsparity gap, now unblocked by this naming decision.solves the scheduled case, which is the one that was losing data.