feat(#114): the outbound validate_startup hook, with its ledger banner (train) - #327
Merged
Conversation
…d directory toggle The remainder of #114. DestinationConnector had no startup-validation hook and FileDestination mkdir'd on write, so an outbound target directory was never validated: a typo'd directory / remote_dir would not fail, it would be CREATED, and every message delivered into it counted and logged as delivered -- because it was. On a first deployment that is a feed landing in a path nobody is watching with no error anywhere. (Nothing is misdelivering today; there are zero deployments.) The item's 6/10 rested on "a clean workaround via the on-demand test probe". Measured against the shipped code before building: FileDestination.test_connection creates the missing directory (exists False -> True) and RemoteFileDestination.test_connection calls ensure_dir, which creates. The probe cannot answer the question the toggle asks, because asking changes the answer. - DestinationConnector.validate_startup(), defaulting to a no-op -- the exact shape of the SourceConnector hook, so the other eleven destination connectors are untouched and this is not a protocol change that ripples. FILE and REMOTEFILE override it; DestinationStartupError mirrors SourceStartupError. - The runner awaits it in _start_outbound right after the build, INSIDE the existing ADR-0031 isolation try, so a refusal takes the same path as a build failure: lane recorded failed with no connector, its delivery worker still spawned, routed rows retried and never dropped. On an outbound that degraded-lane state is what "invalid means not-started" means. Same call on the operator start path (_ensure_destination_built); deliberately NOT on the reload path, whose stated invariant is that a connector build there cannot fail. - validate_directory becomes a both-directions option and the 2026-08-03 outbound WiringError is removed: it existed only because no destination read the setting. - Under the toggle nothing is ever created -- not at start, not on write, and not by POST /connections/{name}/test, which would otherwise silently repair the typo the toggle exists to catch. The REMOTEFILE arm pre-checks with a listing to RECLASSIFY the failure as transient: an SFTP/FTP no-such-dir is permanent, so letting the upload fail on its own would dead-letter live traffic over a merely-unmounted share. - Default unchanged, with one deliberate addition: a target directory the engine actually had to CREATE now logs a WARNING, so that delivery is no longer indistinguishable from a normal one. The FILE path costs no extra syscall (mkdir(exist_ok=True) already probed is_dir on its FileExistsError branch); _RemoteClient.ensure_dir now reports whether it created. The lenient arm is the default because the item's own trigger is the opposite case: an intermittently-available directory must NOT fail startup. Tests were written red first (14 failing across both directions), and both directions were then re-sabotaged to prove they have teeth: dropping the runner call reds the refusal test, and making the toggle non-optional reds the deferral tests.
…NS rows, test-plan row The 2026-07-17 amendment deferred the outbound hook on two grounds: the destination "already mkdirs on write", and it "has the on-demand test probe". The 2026-08-03 follow-on withdrew the second; this amendment withdraws the first and marks that follow-on superseded, since the WiringError it added existed only while the hook did not. CONNECTIONS.md moves validate_directory from "in" to "both" on File and Sftp/Ftp and states the outbound semantics: the target must already exist and accept a write, is then never created at start, on write or by the test probe, and a delivery into a vanished directory fails retryably. It also records the default-arm change that applies to every existing outbound -- a directory the engine had to create is now logged. The master-test-plan row moves 7 -> 9 and names what the two new runner-level cases assert.
The banner half of #114's train. Pairs with w3-outbound-validate (0871320); backlog-hygiene demands a ledger change from a PR that cites an item and touches engine code, and a lane is forbidden to make one. CLAIMS VERIFIED AGAINST THE BRANCH RATHER THAN TRANSCRIBED FROM ITS REPORT: validate_startup exists on the DestinationConnector contract with an override, ADR 0031 carries the amendment with its 2026-08-03 follow-on marked superseded, and 5 WiringError lines are removed. The defect is written in the conditional, as it must be: FileDestination previously mkdir-ed its target on write, so a deploying operator who typo'd remote_dir WOULD get a directory silently created and messages delivered into it -- the feed reading healthy while writing to the wrong place, with nothing reporting it. Recorded because it is the more interesting half: THIS ITEM'S OWN 6/10 RATIONALE WAS RE-MEASURED AND IS FALSE. It claimed a 'clean workaround via the on-demand test probe', but both destinations' test_connection CREATE the directory -- so the act of asking changes the answer. An item's scoring rationale falsified by execution, the same class as #1011's refuted premise. A workaround that alters the state it reports on is not a workaround. INHERITED FAILURE, STATED SO IT IS NOT READ AS MINE. This branch is cut from origin/main, which is still red on test_dast_claims::test_no_file_claims_dast_closes_the_independent_gap. Confirmed INHERITED BY IDENTITY, not by count: the failing line is docs/BACKLOG.md:3738, the #1008 'independent of the gate ... INCOMPLETE' text, while this commit's banner is at line 1686 and contains zero occurrences of 'independen'. It clears when w3-docs-guard-gap (c2b85ee) lands, which fixes both the guard and that prose. Doc guards run locally before handover, which is the practice this session's incident established: 46 passed, 1 failed, that one being the inherited case above. Ledger gates clean; 484 items each declaring exactly one status. Open 181 -> 180.
Two blockquotes, two lines below the SHIPPED banner, asserted in the PRESENT TENSE that DestinationConnector 'still has no validate_startup hook' and FileDestination 'still mkdirs on write'. Both false once the code lands, in an item declaring itself complete. Rewritten to past tense. WHY NO GATE CAUGHT IT, and this is the part worth keeping: the offending quote leads with U+26A0, which is in NEITHER _CLOSED nor _OPEN. So parse_items is structurally silent about it. THE GREEN BANNER GATE PROVES ONE STATUS BANNER PER ITEM; IT CANNOT SEE A PROSE CONTRADICTION TWO LINES AWAY, and reading its green as 'the item is truthful' is reading a different sentence than the one it asserts. Also recorded, because the close REVERSED it: the 2026-08-03 outbound WiringError that quote describes as BUILT was subsequently REMOVED -- it existed only to reject a setting no destination read, and a destination now reads it. A record that lists something as built, when a later change deleted it, is worse than silence. And the REMAINDER quote closed with a BUILD GATE -- 'if the hook is built, build it together with suppressing the mkdir-on-write, because a start-time-only check leaves the run-time fabrication intact under a setting name that promises otherwise.' That gate was HONOURED, and the banner now says so rather than leaving the instruction reading as outstanding work. MY OWN VERIFICATION SCAN THEN PRODUCED A FALSE POSITIVE, WHICH IS ITSELF THE LESSON. It flagged 'only the outbound validation HOOK remains' as a surviving present-tense claim. It is not: it is that phrase QUOTED inside my correction, immediately followed by 'which was true then and is not now'. A substring scan cannot distinguish a CLAIM from a QUOTED AND RETRACTED claim -- the same presence-equals-meaning failure CLAUDE.md section 11 describes for glyphs, and the same shape as the DAST guard reading INCOMPLETE as COMPLETE. I checked the context rather than trusting my own scanner, and the scanner was wrong. Verified: #114 reads closed via parse_items, 484 items each declaring exactly one status, doc guards 30 passed locally before handover.
The outbound validate_startup hook (BACKLOG #114). Rides with its ledger banner because a code lane cannot edit the ledger and a banner alone would assert SHIPPED for code not on main.
The close banner for BACKLOG #114, with the two stale blockquotes rewritten to past tense. parse_items cannot see that class of contradiction: the offending quote leads with a glyph in neither _CLOSED nor _OPEN, so a green banner gate proves one status banner per item, not that the item is truthful.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes BACKLOG #114 -- the outbound
validate_startuphook, landing as a two-lane train: the codeand its ledger banner together, in one PR.
Why a train, and why this one is not optional
A code lane cannot edit
docs/BACKLOG.md(single-writer), so the banner is authored separately. Landingeither half alone is wrong, and the banner half is wrong in a way nothing would catch:
git merge-tree origin/main w3-banner-114merges CLEAN. The resulting tree declares"SHIPPED -- #114 is complete" while
transports/base.pyhas noDestinationConnector.validate_startupand ADR 0031 still reads "remains out of scope and deferred". No conflict, no marker, no gate. Git
conflicts on concurrent edits, never on a claim its own merge falsifies.
Verified on the combined tree instead of on either branch:
transports/base.py:validate_startupat line 498, insideDestinationConnector(class opens at470) -- plus the source-side hook at 447 in
SourceConnector, andDestinationStartupErrorpresent.docs/BACKLOG.md: the#114SHIPPED banner, one status banner for the item.The two lanes touch zero files in common (13 code/doc files vs one ledger file), which is what makes
the train safe rather than merely convenient.
What shipped
DestinationConnector.validate_startup()is a default no-op, so the other eleven destinationconnectors are untouched and this is not a protocol change that ripples.
FileDestinationandRemoteFileDestinationoverride it. The runner awaits it in_start_outboundand_ensure_destination_builtinside the existing ADR 0031 isolationtry, so a refusal is afailedlane with no connector whose delivery worker still spawns -- routed rows are retained, retried and
buildup-alerted rather than dropped, and the count-and-log invariant holds.
The defect, in the conditional (this is a beta with zero deployments):
FileDestinationpreviouslymkdir-ed its target on write, so a deploying operator who typo'dremote_dirwould get a directorysilently created and messages delivered into it -- the feed reading healthy while writing to the wrong
place.
validate_directorybecomes a both-directions option, and the 2026-08-03 outboundWiringErroris removed, because it existed only to reject a setting no destination read.One deliberate change reaches every existing outbound, toggled or not: a target directory the engine
actually had to create now logs a
WARNING, so a delivery into an invented path is no longerindistinguishable from a normal one.
The ledger half fixes a self-contradiction the banner gate cannot see
#114carried two older blockquotes asserting "DestinationConnectorstill has novalidate_startuphook and
FileDestinationstillmkdirs on write" -- false once this lands, and sitting two lines belowthe new SHIPPED banner. Both are rewritten to past tense.
parse_itemsis structurally silent about this. The offending quote leads with a glyph in neither_CLOSEDnor_OPEN, so the gate proves one status banner per item and nothing more. Reading a greenbanner gate as "the item is truthful" is reading a different sentence than the one it asserts.
A second reversal was folded in: the 2026-08-03 blockquote listed the outbound
WiringErroras BUILT,and this change removes it. A record listing something as built when a later change deleted it is
worse than silence. The
REMAINDERquote also closed with a build gate -- build the hook togetherwith suppressing the mkdir-on-write, since a start-time-only check would leave the run-time fabrication
intact under a setting name that promises otherwise. That gate was honoured, and the item now says so
rather than leaving the instruction reading as outstanding work.
Verification
Peer-verified, then independently re-verified by a separate agent against current
origin/mainrather than the branch's base. Three measurement corrections came out of that and are recorded here so
the numbers in the banner are right:
validate_startupoverrides are two (FileDestinationatfile.py:243,RemoteFileDestinationatremotefile.py:667), not one -- which makes the banner's "other eleven destination connectors" exact:13 subclasses minus 2.
WiringErrorlines removed" is exact forconfig/wiring.py; whole-branch is 10 across 5 files,and only one of the five is an actual
raise.--stattotal churn asinsertions.
Drift checked against current
main, which advanced 50 commits past the code lane's base and touched 2of its 13 files -- neither invalidates: the
CONNECTIONS.mdedit is retry-cap prose with novalidate_directorymentions, and thewiring_runner.pyedits sit at lines 4573+ against the lane's1815-2480.
Not measured by the verifier (read-only, no checkout):
ruff,mypy, and the full suite against thecombined tree. CI is the instrument for those, and they are unmeasured rather than assumed green. The
sync-to-async conversion of three runner methods is the change most worth watching in the test legs.