feat: SDK conformance harness — real provider SDKs against the adapters - #64
Merged
Conversation
A nested Go module (conformance/, just conformance, CI step) that
boots stunt adapters and drives them with the official SDKs:
- stripe-go: form+bracket creates, PI create+confirm state machine,
iterator pagination walking has_more, webhook verification through
the SDK's own ConstructEvent HMAC validator (5 checks).
- aws-sdk-go-v2: STS GetCallerIdentity/AssumeRole with REAL SigV4 from
the documented synthetic credentials; full S3 lifecycle incl. binary
byte-exact round-trip and the ListObjectsV2 paginator following
continuations (8 checks).
- go-github: issue CRUD, Link-header pagination via resp.NextPage,
comments, state transitions (5 checks).
First-run findings, all fixed:
- stripe PaymentIntents returned amount as a JSON string — typed SDKs
reject it (real Stripe returns money fields as numbers).
- the router had no greedy path params: S3 keys containing slashes
404'd. A terminal {key+} segment now captures the remaining path
verbatim; S3 object routes use it.
- S3 XML LastModified rendered ...T05Z.000Z (millis appended after the
zone) — the AWS SDK time parser rejects it; now ...T05.000Z.
18 checks, all green; full main-module suite + gates green.
Review findings on PR #64, all addressed: - MAJOR: charges.star had the same amount-as-string bug the conformance run caught in PaymentIntents — form-encoded creates stored "2000" and echoed it as a JSON string, which typed SDKs reject. Coerced at create; charge.New added to the conformance suite as the pin (typed int64 round-trip). - Greedy {key+} capture documented honestly: URL-decoded, outer slashes trimmed (dir/ and dir address the same key) — CHANGELOG and adapters/README now say what it actually does. - conformance/go.mod keeps go 1.24 with the constraint documented: aws-sdk-go-v2/config v1.32.37 requires it; CI's GOTOOLCHAIN=auto upgrades transparently (lowering to 1.23 breaks the build). - Pagination boundary: the iterator count alone cannot distinguish walked pages from one big page — a raw ?limit=2 page now pins len(data)==2 and has_more=true alongside the walk. - Boot fails (not skips) when the adapters dir is absent — absence in this repo is a layout bug, not a green skip; just conformance runs under -race like the root suite; dead duplicated error check removed. 19 conformance checks green; main-module suite + gates green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The SDK conformance harness — real provider SDKs driving stunt adapters, asserting business outcomes rather than wire shapes. Nested Go module
conformance/(own go.mod so the SDK deps never touch the stunt binary's graph),just conformance, wired as a CI step.succeeded, SDK-iterator pagination walkinghas_more, and webhook verification through the SDK's ownConstructEventHMAC validator — signature verifies ANDdata.objectparsesGetCallerIdentity/AssumeRolewith real SigV4 signatures from the documented synthetic credentials; full S3 lifecycle — binary byte-exact round-trip (incl. non-UTF-8), HeadObject, the ListObjectsV2 paginator following continuations, DeleteObjectLink-header pagination viaresp.NextPage, comments, state transitionsEach suite also emits a scoreboard (TSV via
RUN_CONFORMANCE_SCOREBOARD) — the same output feeds the case-study/SEO pipeline, so the test pipeline and the content pipeline are one thing.Bugs it found on its first run — all fixed
amountas a JSON string ("amount":"4200"). Real Stripe returns money fields as JSON numbers; stripe-go's typed deserializer rejects the response outright. Invisible to every existing test (they unmarshal intoany). Coerced at create and render./{bucket}/{key}cannot matchphotos/2024/a.jpg— real S3 keys contain slashes routinely, so every nested key 404'd. A terminal{key+}segment now captures the remaining path verbatim (pinned by unit tests + the conformance run); S3 object routes use it.LastModifiedrendered...T18:13:05Z.000Z— millis appended after an already-terminated timestamp, which the AWS SDK's time parser rejects. Now...T18:13:05.000Zlike real S3.Verification
18 conformance checks green; main-module full suite, parse guard, QC boot, adapter lint, the all-adapter input-safety sweep, gofmt, vet — green. (One deliberate deviation documented in-test: the adapter pins the acacia API shape while current SDK lines pin dahlia-era versions — webhook verification uses the SDK's
IgnoreAPIVersionMismatchand asserts the adapter's pinned version explicitly.)