Add plugin hook-info field-shape requirements 10-19..10-22 - #72
Add plugin hook-info field-shape requirements 10-19..10-22#72wangyb-A wants to merge 12 commits into
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
cf46661 to
5a0dd6f
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Claude AI reviewThis PR adds five additive, declarative plugin-suite requirement YAMLs ( I reviewed each file against the engine that evaluates it ( No actionable findings. Verifications that passed:
Residual test risk (inherent, not a PR defect):
Reviewed commit |
This comment has been minimized.
This comment has been minimized.
Review follow-up: the replay-start matcher only probed updatedOperationsCount, so a replay record missing requestId, operationsCount, or executionStartTimestamp still passed. Add decoupled count:2 presence probes for the three inherited fields (mirroring the existing end-side block). updatedOperationsCount deliberately keeps only the replay-specific nonzero probe: whether an SDK surfaces an empty updated-operations collection before the first invocation is an unasserted implementation detail.
This comment has been minimized.
This comment has been minimized.
Review follow-ups: - error/executionError absence guards used /.+/ with count: 0, which an empty-string error value would slip past; use /.*/ so any present key (even empty) violates the guard (10-19, 10-20, 10-21). - Prose still claimed input/result values are asserted BY VALUE after the GA descope; rewritten as 'dumped when exposed but unasserted -- experimental' so the YAML contract matches the executable matchers (10-19, 10-20, 10-22). - 10-22 description narrowed to delta-item shape + full-map MEMBERSHIP, matching what inFullMap actually checks.
Requirements are plain test descriptions: executionInput, executionResult, and operation result no longer appear in the canonical dump schemas, and all GA-scope / experimental design commentary is removed. Assertions are unchanged.
| - match: | ||
| plugin: CONFPLUGIN | ||
| hook: operation-start | ||
| type: /(?i)context/ | ||
| subType: /(?i)branch/ | ||
| isReplay: false | ||
| count: 2 | ||
| # Both branch functions run live at least once each (branch A in invocation | ||
| # 1, branch B in the replay invocation) with children-replay=false. At-least: | ||
| # some SDKs also re-run a COMPLETED childless branch function during replay, | ||
| # legitimately reporting false again. | ||
| - match: | ||
| plugin: CONFPLUGIN | ||
| hook: fn-start | ||
| subType: /(?i)branch/ | ||
| isReplayingChildren: false | ||
| min_count: 2 | ||
| # THE field-surface probe: every branch fn-start record carries the | ||
| # children-replay indicator key — including branch A's REPLAY re-run. The | ||
| # minimum of 3 (branch A live + branch A replay + branch B live) cannot be | ||
| # satisfied by the two live records alone, so the replayed re-run must also | ||
| # bear the key. The indicator's TRUE value depends on service-side | ||
| # checkpoint timing (read from the checkpointed context details) and proved | ||
| # nondeterministic across live runs, so values are dumped for observability | ||
| # but only key presence is asserted. | ||
| - match: | ||
| plugin: CONFPLUGIN | ||
| hook: fn-start | ||
| subType: /(?i)branch/ | ||
| isReplayingChildren: /.*/ | ||
| min_count: 3 |
There was a problem hiding this comment.
Codex AI review
High: These aggregate counts do not correlate records to either branch. Branch A's live and replay records can alone satisfy min_count: 2 when its replay value is legitimately false, allowing an incorrect value for branch B. Likewise, permitted extra reruns can satisfy min_count: 3 even when another fn-start omits the key, contradicting the “every record” requirement. Give branches stable names or bind their IDs and assert each branch separately. To verify key presence across an unbounded number of reruns, emit an explicit hasIsReplayingChildren flag and assert that no record reports false.
Codex AI reviewFound one high-severity false-positive in requirement 10-23. Review was static as required; runtime behavior was not exercised. Reviewed commit |
Summary
Adds 5 plugin-suite requirements (10-19..10-23) probing the field shape of every plugin hook's info parameter across SDKs, using the union of SDK field sets as the reference schema.
Design: each handler logs one single-line JSON record per hook event — a canonical camelCase dump of that hook's own info. Unexposed fields are omitted, so a missing key = a failed assertion = an SDK parity gap. Maps →
<name>Count; timestamps → ISO-8601 regex; presence-only probes via/.*/; key-absence guards via/.*/+count: 0. Payload values (execution input/result, operation result) are not part of the asserted schemas. 10-14 on main is untouched.subType,isReplayingChildrenLive validation (us-west-2)
isFirstInvocationon end infoisReplayisReplayingChildrenEvery red is a filed SDK gap: aws-durable-execution-sdk-python#617, aws-durable-execution-sdk-js#803, aws-durable-execution-sdk-java#604. Full 10-1..10-18 regression green everywhere.
Land this first — the companion handler PRs (python#615, js#800, java#602) reference these IDs.