feat: add itemNamer for map operation - #576
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
dc25639 to
3eec842
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
3eec842 to
58be1d7
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
| public NestingType nestingType; | ||
| private NestingType nestingType; | ||
| private Integer maxConcurrency; | ||
| private CompletionConfig completionConfig; |
There was a problem hiding this comment.
This narrows MapConfig.Builder.nestingType from public to private, an unrelated source-incompatible change bundled into this feature PR. Any external code that read/wrote the field directly (MapConfig.builder().nestingType = ...) will fail to compile against the new SDK. AGENTS.md states "Do not change public API signatures ... without instruction."
Encapsulating the field is the right long-term design (it matches how the other builder fields and ParallelConfig are declared), but since it is a breaking change orthogonal to itemNamer, consider either splitting it into its own change with a note in the changelog/release notes, or explicitly calling it out in the PR description so it is a deliberate, documented break rather than an incidental one.
There was a problem hiding this comment.
Correct, and reverted in 0d16426. Confirmed against origin/main, which has public NestingType nestingType; — this branch narrowed it to private. That was incidental tidying on my part with no connection to itemNamer, and you are right that it breaks already-compiled clients. Restored to public; whether that field should be public at all is a separate call, not something this PR should decide as a side effect.
Two other findings from the same round were raised outside this thread, so recording the outcome here for traceability:
Iteration names resolved after START (fixed in the same commit). execute() sends the START checkpoint and calls addAllItems() on the next line, so the ParameterValidator call added in d61dd05 threw after the map was STARTED and its start plugin hook had fired. If the caller caught the IllegalArgumentException, a permanently STARTED map and an unbalanced plugin lifecycle were left behind. Name resolution and validation now happen in the MapOperation constructor, before any checkpoint. Added testCaughtItemNamerValidationLeavesNoHalfOpenMap, which swallows the failure and asserts the rejected map was never checkpointed and that replay still succeeds — verified non-vacuous by running it against d61dd05, where it fails with failed map should not be checkpointed ... but was: TestOperation@f438904.
Parameterizing MapConfig by input type — not done here, deliberately. The WaitForConditionConfig<T> precedent is real and the ergonomic complaint is legitimate; casting Object for domain items is poor. Two reasons I have left it out of this PR rather than silently doing it: MapConfig appears in 28 files, so it turns a focused feature change into a public-API reshape reviewed under the wrong heading; and it sits in tension with the visibility finding above, where the standard applied was to avoid incidental API changes in a feature PR. Worth noting the change is less breaking than it appears — MapConfig<I> erases to MapConfig, so the four DurableContext.map(...) overloads stay binary compatible and existing source keeps compiling with raw-type warnings. Happy to do it as a follow-up, or here if you would rather it land together.
1112 sdk tests and 393 integration tests pass; spotless:check clean project-wide.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Adds itemNamer functionality to MapConfig, allowing custom naming of map
iterations. This feature provides parity with the Python SDK implementation
while adapting to Java's existing API patterns.
Changes:
- Add itemNamer: BiFunction<Object, Integer, String> field to MapConfig
- Update MapOperation to use custom iteration names when itemNamer is provided
- Add comprehensive unit tests for both MapConfig and MapOperation
- Maintain backward compatibility (all existing tests pass)
Usage:
MapConfig config = MapConfig.builder()
.itemNamer((item, index) -> "process-" + item + "-iteration-" + index)
.build();
Resolves: #528
MapConfig.itemNamer makes requirement 9-13 (map with a custom item namer) satisfiable for Java, so drop its NotImplemented declaration and add the handler plus template resource. Validated end to end against us-west-2: the java map suite reports 9-13 PASSED with 15 PASSED / 5 NOT_IMPLEMENTED / 0 FAILED, and the recorded history carries the custom names (Map "named-items", iterations "item-1" and "item-2"), matching the JS and Python results exactly.
Spotless: java.util.function.BiFunction was wedged between the two org.slf4j imports, and MapConfig plus both new test files were missing trailing newlines. Fixed via spotless:apply; mvn spotless:check now passes. Docs: the getter and builder Javadoc claimed the default iteration name is "map-item-N". The actual default from MapOperation.addUnskippedItems is "<mapName>-iteration-N". Corrected both occurrences. Tests: the existing unit tests only covered the config POJO and MapOperation construction, so nothing executed the naming path. Added LocalDurableTestRunner integration tests that run a map and assert the custom names reach the child operations and survive replay without re-execution, plus a test pinning the default naming so the Javadoc and runtime cannot drift again. Verified the new test is discriminating by mutation: with the itemNamer branch disabled it fails on "custom iteration name should be checkpointed". Also pinned the duplicate-name and null-name behavior surfaced in review. Both are tolerated and results stay correct, but a null from the namer is passed through as the operation name rather than falling back to the default, and duplicates make name-based lookup ambiguous.
…e custom names A null from the itemNamer previously became the iteration's operation name, suppressing the long-standing "<mapName>-iteration-N" default and leaving the iteration unaddressable by name. It now falls back to the default, per item rather than all-or-nothing. Custom names also bypassed the validation that DurableContextImpl.mapAsync applies to the map's own name, so an empty, over-long or non-ASCII name reached the backend and failed at checkpoint time in the cloud. The namer result is now passed through ParameterValidator.validateOperationName, which fails fast at map time. Confirmed by mutation that an empty name previously succeeded. Javadoc on both the getter and the builder now documents the null fallback, the name constraints, and the determinism requirement that replay's name comparison imposes.
…es before START Builder.nestingType was public on main and this branch had narrowed it to private, which breaks source and binary compatibility for already-compiled clients. Restored to public; narrowing it is a separate decision, not a side effect of adding itemNamer. Iteration names are now resolved and validated in the MapOperation constructor instead of in addUnskippedItems. addUnskippedItems runs immediately after the map checkpoints START and fires its start plugin hook, so a namer that threw or produced an invalid name left a permanently STARTED map with an unbalanced plugin lifecycle whenever the caller caught the IllegalArgumentException. Resolving in the constructor means the failure happens before any checkpoint. Added testCaughtItemNamerValidationLeavesNoHalfOpenMap, which swallows the validation failure and asserts the rejected map was never checkpointed and that replay still succeeds. Verified against the previous commit, where it fails with "failed map should not be checkpointed ... but was: TestOperation".
MapConfig is now MapConfig<I> with Builder<I>, and itemNamer is typed
BiFunction<I, Integer, String> instead of BiFunction<Object, Integer, String>,
following the existing WaitForConditionConfig<T> precedent. Callers naming
domain objects no longer need a cast:
MapConfig.<Order>builder().itemNamer((order, i) -> "order-" + order.id())
The map APIs accept MapConfig<? super I> rather than MapConfig<I>. This matters
for source compatibility: MapConfig.builder().build() in an argument position
infers MapConfig<Object>, which does not match MapConfig<I>, so the stricter
signature broke every existing inline call site at compile time. The
consumer-position wildcard accepts both MapConfig<Object> from existing code and
an explicitly typed MapConfig<Order>. Confirmed by leaving all pre-existing call
sites in examples/ and conformance-tests/ untouched; they still compile.
MapConfigItemNamerTest declarations moved from the raw MapConfig type to
MapConfig<Object>, since a raw type erases itemNamer()'s return and made apply()
yield Object. Added coverage for the cast-free domain-typed path at both the
config level and end to end through map().
Not related to itemNamer. PR #574 (a46235f) merged plugin conformance handlers that spotless had never checked, because check-spotless.yml filters on sdk/**, sdk-testing/**, sdk-integration-tests/** and examples/** while running spotless:check project-wide via --file pom.xml. A PR touching only conformance-tests/** therefore never triggers the job, but the next PR that touches sdk/** inherits the failure. This is the second time on this branch: 570a195 left the same situation and was resolved by rebasing because main had already been fixed. This time main is the source, so the reformat has to land here to get CI green. Reformatted with JDK 17 to match the CI runner and verified the result is stable under JDK 21, so it will not flip-flop between local and CI runs. Only the six inherited files changed; no sdk/ or sdk-integration-tests/ file was touched.
a9f68ac to
4c94a18
Compare
When a completed map's result is small enough to replay from the checkpoint payload, its children are enqueued but never executed, so the per-operation validateReplay name comparison never runs. A namer whose output changed between invocations therefore replayed silently under the old checkpointed names. MapOperation now compares the regenerated iteration names against the checkpointed child operations in the SUCCEEDED replay path and raises NonDeterministicExecutionException on mismatch, matching how the per-operation guard reports a name mismatch. Added ExecutionManager.peekOperation for this: the existing getOperationAndUpdateReplayState transitions REPLAY to EXECUTION as a side effect when the looked-up operation is absent or non-terminal, which would corrupt replay state if used for read-only validation. Tests: testChangedItemNamerOnCachedReplayIsRejected covers the reported gap, and testStableItemNamerReplaysCleanlyFromCache is the control proving an unchanged namer still replays from cache. Verified by mutation that removing the new call makes the first test fail with "expected: not equal but was: <SUCCEEDED>", confirming the gap was real and that the guard is what closes it. This also supersedes the caveat noted earlier on this PR, where a non-deterministic namer was observed replaying without the guard firing.
Codex AI review
Reviewed commit |
Claude AI reviewReview: feat: add itemNamer for map operation (PR #576)No actionable findings. This is a clean, well-tested change. I traced the full path — Verified correct:
Test coverage is comprehensive across config, construction, typed items, null/partial-null fallback, invalid-name rejection, duplicate names, changed-vs-stable namer on cached replay, and half-open recovery. Residual test risk (disclosed; read-only review, no build/run):
Reviewed commit |
Description
Adds
itemNamerfunctionality toMapConfig, allowing custom naming of map iterations. This feature provides parity with the Python SDK implementation (PR #387) while adapting to Java's existing API patterns.Changes
MapConfig.java:
itemNamer: BiFunction<Object, Integer, String>fielditemNamer()methoditemNamer()configurationMapOperation.java:
itemNameris providedUnit Tests:
MapConfigItemNamerTest.java: 9 comprehensive tests covering all itemNamer functionalityMapOperationItemNamerTest.java: 3 integration tests for MapOperation constructionUsage
Testing
Notes
branch(String name, ...), so noParallelBranchwrapper was needed (unlike Python)