feat: load plugins dynamically from Lambda layers

[zhongkechen]

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Issue Link, if available

Closes #605

Description

• Add a versioned DurableExecutionPluginProvider SPI discovered with ServiceLoader when DURABLE_EXECUTION_PLUGINS is set.
• Load environment-selected plugins first in configured order, followed by plugins registered through DurableConfig.withPlugins(...); both sources remain additive.
• Add otel-invocation and otel-execution providers for InvocationOtelPlugin and ExecutionOtelPlugin.
• Validate provider names, duplicates, API compatibility, plugin types, and creation failures with actionable configuration errors.
• Document Lambda layer packaging, provider authoring, selector names, and OTel layer configuration.

Demo/Screenshots

Not applicable; this is SDK configuration and plugin discovery behavior.

Checklist

• I have filled out every section of the PR template
• I have thoroughly tested this change

Testing

Unit Tests

Yes. Added loader coverage for ordering, additive duplicate types, allow-list behavior, malformed configuration, discovery failures, compatibility validation, and plugin creation failures. Added provider tests for both OTel plugins.

Integration Tests

Yes. Added lifecycle coverage confirming dynamically loaded and explicitly configured plugins both receive events.

Also ran mvn clean install successfully across all reactor modules.

Examples

No new example was required. The configuration guide and OTel plugin README now include Lambda layer and environment-variable examples.

[github-actions]

Codex AI review

[P1] Package OTel runtime dependencies in the layer — otel-plugin/README.md:91

The published OTel JAR is thin, but the instructions package only that JAR. Dynamic creation requires its OpenTelemetry SDK/exporter dependencies, which the ADOT agent does not expose to the application class loader, causing cold-start linkage failures. Package the runtime dependency closure in java/lib or publish a complete layer artifact.

[P2] Avoid rediscovering plugins when copying configuration — sdk/src/main/java/software/amazon/lambda/durable/DurableConfig.java:106

LocalDurableTestRunner copies getPluginRunner().getPlugins() into a new builder. Those already include dynamic plugins, and build() now loads them again, so hooks run twice; repeated runner configuration can add more copies. Preserve explicit and dynamic sources separately or provide a copy path that disables rediscovery, with an integration test using actual environment discovery.

[P3] Reject provider names containing commas — sdk/src/main/java/software/amazon/lambda/durable/DynamicPluginLoader.java:106

A provider name containing , passes validation but can never be selected because configuration is split on commas. Reject delimiter-containing names and add a validation test.

Reviewed commit 38f78ef81ab7a4d20578fbd529c2524391a88e54. Workflow run

[SilanHe]

nit: This comment is wrong. If we're returning a noOp runner then it means that no plugins were loaded dynamically

[zhongkechen]

if env var is empty and plugins are specified in DurableConfig.Builder, a real runner will be created.

[github-actions]

Claude AI review

Review: feat: load plugins dynamically from Lambda layers

I reviewed the full SHA-anchored diff (DynamicPluginLoader, the new DurableExecutionPluginProvider SPI, the two OTel providers + service file, the DurableConfig wiring, and the added tests/docs) against the base revision.

No actionable correctness, compatibility, or lifecycle findings. The loader is solid on the points that matter here:

• Ordering & additivity (DynamicPluginLoader.java:437-442) match the documented contract — dynamic plugins first in DURABLE_EXECUTION_PLUGINS order, then explicit withPlugins(...), both additive — and are covered by tests.
• DurableConfig wiring (DurableConfig.java:116 → plugins.isEmpty()) is behavior-preserving when the env var is unset: loadConfiguredPlugins returns List.copyOf(builder.plugins), so the no-op-runner path is unchanged.
• Validation is thorough: empty/duplicate names, unknown providers, API-version mismatch, non-concrete plugin types, declared-vs-actual type mismatch, null plugins, and discovery/creation failures all fail fast with clear IllegalStateException messages, and each branch has a unit test.
• Error handling correctly distinguishes wrapping bands — ServiceConfigurationError | LinkageError during discovery, RuntimeException | LinkageError during version/type/creation calls — and the getName()-thrown IllegalStateException correctly propagates without being re-wrapped by the discovery catch.
• The OTel providers declare distinct names (otel-invocation, otel-execution), so the shared META-INF/services file won't trip the duplicate-name check.
• The cross-module package-private access from PluginIntegrationTest (same package software.amazon.lambda.durable, unnamed-module classpath) is valid.

Residual risk (non-blocking)

1. Production env-var path is not exercised end-to-end. Tests only call the injectable 3-arg loadConfiguredPlugins(String, Iterable, List) overload (including the integration test). The actual runtime path — loadConfiguredPlugins(List) reading System.getenv(DURABLE_EXECUTION_PLUGINS), resolving the TCCL, and running a real ServiceLoader through DurableConfig construction — has no direct test. The OTel ServiceLoader registration tests cover discovery in isolation, but not the env-var → DurableConfig wiring. Consider an integration test that sets the env var (or drives the 1-arg overload) so the getenv + classloader-fallback + ServiceLoader wiring is regression-protected.

2. Discovery is fail-closed across all providers, not just requested ones (DynamicPluginLoader.java:463-486). indexProviders eagerly instantiates and calls getName() on every discovered provider. If an unrelated provider JAR on the class path (e.g. from a second Lambda layer) fails to link or throws in getName(), all DurableConfig construction fails even when DURABLE_EXECUTION_PLUGINS names only a healthy provider — which slightly undercuts the "only named providers are used" isolation stated in the SPI Javadoc. This appears intentional (the PR lists discovery failures as a deliberate stop condition), so I'm flagging it as risk rather than a defect; if stronger isolation is desired later, resolving names lazily per requested provider would contain the blast radius.

Reviewed commit 38f78ef81ab7a4d20578fbd529c2524391a88e54. Workflow run