Add a challenge-driving AUTH pillar step for end-to-end Basic and Digest authentication

[OmarAlJarrah]

Summary

sdk-core ships a complete, spec-mandated (AUTH-12..26), unit-tested WWW-Authenticate challenge subsystem — AuthChallengeParser, BasicChallengeHandler, DigestChallengeHandler, CompositeChallengeHandler, and the ChallengeHandler strategy interface — but no shipped AUTH-pillar step ever drives a 401 challenge through a ChallengeHandler into a rebuilt request. The two concrete steps that occupy the AUTH pillar (BearerTokenAuthStep/AsyncBearerTokenAuthStep and KeyCredentialAuthStep) never touch a ChallengeHandler. The handler classes are public and tested, but nothing wires them into the pipeline.

Ship a ChallengeAuthStep (sync + async) that occupies the single AUTH pillar, optionally stamps an initial credential, and on a 401 + WWW-Authenticate parses the header via AuthChallengeParser, delegates to a caller-supplied ChallengeHandler, and returns the rebuilt request through the existing replayability-gated challenge path. Include a worked example wiring Basic and Digest.

Problem

This SDK is a platform: its consumers are external generated service clients, downstream SDKs, and application code assembled on top of the pipeline primitives. The challenge subsystem is offered as building blocks, but the pillar steps the SDK ships to make those blocks usable (BearerTokenAuthStep, KeyCredentialAuthStep) drive only their own schemes. The only route a consumer has to Basic or Digest today is to hand-roll their own AuthStep/AsyncAuthStep subclass and wire the parser and handler into the protected challenge hook themselves — an undocumented extension pattern, untested, that every consumer and every generated client would have to reinvent identically for a capability the SDK advertises as shipped.

Concretely today:

• Digest has no shipped driver. DigestChallengeHandler (sdk-core/src/main/kotlin/org/dexpace/sdk/core/auth/DigestChallengeHandler.kt) fully implements RFC 7616 (MD5/MD5-sess/SHA-256/SHA-256-sess, per-nonce nc, SecureRandom cnonce — AUTH-15..22), and README.md:136 advertises "auth covers KeyCredential, cached BearerToken, and RFC 7616 Digest" as a shipped feature. But no step calls it. A consumer targeting a Digest-protected origin gets no turnkey path — only a subclass they must write and maintain against the pillar's protected contract.
• The ChallengeHandler strategy has no in-tree driver. No shipped pipeline step or transport calls ChallengeHandler.handleChallenges. The only in-tree invocations are CompositeChallengeHandler delegating to a wrapped handler (sdk-core/src/main/kotlin/org/dexpace/sdk/core/auth/CompositeChallengeHandler.kt:37) and the interface's own convenience-overload self-delegation (ChallengeHandler.kt:71). architecture.md:225-226 lists only BearerTokenAuthStep, KeyCredentialAuthStep, and AuthStep as the credential-stamping steps — there is no challenge-driving counterpart.
• BearerTokenAuthStep's challenge hook is bearer-specific. Its authorizeRequestOnChallenge (BearerTokenAuthStep.kt:123) evicts and re-fetches a bearer token and explicitly bails on any non-Bearer challenge (offersBearerChallenge, BearerTokenAuthStep.kt:132, 221). It is not a general challenge driver and cannot be repurposed as one.
• ProxyOptions.challengeHandler is a dead slot. The field exists (sdk-core/src/main/kotlin/org/dexpace/sdk/core/util/ProxyOptions.kt:69) but both transports actively refuse it: OkHttp warns-and-ignores at OkHttpTransport.kt:537, the JDK transport at JdkHttpTransport.kt:548. So even the one place a ChallengeHandler is surfaced in a public API is inert.

Because the challenge subsystem is a core primitive, this gap is platform-wide: every generated client and downstream SDK built on the pipeline inherits it, and none can reach Basic/Digest through a supported, tested step — they can only replicate the same bespoke subclass.

Proposed approach

Add ChallengeAuthStep and AsyncChallengeAuthStep under org.dexpace.sdk.core.http.pipeline.steps, as concrete subclasses of the existing AuthStep / AsyncAuthStep pillar bases. This adds no new runtime dependency — the parser and handlers already live in sdk-core and use only JDK stdlib (java.util.Base64, java.security.MessageDigest, java.security.SecureRandom), so the near-zero-dep core constraint is preserved. Nothing about this belongs in an adapter module.

Construction takes a ChallengeHandler (single handler, or a composite via ChallengeHandler.of(...)) and an optional initial Credential/handler-derived header to stamp pre-emptively:

• authorizeRequest (sync) / authorizeRequestAsync (async): if an initial credential is configured, stamp it; otherwise pass the request through unstamped (many Digest servers expect a bare first request and answer 401). The existing HTTPS guard (AUTH-28) and cross-origin suppression (AUTH-29) in the base already gate the stamping path.
• authorizeRequestOnChallenge (AuthStep.kt:145) / authorizeRequestOnChallengeAsync (AsyncAuthStep.kt:135): parse the WWW-Authenticate header with AuthChallengeParser.parse(...), call handler.handleChallenges(request.method, request.url.toURI(), challenges, isProxy = false), and if it returns an AuthorizationHeader, rebuild the request with that header set and return it. Returning null (handler can't satisfy any offered challenge) surfaces the 401 unchanged.

Reusing the base hooks means the step inherits the entire pillar contract for free: the single-re-drive-with-fresh-cursor semantics (AUTH-30, PIPE-15), the replayability gate on the sync path (AuthStep.kt:115, AUTH-31), close-out of the superseded 401 (AUTH-32, PIPE-40), and pass-through of a 401 that carries no WWW-Authenticate (AUTH-33). Scope is deliberately narrow: this step reacts only to 401 + WWW-Authenticate (origin auth), matching the base pillar; proxy 407/Proxy-Authenticate is out of scope (see non-goals).

Ship the worked example in the existing sdk-example module: one pipeline wiring BasicChallengeHandler, one wiring DigestChallengeHandler, and one wiring both via ChallengeHandler.of(digest, basic) (Digest first, per AUTH-23), so the strategy classes are discoverable from a runnable sample rather than only from tests.

Scope and non-goals

In scope

• ChallengeAuthStep (sync) and AsyncChallengeAuthStep (async) as AUTH-pillar steps.
• Optional initial-credential stamping; challenge-driven rebuild on 401 + WWW-Authenticate.
• A runnable Basic/Digest/composite example in sdk-example.
• apiDump regeneration for the new public types.

Non-goals

• Proxy authentication (407 / Proxy-Authenticate). The base pillar reacts to 401 only, and neither transport wires a custom ChallengeHandler into its proxy-auth path: OkHttp's proxyAuthenticator only emits Basic from ProxyOptions.username/password (OkHttpTransport.kt:523), and java.net.http.HttpClient exposes no per-407 hook at all (JdkHttpTransport.kt:535). Reviving ProxyOptions.challengeHandler is a separate track.
• Changing the async replayability behaviour. AsyncAuthStep does not currently apply the sync path's replayability gate (noted in AUTH-31); AsyncChallengeAuthStep inherits whatever the base does today and does not attempt to close that gap here.
• Multi-round challenge negotiation. The pillar drives exactly one re-challenge (AUTH-30); this step keeps that.
• Any new third-party runtime dependency in sdk-core.

Acceptance criteria

• ChallengeAuthStep and AsyncChallengeAuthStep exist under http.pipeline.steps, extend AuthStep/AsyncAuthStep, and occupy Stage.AUTH (one per pipeline, AUTH-27).
• With no initial credential, a first request is sent unstamped; a 401 + WWW-Authenticate is parsed via AuthChallengeParser, routed through the supplied ChallengeHandler, and the returned AuthorizationHeader is set on a single rebuilt re-drive.
• A handler returning null (no satisfiable challenge) surfaces the original 401 unchanged; a 401 without WWW-Authenticate passes through without consulting the handler (AUTH-33).
• Basic (Authorization: Basic base64(user:pass), AUTH-14) and Digest (RFC 7616 response, AUTH-15..22) both succeed end-to-end against a mockwebserver3 server that answers 401 then 200, exercised sync and async.
• A composite (ChallengeHandler.of(digest, basic)) offered both schemes answers with Digest (AUTH-23).
• The 401 body is closed on re-drive and on a hook throw (AUTH-32, PIPE-40); a non-replayable body on the sync path skips the replay and leaves the original 401 unclosed (AUTH-31).
• sdk-example gains a runnable sample wiring Basic, Digest, and the composite.
• ./gradlew build is green, including apiCheck after apiDump, ktlint/detekt, and the coverage floor.

References

• Challenge subsystem (already shipped): AUTH-12, AUTH-13 (parser), AUTH-14 (Basic), AUTH-15–AUTH-22 (Digest), AUTH-23, AUTH-24, AUTH-25 (composition / handler contract), AUTH-26 (key-credential stamping) — docs/product-spec.md.
• AUTH pillar contract this step inherits: AUTH-27 (single AUTH step), AUTH-28 (HTTPS guard), AUTH-29 (cross-origin suppression), AUTH-30 (401 + WWW-Authenticate → hook → single fresh-cursor re-drive), AUTH-31 (replayability gate; note the async gap), AUTH-32 (close 401 on hook throw), AUTH-33 (401 without WWW-Authenticate passes through), AUTH-38 (async errors via failed future).
• Re-drive discipline: PIPE-15 (fork a fresh cursor per re-drive), PIPE-40 (release superseded responses, never close the returned one).
• Code: ChallengeHandler.kt, AuthChallengeParser.kt, BasicChallengeHandler.kt, DigestChallengeHandler.kt, CompositeChallengeHandler.kt (sdk-core/.../auth/); AuthStep.kt:145, AsyncAuthStep.kt:135, BearerTokenAuthStep.kt:123, KeyCredentialAuthStep.kt (sdk-core/.../http/pipeline/steps/); ProxyOptions.kt:69; OkHttpTransport.kt:537; JdkHttpTransport.kt:548; README.md:136; docs/architecture.md:225-226.