[feat] PIP-468: V5 end-to-end encryption API redesign#25682
Open
merlimat wants to merge 1 commit intoapache:masterfrom
Open
[feat] PIP-468: V5 end-to-end encryption API redesign#25682merlimat wants to merge 1 commit intoapache:masterfrom
merlimat wants to merge 1 commit intoapache:masterfrom
Conversation
…verage Reshape the V5 end-to-end encryption surface to make first-time setup straightforward and to make misconfiguration harder to express: - Split the dual-role CryptoKeyReader into role-specific PublicKeyProvider (producer side) and PrivateKeyProvider (consumer side) interfaces. Both signatures are async (CompletableFuture<EncryptionKey>) so KMS- backed implementations don't need to block. CryptoKeyProvider extends both for convenience when the same store serves both sides. - Split the single CryptoFailureAction into ProducerCryptoFailureAction (FAIL, SEND_UNENCRYPTED) and ConsumerCryptoFailureAction (FAIL, DISCARD, CONSUME). Compile-time prevents producer-side DISCARD/CONSUME misconfigurations. - Split EncryptionPolicy into ProducerEncryptionPolicy and ConsumerEncryptionPolicy, each with its own builder. ConsumerEncryption Policy lets the user skip the privateKeyProvider when the failure action is DISCARD or CONSUME — matching v4's "no key reader, just receive raw bytes" mode. - Convert EncryptionKeyInfo to EncryptionKey (final class with private ctor + of(...) factories), avoiding records' broken byte[] equality. - Ship a batteries-included PemFileKeyProvider for the common load-from-disk case, eliminating the v4 friction of having to implement the SPI from scratch just to encrypt one message. - Internal CryptoKeyReaderAdapter now exposes role-specific forProducer / forConsumer factories that bridge to the v4 CryptoKeyReader. Documents the sync-blocking constraint imposed by v4's MessageCrypto path (orthogonal to this change). Tests: - PemFileKeyProviderTest — read registered keys, missing key / file failure modes, dual-role usage. - EncryptionPolicyTest — builder validation: required fields, default failure actions, DISCARD/CONSUME without a provider. - V5EncryptionTest — end-to-end round trip on a single segment, on a 3-segment scalable topic, and CONSUME / DISCARD action behavior with no provider configured.
merlimat
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reshape the V5 end-to-end encryption surface so first-time setup is straightforward and misconfiguration is harder to express:
CryptoKeyReaderintoPublicKeyProvider(producer side) andPrivateKeyProvider(consumer side). Both returnCompletableFuture<EncryptionKey>so KMS-backed implementations don't have to block.CryptoKeyProviderextends both for in-process use.CryptoFailureActionintoProducerCryptoFailureAction(FAIL,SEND_UNENCRYPTED) andConsumerCryptoFailureAction(FAIL,DISCARD,CONSUME). Compile-time prevents producer-side DISCARD/CONSUME misconfigurations.EncryptionPolicyintoProducerEncryptionPolicyandConsumerEncryptionPolicy. The consumer policy accepts a nullprivateKeyProviderwhen the failure action isDISCARDorCONSUME— matching v4's "no key reader, just receive raw bytes" mode.EncryptionKeyInfo→EncryptionKey(final class, private ctor +of(...)factories). Avoids records' brokenbyte[]equality.PemFileKeyProviderfor the common load-from-disk case. Eliminates v4's friction of having to implement the SPI from scratch just to encrypt one message.CryptoKeyReaderAdapterexposes role-specificforProducer/forConsumerfactories that bridge to v4's combinedCryptoKeyReader.Test plan
PemFileKeyProviderTest(5 unit tests) — registered keys, missing key, missing file, dual-role usage.EncryptionPolicyTest(9 unit tests) — builder validation, defaults, optional consumer provider for DISCARD/CONSUME.V5EncryptionTest(4 end-to-end tests):CONSUMEwithout a provider delivers raw encrypted bytes.DISCARDwithout a provider silently drops the message.pulsar-client-api-v5,pulsar-client-v5,pulsar-brokercheckstyle clean.