Conversation
alanshaw
left a comment
alanshaw
left a comment
There was a problem hiding this comment.
👍 I'm fine with this, for the reason of switching to PEM is mildy better from the standpoint of developer familiarity - I'm not convinced there's really any concrete advantages beyond that, which the existing format doesn't already support.
| - PEM files (with inconsistent encryption practices). | ||
| - JSON files (non-standard, service-specific schemas). | ||
| - Base64 encoded strings (raw key material, no metadata). | ||
| - Multibase base64 padded strings (uncommon in standard tooling). |
There was a problem hiding this comment.
Ok a few observations:
- I believe you introduced PEM.
- I don't know if we've ever actually used JSON to pass around private keys although I know the w3 CLI has the option to print them as JSON.
- I don't believe we've ever passed plain base64 encoded strings around but maybe I'm incorrect.
So there was only really the 1 format prior - for which some of the reasons below don't apply.
I'm not trying to oppose the proposal here but I think if this was phrased more as "switch from multibase base64 padded private keys to PEM" then I think it would be more accurate.
There was a problem hiding this comment.
You're absolutely right on all points - thank you for the clarification.
I don't believe we've ever passed plain base64 encoded strings around but maybe I'm incorrect.
You're correct - looking at my shell history, we've been using multibase base64 padded strings, not plain base64.
I believe you introduced PEM
Yes, I did introduce PEM to the storage node. The current state is:
storage id genreturns JSON with a DID and multibase-base64-padded private key.- I added
--type=pemoption for PEM output (needed for Curio's API authentication). - I added
storage id parseto convert between formats, and to extract DIDs from PEM files.
The PEM format was specifically introduced to work with Curio's API, which requires PEM-formatted public keys for authentication.
I think if this was phrased more as "switch from multibase base64 padded private keys to PEM" then I think it would be more accurate
Yeah, the accurate framing is probably: "Switch from multibase base64 padded private keys to PEM". JSON is only used as an output wrapper in some cases, not as a key format itself.
It's worth noting that at the end of this doc the need for custom tooling to extract DIDs from PEM files is outlined. Curious what your thoughts are on this.
|
|
||
| ### Why PEM? | ||
|
|
||
| 1. **Universal tooling support**: OpenSSL, ssh-keygen, and all major crypto libraries support PEM natively. |
There was a problem hiding this comment.
Yes, but typically we want to display the DID at the same time, so we have to dip back into our own tooling anyways.
There was a problem hiding this comment.
| ### Why PEM? | ||
|
|
||
| 1. **Universal tooling support**: OpenSSL, ssh-keygen, and all major crypto libraries support PEM natively. | ||
| 2. **Self-documenting format**: Clear headers (`-----BEGIN PRIVATE KEY-----`) identify content type. |
There was a problem hiding this comment.
🤷 the multibase encoded string is also self describing - the decoded bytes are also prefixed with the alogorithm. PEM has the advantage of being human readable to some extent.
There was a problem hiding this comment.
My main concerns with multibase for private key material center around tooling, file conventions, and security.
-
Most common cryptographic tools (OpenSSL, ssh-keygen, keytool, etc.) have native support for PEM format when working with private keys. With multibase-encoded keys, we typically need custom tooling or conversion steps for common operations like key generation, inspection, or format conversion.
-
PEM has well-established conventions -
.pemfiles, standard permissions (600), and clear practices around key storage. With multibase strings, we'd need to define our own conventions: What file extension? Plain text files? JSON wrapper? -
PEM has built-in support for passphrase encryption. For multibase, we'd need to implement our own encryption scheme or wrap it in another format, essentially recreating what PEM already provides
At the end of the day, specifically for the storage node, I'd prefer to maintain key material in a file, rather than a string passed over the CLI or environment variable.
|
|
||
| 1. **Universal tooling support**: OpenSSL, ssh-keygen, and all major crypto libraries support PEM natively. | ||
| 2. **Self-documenting format**: Clear headers (`-----BEGIN PRIVATE KEY-----`) identify content type. | ||
| 3. **Algorithm agnostic**: Supports our current Ed25519 keys and future algorithms (RSA, ECDSA, etc.). |
There was a problem hiding this comment.
Again, same for existing format.
| 2. **No environment variables**: Private keys MUST NOT be passed via environment variables. | ||
| 3. **Encryption at rest**: Keys SHOULD be encrypted when stored. | ||
| 4. **Memory handling**: Keys SHOULD be loaded once at startup, not repeatedly read. | ||
| 5. **Secure deletion**: Key material SHOULD be securely overwritten when removed from memory. |
There was a problem hiding this comment.
We have historically used AWS SSM for deployed services and would continue to do so regardless of encoding (I understand this is not the same situation for storage nodes).
| ## DID Derivation from Private Keys | ||
|
|
||
| Users often need to derive a DID (Decentralized Identifier) from the private key for identity purposes. This section briefly covers how to extract the DID from a PEM-encoded private key. | ||
|
|
||
| ### What is a DID and How do we Encode ours? | ||
|
|
||
| A DID is a decentralized identifier that corresponds to your public key. In our system, Ed25519 keys produce DIDs in the format: `did:key:...` where the suffix is a Base58BTC-encoded representation of the public key with multicodec prefixes. | ||
|
|
||
| ### Deriving DID from PEM | ||
|
|
||
| At present, there does not exist any standard tooling to derive a DID used by our system from a PEM File containing a private key, or public key for that matter. This necessitates the creation of a custom CLI tool which allows users to derive a DID string from their public key. |
There was a problem hiding this comment.
@alanshaw assuming we align on PEM as our mechanism for handling key material, curious to hear your thoughts on these bits.
2 participants
📘 Preview