diff --git a/docs/maintain-vault-connection.md b/docs/maintain-vault-connection.md new file mode 100644 index 00000000000..96ba78238c4 --- /dev/null +++ b/docs/maintain-vault-connection.md @@ -0,0 +1,115 @@ +# Maintain the Vault connection + +Installation of the [keyring vault component](use-keyring-vault-component.md) is the first step. The most common production failure of `keyring_vault` is a lost connection to Vault, not a misconfigured component. Two conditions cause the loss: + +* The token used by the component expires. + +* The Vault server seals. + +Either condition prevents Percona Server from fetching master keys. + +The `keyring_vault` component reads the `token` value from the configuration file at startup. The component then uses the stored token for the lifetime of the `mysqld` process. The component does not renew the token and does not reread the configuration file at runtime. Renewal and rotation must run outside MySQL, on the Vault side or through a helper process. + +## Renew the Vault token + +Every Vault token has a time-to-live (TTL). When the TTL elapses, Vault revokes the token. Any subsequent Application Programming Interface (API) call returns HTTP 403, including the master-key fetch performed by `keyring_vault`. + +The following symptoms indicate an expired or revoked token: + +* `mysqld` startup fails after a restart with keyring component initialization or fetch errors. + +* The statement `ALTER INSTANCE ROTATE INNODB MASTER KEY` fails. + +* Opening an encrypted table after a restart fails with a keyring error. + +* The row in `performance_schema.keyring_component_status` shows the component as loaded but unable to reach Vault. + +Because the component does not renew tokens, choose one of the following patterns to keep the configured token valid: + +### Option 1 (recommended): Periodic tokens + +A [periodic token :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/concepts/tokens#periodic-tokens) has no maximum TTL. The token continues to work indefinitely as long as a renewal occurs within each period. This renewal model matches the lifetime of a long-running database server. + +Create a Vault role that issues periodic tokens, then mint the token referenced by `component_keyring_vault.cnf`: + +```bash +vault write auth/token/roles/percona-keyring \ + allowed_policies="percona-keyring-policy" \ + period="24h" \ + renewable=true + +vault token create -role=percona-keyring -format=json +``` + +Store the issued token in the `token` field of the `keyring_vault` configuration file. Pair the periodic token with a renewer (see Option 2) so a renewal occurs at least once per `period`. + +### Option 2: Sidecar renewer + +A sidecar process must call `vault token renew` before each expiry. The requirement applies whether the token is periodic or has a fixed maximum TTL. The following table compares two common implementations: + +| Implementation | Description | +|---|---| +| `vault agent` | Configure `vault agent` with an `auto_auth` block. The agent authenticates, writes the issued token to a sink, and renews the token automatically. Configure `component_keyring_vault.cnf` to read a token issued by the same auth method, or generate the configuration with agent templating. | +| `systemd` timer or cron job | Schedule `vault token renew` at an interval shorter than the token TTL, for example hourly for a 24-hour token. Run the job under a dedicated service account, not as `root`, and forward renewal failures to the alerting system. | + +Configure alerts on renewal failure. Without alerts, a failed renewer behaves the same as no renewer at all. The failure surfaces only at the next `mysqld` restart, when startup fails. + +### Operational checklist + +* Treat the `token` value as a secret equal in sensitivity to the Vault unseal keys. Restrict file permissions on `component_keyring_vault.cnf` to the `mysql` user. + +* Monitor the token TTL with `vault token lookup`. Alert when the remaining TTL drops below a safe threshold. + +* Rotate the token only during a maintenance window. Update the configuration file, then restart `mysqld`. The component reads the file only at startup. + +* Assign one token to one server. The one-token-per-server rule complements the `secret_mount_point` warning in [Configure the keyring vault component](use-keyring-vault-component.md#configure-the-keyring-vault-component). + +## Handle a sealed Vault + +The `keyring_vault` component calls the Vault HTTP API at startup and during every key operation. A [sealed Vault :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/concepts/seal) returns HTTP 503 for any request to the secrets engine. The component cannot read or write keys against a sealed Vault, even with a valid token. + +Vault becomes sealed in the following situations: + +* A host reboot or package upgrade restarts the Vault process. + +* An operator runs `vault operator seal`. + +* Vault seals itself in response to a detected integrity issue. + +Percona Server behavior during a sealed Vault depends on timing: + +| State | Behavior | +|---|---| +| `mysqld` running, keys cached in memory | Reads and writes against the open encrypted tables continue to succeed. | +| `mysqld` running, additional key fetch required | Operations that require an additional key fetch fail. Examples include opening an encrypted table that was not already open. Other examples include rotating the master key and creating an encrypted tablespace. | +| `mysqld` restarted while Vault remains sealed | `InnoDB` cannot unwrap tablespace keys. Startup fails or encrypted tables remain inaccessible until Vault is unsealed. | + +### Recommended practice + +* Unseal Vault before starting or restarting Percona Server. Confirm that `vault status` reports `Sealed` as `false`. + +* Automate the unseal step with [auto-unseal :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/concepts/seal#auto-unseal). Use a cloud KMS, Hardware Security Module (HSM), or Transit secret engine as the backing store. A Vault restart then completes without manual intervention. A Vault reboot outside business hours blocks every Percona Server that depends on the sealed Vault. + +* Order the startup dependencies. Configure `mysqld` to depend on Vault being unsealed, not only on Vault being reachable. Apply the dependency when both processes run on the same host or under the same orchestrator. + +* Monitor the `/v1/sys/health` endpoint of Vault. The endpoint reports seal status. Alert on `sealed=true` independently of database telemetry, so operators can respond before the next `mysqld` restart. + +* Keep unseal keys and recovery keys offline and distributed across custodians. The custody practice belongs to the Vault domain rather than Percona. Lost unseal keys produce permanent data loss for every Percona Server that depends on the sealed Vault. + +## See also + +Percona Server documentation: + +* [Data at Rest Encryption](data-at-rest-encryption.md) describes how Percona Server uses the keyring to protect tablespace data. + +* [Keyring components overview](keyring-components-plugins-overview.md) compares the available keyring components. + +* [Rotate the master encryption key](rotate-master-key.md) covers the full rotation procedure and required privileges. + +* [Use the keyring vault component](use-keyring-vault-component.md) describes installation and configuration of the `keyring_vault` component. + +HashiCorp documentation: + +* [Installing Vault :octicons-link-external-16:](https://www.vaultproject.io/docs/install/index.html) + +* [Production Hardening :octicons-link-external-16:](https://learn.hashicorp.com/vault/operations/production-hardening) diff --git a/docs/use-keyring-vault-component.md b/docs/use-keyring-vault-component.md index f41cb1f834d..e79fa4359f4 100644 --- a/docs/use-keyring-vault-component.md +++ b/docs/use-keyring-vault-component.md @@ -1,117 +1,192 @@ # Use the keyring vault component -The `keyring_vault` component extends the server capabilities and provides an interface for the database with a [HashiCorp Vault :octicons-link-external-16:](https://www.hashicorp.com/products/vault/data-protection) server to store key and secure encryption keys. +The `keyring_vault` component connects Percona Server to a [HashiCorp Vault :octicons-link-external-16:](https://www.hashicorp.com/products/vault/data-protection) server. The component stores and retrieves the encryption keys that protect data at rest. --8<--- "keyring-components-installation.md" -The following example is a global manifest file that does not use local manifests: +## Configure the keyring vault component -```json -{ - "read_local_manifest": false, - "components": "file://component_keyring_vault" -} -``` +The configuration settings live in either a global configuration file or a local configuration file. -The following is an example of a global manifest file that points to a local manifest file: +The component connects to the [HashiCorp Vault server :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/install) over Hypertext Transfer Protocol Secure (HTTPS) for production deployments. The component also accepts plain `http://` URLs for development. Prepare the certificate and key files for the secure connection. Each Vault server instance requires three artifacts: -```json -{ - "read_local_manifest": true -} -``` +- An organizational Certificate Authority (CA) + +- A private vault key + +- A server certificate signed by the CA + +You can generate the artifacts with [OpenSSL :octicons-link-external-16:](https://www.openssl.org/docs/man3.0/index.html) or reuse existing files. The key files contain sensitive material. Store the key files and the password for each key in a secure location. + +You can also [build a CA inside Vault :octicons-link-external-16:](https://developer.hashicorp.com/vault/tutorials/secrets-management/pki-engine). The Vault CA can then issue the server certificate. + +### Configuration parameters + +The `component_keyring_vault.cnf` file contains required and optional parameters. + +#### Required parameters + +* `secret_mount_point` — the mount point name where `keyring_vault` stores the keys. + +* `token` — a token issued by the Vault server. + +* `vault_url` — the address of the Vault server. The address must start with `http://` or `https://`. Use `https://` for production traffic. Use `http://` only for development. The component follows HTTP redirects returned by Vault. This behavior supports cluster setups where a load balancer redirects clients to the active Vault node. + +#### Optional parameters + +The component checks the value of `read_local_config` before any other parameter. + +* `read_local_config` — Default: `false`. Declares whether the component reads the local configuration file. Use this parameter only in the global configuration file. The allowed values are `true` or `false`. + + `false` — the component processes the other parameters in the global configuration file and ignores the local configuration file. + + `true` — the component ignores the other parameters in the global configuration file and reads the local configuration file. + +* `secret_mount_point_version` — Default: `AUTO`. The Key-Value (KV) Secrets Engine version (`kv` or `kv-v2`). The allowed values are `AUTO`, `1`, and `2`. + +* `timeout` — Default: 15. The duration in seconds applied to both the connection phase and the total operation. The allowed range is 0 through 86400. Set the value to 0 to wait indefinitely. -The following is an example of a local manifest file: +* `vault_ca` — Default: unset (fallback to the host trust store). The path to the CA certificate that signed the certificate of the Vault server. Set this parameter when the host does not trust the CA of the Vault server. The component always verifies the peer certificate and the hostname when `vault_url` uses `https://`. + +The component trims leading and trailing whitespace from each parameter value before parsing. + +Run `SELECT * FROM performance_schema.keyring_component_status` to inspect the loaded configuration. The view reports each parameter value and the `Active` or `Disabled` status. The view also reports the component identity fields: `Component_name`, `Author`, `License`, `Implementation_name`, and `Version`. + +#### Example configuration + +The following example shows a `component_keyring_vault.cnf` file in JSON format with sample values for each parameter. ```json { - "components": "file://component_keyring_vault" + "timeout": 15, + "vault_url": "https://vault.public.com:8202", + "secret_mount_point": "secret", + "secret_mount_point_version": "AUTO", + "token": "{randomly-generated-alphanumeric-string}", + "vault_ca": "/data/keyring_vault_confs/vault_ca.crt" } ``` -The configuration settings are either in a global configuration file or a local configuration file. +!!! warning -The component communicates with the [Hashicorp Vault server :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/install). Prepare the certificate and key files for a secure HTTPS connection to the server. You must have an organizational Certificate Authority (CA), a private vault key, and certificate for the Hashicorp Vault server instance. + Each `secret_mount_point` must serve only one Percona Server instance. Multiple servers that share a `secret_mount_point` write to the same Vault namespace. The shared namespace exposes the following risks: -You can use [OpenSSL :octicons-link-external-16:](https://www.openssl.org/docs/man3.0/index.html) to generate these files or use existing files. The key files contain sensitive information. Store these key files and the password used to create each key in a secure location. + | Risk | Mechanism | + |---|---| + | Permanent data loss | Key writes from one server can overwrite key writes from another server. Overwritten keys cannot decrypt previously encrypted data. Cloned servers that retain a source `server_uuid` collide on InnoDB master-key names. User-named keys created through the keyring user-defined function (UDF) surface collide on any duplicate name. | + | Unauthorized key disclosure | Every server with access to the shared mount point can read every key stored by every other server. The component applies no per-server scoping inside the mount. | + | Iteration noise | The keyring metadata iterator returns all keys under the mount, including keys owned by other servers. Rotation and audit tooling sees foreign keys. | -You can use the Hashicorp Vault to [build your own CA :octicons-link-external-16:](https://developer.hashicorp.com/vault/tutorials/secrets-management/pki-engine), if needed, and then create a Hashicorp Vault server certificate. + Assign a unique `secret_mount_point` to each Percona Server instance. +The component fetches the key type and data from the Vault server on the first request for that key. -The `component_keyring_vault.cnf` file contains the following information: +#### Validation rules -* `read_local_config [optional]` - this option can be used only in the global configuration file. This option indicates whether the component should read configuration information from the local configuration file. The allowed values are `true` or `false`. If you do not use this option, the component uses only the global configuration file. +The component validates each parameter at startup. The following table lists the rejected configurations and their effects: - If you use the `read_local_config` option in the global configuration file along with other items, the component checks the `read_local_config` item value first: +| Condition | Result | +|---|---| +| `secret_mount_point` ends with `/` | The component fails to initialize. | +| `secret_mount_point` starts with `/` | The component fails to initialize. | +| `timeout` exceeds 86400 | The component fails to initialize. | +| `vault_ca` is set together with an `http://` `vault_url` | The component fails to initialize. | +| `vault_url` does not start with `http://` or `https://` | The component fails to initialize. | +| `vault_url` uses `https://` but `vault_ca` is unset | The component logs an error and falls back to the host trust store for certificate validation. | - `false` - the component processes other items in the global configuration file and ignores the local configuration file. +### `secret_mount_point_version` values - `true` - the component ignores other items in the global configuration file and attempts to read the local configuration file. +The `secret_mount_point_version` parameter accepts one of the following values: -* `timeout` - the duration in seconds for the Vault server connection timeout. The default value is 15. The allowed range is from 0 to 86400. The timeout can be also disabled to wait an infinite amount of time by setting this variable to 0. +| Value | Description | +|---|---| +| `1` | Works with `KV Secrets Engine - Version 1 (kv)`. The component uses `secret_mount_point` directly when forming key operation URLs. For example, the URL for a key named `skey` is `/v1//skey`. | +| `2` | Works with `KV Secrets Engine - Version 2 (kv-v2)`. Initialization splits the `secret_mount_point` parameter into two parts: The component uses both parts to form key access URLs, for example `/v1//data//skey`. | +| `AUTO` | The component probes the secrets engine to detect `kv` or `kv-v2`. The component then either uses `secret_mount_point` directly or splits the parameter into a mount point path and a directory path. See [Auto-detection algorithm](#auto-detection-algorithm). | +| Not listed | The component behaves as if `secret_mount_point_version` is set to `AUTO`. | -* `vault_url` - the Vault server address. +The component fails to initialize for any other value. Numeric values other than `1` or `2` produce an error. Non-numeric values other than `AUTO` produce a separate error. -* `secret_mount_point` - the mount point name where the keyring_vault stores the keys. +A version mismatch causes one of the following failures: -* `token` - a token generated by the Vault server. +| Configured value | Actual engine version | Result | +|---|---|---| +| `2` | `kv` (Version 1) | The component runs auto-detection during initialization to confirm the configured value. The component fails to initialize when the probe finds no `kv-v2` mount. The error message reads `Auto-detected mount point version is not the same as specified in 'secret_mount_point_version'`. | +| `1` | `kv-v2` (Version 2) | The component initializes, but every keyring operation fails. | -* `secret_mount_point_version [optional]` - the `KV Secrets Engine version (kv or kv-v2)` used. The allowed values are `AUTO`, `1`, and `2`. The default value is `AUTO`. +#### Auto-detection algorithm -* `vault_ca [optional]` - if the machine does not trust the Vault’s CA certificate, this variable points to the CA certificate used to sign the Vault’s certificates. +The component runs the auto-detection probe during initialization in two cases: -??? example "Example of a configuration file in JSON format" +* `secret_mount_point_version` is set to `AUTO`. - ```json - { - "timeout": 15, - "vault_url": "https://vault.public.com:8202", - "secret_mount_point": "secret", - "secret_mount_point_version": "AUTO", - "token": "{randomly-generated-alphanumeric-string}", - "vault_ca": "/data/keyring_vault_confs/vault_ca.crt" - } - ``` +* `secret_mount_point_version` is set to `2`. -!!! warning +The probe walks `secret_mount_point` from the longest prefix to the shortest. The probe queries the Vault metadata configuration endpoint for each prefix. The first prefix that returns a `kv-v2` response defines the mount point path. The remainder of `secret_mount_point` becomes the directory path. The component falls back to `kv` when no prefix returns `kv-v2`. - Each `secret_mount_point` must be used by only one server. The behavior is unpredictable if multiple servers use the same `secret_mount_point`. +The probe writes one informational message per prefix to the server log. The messages identify successful matches, unreadable responses, and rejected prefixes. -The first time a key is fetched from a keyring, the `keyring_vault` communicates with the Vault server to retrieve the key type and data. +### Upgrade from Vault Secrets Engine Version 1 to Version 2 -### secret_mount_point_version information +Use either of the following methods to upgrade from Version 1 to Version 2: -The `secret_mount_point_version` can be either a `1`, `2`, `AUTO`, or the `secret_mount_point_version` parameter is not listed in the configuration file. +* Set `secret_mount_point_version` to `AUTO` in the `keyring_vault` configuration file on every Percona Server, or omit the parameter. The `AUTO` value triggers autodetection during component initialization. -| Value | Description | -|----------------- | ---------------------------------------------------- | -| 1 | Works with `KV Secrets Engine - Version 1 (kv)`. When forming key operation URLs, the `secret_mount_point` is always used without any transformations. For example, to return a key named `skey`, the URL is /v1//skey | -| 2 | Works with `KV Secrets Engine - Version 2 (kv)` The initialization logic splits the `secret_mount_point` parameter into two parts:
  • The `mount_point_path` - the mount path under which the Vault Server secret was created
  • The `directory_path` - a virtual directory suffix that can be used to create virtual namespaces with the same real mount point
For example, both the `mount_point_path` and the `directory_path` are needed to form key access URLs: /v1//skey | -| AUTO | An autodetection mechanism probes and determines if the secrets engine version is `kv` or `kv-v2` and based on the outcome will either use the `secret_mount_point` as is, or split the `secret_mount_point` into two parts.| -| Not listed| If the `secret_mount_point_version` is not listed in the configuration file, the behavior is the same as `AUTO`.| +* Set `secret_mount_point_version` to `2` to ensure each component initializes only after the upgrade from `kv` to `kv-v2` completes. -If you set the `secret_mount_point_version` to `2` but the path pointed by `secret_mount_point` is based on `KV Secrets Engine - Version 1 (kv)`, an error is reported, and the component fails to initialize. +!!! note -If you set the `secret_mount_point_version` to `1` but the path pointed by `secret_mount_point` is based on `KV Secrets Engine - Version 2 (kv-v2)`, the component initialization succeeds but any MySQL keyring-related operations fail. + The `keyring_vault` component does not use the built-in key versioning of `kv-v2`. The component encodes each version into the keyring key name. -### Upgrade from Vault Secrets Engine Version 1 to Version 2 +## Services exposed by the component -You can upgrade from the Vault Secrets Engine Version 1 to Version 2. - -Use either of the following methods: +The `keyring_vault` component implements the following services for keyring operations. Other components and the server core consume these services to manage keys backed by Vault: -* Set the `secret_mount_point_version` to `AUTO` or the variable is not set in the `keyring_vault` component configuration files in all Percona Servers. The `AUTO` value ensures the autodetection mechanism is invoked during the component initialization. +| Service | Purpose | +|---|---| +| `keyring_aes` | Performs Advanced Encryption Standard (AES) encryption and decryption with keys stored in Vault. | +| `keyring_component_metadata_query` | Reports the component metadata, including the loaded configuration values. | +| `keyring_component_status` | Reports the component status. | +| `keyring_generator` | Generates keys on demand and stores the keys in Vault. | +| `keyring_keys_metadata_iterator` | Iterates over key metadata for inspection. | +| `keyring_load` | Loads the keyring during server startup and reloads on demand. | +| `keyring_reader_with_status` | Reads keys and exposes the read status. | +| `keyring_writer` | Writes keys to Vault. | -* Set the `secret_mount_point_version` to `2` to ensure that components do not initialize unless the `kv` to `kv-v2` upgrade completes. +The `keyring_aes` service supports server-side encryption at rest. The service also supports the AES UDFs when paired with the encryption UDF component. -!!! note +## Operate the vault component + +The shared [Operate the keyring](#operate-the-keyring) section covers rotation, monitoring, and backup procedures that apply to every keyring component. The following notes describe behaviors that are specific to `keyring_vault`. + +### Rotate the master key + +Each `ALTER INSTANCE ROTATE INNODB MASTER KEY` statement writes a master key entry to Vault under the configured `secret_mount_point`. The previous key remains in Vault history when `secret_mount_point_version` is `2`. The previous key also remains when the AUTO probe identifies a `kv-v2` mount. The component contacts Vault during the rotation, so a slow or unreachable Vault delays statement completion. Schedule rotation during a low-load window. + +### Monitor the component + +Compare the values returned by `performance_schema.keyring_component_status` against `component_keyring_vault.cnf`. The comparison detects drift in `vault_url`, `secret_mount_point`, `vault_ca`, and `secret_mount_point_version`. Watch the server error log for `HTTP 403` (token expiration) and `HTTP 503` (sealed Vault). Configure alerts on both patterns. + +### Back up and restore + +A Vault-backed restore requires the following on the destination host: + +* Network reachability to the same Vault server, with valid name resolution and certificate trust. + +* A valid token with read access to the configured `secret_mount_point`. + +* A `server_uuid` distinct from the source host. A duplicated `server_uuid` triggers the data-loss case described in [Configure the keyring vault component](#configure-the-keyring-vault-component). + +* `kv-v2` history retention that covers the time between backup and restore. A backup taken before a master-key rotation cannot decrypt later tablespaces. The post-rotation key must remain in Vault history for the restore to succeed. + +## See also + +* [Data at Rest Encryption](data-at-rest-encryption.md) describes how Percona Server uses the keyring to protect tablespace data. + +* [Get Started with component keyring](quickstart-component-keyring.md) walks through a minimal setup. - The `keyring_vault` component that works with `kv-v2` secret engines does not use the built-in key versioning capabilities. The keyring key versions are encoded into key names. +* [Keyring components overview](keyring-components-plugins-overview.md) compares the available keyring components. -!!! admonition "See also" +* [Maintain the Vault connection](maintain-vault-connection.md) covers token expiration, token renewal patterns, and Vault seal handling. - [Hashicorp Documentation: Installing Vault :octicons-link-external-16:](https://www.vaultproject.io/docs/install/index.html) - - [Hashicorp Documentation: Production Hardening :octicons-link-external-16:](https://learn.hashicorp.com/vault/operations/production-hardening -) - \ No newline at end of file +* [Rotate the master encryption key](rotate-master-key.md) covers the full rotation procedure and required privileges. diff --git a/docs/using-amz-kms.md b/docs/using-amz-kms.md index 3b266967cc9..ada547773e8 100644 --- a/docs/using-amz-kms.md +++ b/docs/using-amz-kms.md @@ -1,93 +1,78 @@ # Use the Amazon Key Management Service (AWS KMS) -Percona Server for MySQL supports the [Amazon Key Management Service (AWS KMS) :octicons-link-external-16:](https://aws.amazon.com/kms/). Percona Server -generates the keyring keys. Amazon Web Services (AWS) encrypts the keyring data. +Percona Server for MySQL supports the [Amazon Key Management Service (AWS KMS) :octicons-link-external-16:](https://aws.amazon.com/kms/). Percona Server generates the keyring keys. Amazon Web Services (AWS) encrypts the keyring data. -The AWS KMS lets you create and manage cryptographic keys across AWS services. For more information, see the -[AWS Key Management Service Documentation :octicons-link-external-16:](https://docs.aws.amazon.com/kms/). +The AWS KMS lets you create and manage cryptographic keys across AWS services. For more information, see the [AWS Key Management Service Documentation :octicons-link-external-16:](https://docs.aws.amazon.com/kms/). -To use the AWS KMS component, do the following: +## Prerequisites -* Have an AWS user account. This account has an access key and a secret key. +Complete the following before you install the AWS KMS component: -* Create a KMS key ID. The KMS key can then be referenced in the configuration -either by its ID, alias (the key can have any number of aliases), or ARN. +* Create an AWS user with an access key and a secret key. -## Component installation +* Create a KMS key. Reference the key by its identifier, alias, or Amazon Resource Name (ARN). Each key can have any number of aliases. --8<--- "keyring-components-installation.md" -For more information, see [Installing and Uninstalling Components :octicons-link-external-16:](https://dev.mysql.com/doc/refman/{{vers}}/en/component-loading.html). +For more information, see [Installing and uninstalling components :octicons-link-external-16:](https://dev.mysql.com/doc/refman/{{vers}}/en/component-loading.html). -The following example is a global manifest file that does not use local -manifests: +## Configuration settings -```json -{ - "read_local_manifest": false, - "components": "file://component_keyring_kms" -} -``` +The configuration settings live in either a global configuration file or a local configuration file. -The following is an example of a global manifest file that points to a local manifest file: +### Configuration parameters -```json -{ - "read_local_manifest": true -} -``` +The `component_keyring_kms.cnf` file contains the following parameters: -The following is an example of a local manifest file: +* `read_local_config [optional]` — declares whether the component reads the local configuration file. Use this parameter only in the global configuration file. The allowed values are `true` or `false`. -```json -{ - "components": "file://component_keyring_kms" -} -``` +* `path` — the path to the keyring file on the local host. The file uses JSON format. -The configuration settings are either in a global configuration file or a local -configuration file. The settings are the same. +* `read_only [optional]` — when `true`, the component prevents runtime changes to the keyring. The default value is `false`. -The KMS configuration file has the following options: +* `kms_key` — the identifier of an AWS KMS master key. Create the key before you create the manifest file. The identifier accepts one of the following formats: -* read_local_config + * Universally Unique Identifier (UUID) -* path - the location of the JSON keyring database file. + * Alias -* read_only - if true, the keyring cannot be modified. + * Amazon Resource Name (ARN) -* kms_key - the identifier of an AWS KMS master key. The user must create this key before creating the manifest file. The identifier can be one of the -following: + For more information, see [Finding the key ID and key ARN :octicons-link-external-16:](https://docs.aws.amazon.com/kms/latest/developerguide/find-cmk-id-arn.html). - * UUID +* `region` — the AWS region that hosts the KMS key. Each Hypertext Transfer Protocol (HTTP) request connects to this region. - * Alias +* `auth_key` — the access key for the AWS user. The user must have permission to access the KMS key. + +* `secret_access_key` — the secret key for the AWS user. - * ARN +!!! warning -For more information, see [Finding the key ID and key ARN :octicons-link-external-16:](https://docs.aws.amazon.com/kms/latest/developerguide/find-cmk-id-arn.html). + The configuration file contains authentication credentials. Restrict read access on the file to the `mysql` user. -* region - the AWS where the KMS is stored. Any HTTP request connect to this region. +#### Example configuration -* auth_key - an AWS user authentication key. The user must have access to the KMS key. +The following example shows a `component_keyring_kms.cnf` file in JSON format with placeholder values. Replace each placeholder with the value for the target AWS KMS deployment. + +```json +{ + "read_local_config": false, + "path": "/usr/local/mysql/keyring-mysql/aws-keyring-data", + "region": "", + "kms_key": "", + "auth_key": "", + "secret_access_key": "" +} +``` -* secret_access_key - the secret key (API “password”) for the AWS user. +For more information, see [Keyring component installation :octicons-link-external-16:](https://dev.mysql.com/doc/refman/{{vers}}/en/keyring-component-installation.html). -!!! note +## See also - The configuration file contains authentication information. Only the MySQL process should be able to read this file. +* [Data at Rest Encryption](data-at-rest-encryption.md) describes how Percona Server uses the keyring to protect tablespace data. -??? example "Example of a configuration file in JSON format" +* [Get Started with component keyring](quickstart-component-keyring.md) walks through a minimal setup. - ```json - { - "read_local_config": "true/false", - "path": "/usr/local/mysql/keyring-mysql/aws-keyring-data", - "region": "eu-central-1", - "kms_key": "UUID, alias or ARN as displayed by the KMS console", - "auth_key": "AWS user key", - "secret_access_key": "AWS user secret key" - } - ``` +* [Keyring components overview](keyring-components-plugins-overview.md) compares the available keyring components. -For more information, see [Keyring Component installation :octicons-link-external-16:](https://dev.mysql.com/doc/refman/{{vers}}/en/keyring-component-installation.html). +* [Rotate the master encryption key](rotate-master-key.md) covers the full rotation procedure and required privileges. diff --git a/mkdocs-base.yml b/mkdocs-base.yml index c7d4f6dc5e7..124d8bde589 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -353,6 +353,7 @@ nav: - quickstart-component-keyring.md - use-keyring-file.md - use-keyring-vault-component.md + - maintain-vault-connection.md - using-kmip.md - using-amz-kms.md - encrypt-file-per-table-tablespace.md diff --git a/snippets/keyring-components-installation.md b/snippets/keyring-components-installation.md index e164a0af3aa..44b75850d6a 100644 --- a/snippets/keyring-components-installation.md +++ b/snippets/keyring-components-installation.md @@ -1,16 +1,181 @@ -The component must be installed with a manifest. A keyring component is not loaded with the `--early-plugin-load` option on the server. The server uses a manifest and the component consults its configuration file during initialization. You should only load a keyring component with a manifest file. Do not use the `INSTALL_COMPONENT` statement, which loads the keyring components too late in the startup sequence of the server. For example, `InnoDB` requires the component, but because the components are registered in the `mysql.component` table, this table is loaded after `InnoDB` initialization. +Percona Server uses a keyring to protect the master keys that encrypt data at rest. The server loads the keyring component from a manifest file. The component then reads a separate configuration file during initialization. Two alternative load mechanisms cannot load the keyring early enough: `--early-plugin-load` and `INSTALL COMPONENT`. -You should create a global manifest file named `mysqld.my` in the installation directory and, optionally, create a local manifest file, also named `mysqld.my` in a data directory. +## Available keyring components -To install a keyring component, do the following: +Percona Server provides the following keyring components: -1. Write a manifest in a valid JSON format +| Component | Backend | +|---|---| +| `component_keyring_file` | Local file on disk | +| `component_keyring_kmip` | Key Management Interoperability Protocol (KMIP) server | +| `component_keyring_kms` | Cloud Key Management Service (KMS) | +| `component_keyring_vault` | HashiCorp Vault | -2. Write a configuration file +Each component uses the same manifest mechanism. Each component reads its own configuration file. The name of the configuration file matches the component name with a `.cnf` extension. For example, `component_keyring_vault` reads `component_keyring_vault.cnf`. -A manifest file indicates which component to load. If the manifest file does not exist, the server does not load the component associated with that file. During startup, the server reads the global manifest file from the installation directory. The global manifest file can contain the required information or point to a local manifest file located in the data directory. If you have multiple server instances that use different keyring components use a local manifest file in each data directory to load the correct keyring component for that instance. +## Manifest files + +### Manifest fields + +The manifest is a JavaScript Object Notation (JSON) object with the following fields: + +| Field | Type | Required | Description | +|---|---|---|---| +| `components` | string | Conditional | URL-style locator for the component to load. The format is `file://`. Required when the manifest declares the component directly. | +| `read_local_manifest` | boolean | No | When `true`, the global manifest delegates to the local manifest in the data directory. When `false` or absent, the global manifest declares the component directly. | + +### Manifest examples + +Replace `` in the following examples with one of the components in [Available keyring components](#available-keyring-components). + +The following example shows a global manifest that loads a component directly: + +```json +{ + "components": "file://" +} +``` + +The following example shows a global manifest that delegates to the local manifest: + +```json +{ + "read_local_manifest": true +} +``` + +The following example shows a local manifest: + +```json +{ + "components": "file://" +} +``` + +### Manifest locations + +Percona Server reads two manifest files at startup: + +| Manifest | Location | Purpose | +|---|---|---| +| Global manifest | The directory that contains the `mysqld` binary | Default manifest for the server installation | +| Local manifest | Data directory | Per-instance override for hosts that run multiple instances with different keyring components | + +Both files use the name `mysqld.my`. The global manifest declares the component directly or delegates to the local manifest. + +### Why the manifest is the only supported load path + +The keyring must load before `InnoDB` opens an encrypted page. Any mechanism that depends on the Structured Query Language (SQL) layer therefore loads the keyring too late. + +A `mysqld` startup proceeds in the following order: + +1. `mysqld` parses startup configuration and reads the manifest file from the server installation directory. + +2. The server loads each component named in the manifest. + +3. `InnoDB` initializes, replays the redo log, and opens tablespaces. + +4. The SQL layer accepts connections. + +The keyring must load between steps 1 and 3. The following table explains why each alternative mechanism fails to load the keyring within this window: + +| Mechanism | Reason for failure | +|---|---| +| [`INSTALL COMPONENT`](install-component.md) | The statement runs as SQL and cannot execute until step 4. The registration record lives in `mysql.component`, an `InnoDB` table that the server reads only after `InnoDB` initializes. The system tablespace cannot decrypt without the keyring, which creates a circular dependency. Crash recovery also runs before the SQL layer, so the encrypted redo log must remain readable without SQL. | +| `--early-plugin-load` | The flag applies to legacy keyring plugins, not components. Plugins and components load through separate subsystems. The flag cannot locate component entry points. | + +A component registered through `INSTALL COMPONENT` on a running server does not persist across restarts. On the next restart, `InnoDB` cannot unwrap tablespace keys because no manifest file exists on disk. A missing or malformed `mysqld.my` therefore prevents startup for any instance with encrypted tablespaces. + +## Install a keyring component + +To install a keyring component: + +1. Stop the `mysqld` process. + +2. Create the manifest file at the chosen location. Use one of the examples in [Manifest examples](#manifest-examples) as a template. + +3. Create the configuration file in the same directory as the manifest that declares the component. The name of the configuration file matches the component name with a `.cnf` extension. + +4. Populate the configuration file with parameters for the chosen component. The required parameters depend on the component. + +5. Set the file ownership of the manifest file and the configuration file to the `mysql` user. + +6. Restrict read access on the configuration file to the `mysql` user. The configuration file may contain credentials. + +7. Start the `mysqld` process. + +## Verify the installation + +Connect to the server and run the following query: + +```sql +SELECT * FROM performance_schema.keyring_component_status; +``` + +The query returns one row per status field. The `Component_status` field reports `Active` when the component loaded. The `Component_status` field reports `Disabled` when the component failed to load. Inspect the server error log to identify the cause of any failure. + +## Operate the keyring + +### Rotate the master key + +The InnoDB master key wraps the tablespace keys that protect data on disk. Rotate the master key on a scheduled cadence. Scheduled rotation limits the volume of data wrapped by any single master key. See [Data at Rest Encryption](data-at-rest-encryption.md) for the role of the master key. + +Run the following statement to rotate the master key: + +```sql +ALTER INSTANCE ROTATE INNODB MASTER KEY; +``` + +The statement generates a master key and stores the key in the keyring. The server then wraps subsequent tablespace keys with the stored key. + +The rotation operation is fast. The server does not re-encrypt the tablespace data. + +Earlier master keys must remain in the keyring. Deleting an earlier master key prevents the server from reading data that was wrapped with that key. + +For required privileges and the full rotation procedure, see [Rotate the master encryption key](rotate-master-key.md). + +### Monitor the keyring + +Use the query in [Verify the installation](#verify-the-installation) to inspect the keyring at runtime. Configure alerts on `Component_status` transitions to `Disabled`. Watch the server error log for keyring-related errors. The server writes manifest parse errors, configuration parse errors, and backend connection errors to the error log. + +### Back up and restore + +A backup of an encrypted tablespace requires keyring access on the destination host. The destination host must read the same key material that encrypted the backup. + +The following table summarizes the keyring requirement for each backend: + +| Backend | Backup requirement | +|---|---| +| `component_keyring_file` | Copy the keyring file alongside the data backup. The destination host must read the same key material. | +| `component_keyring_vault` | Configure the destination host to authenticate with the same Vault server and access the same `secret_mount_point`. | +| `component_keyring_kmip` | Configure the destination host to authenticate with the same KMIP server and access the same key namespace. | +| `component_keyring_kms` | Configure the destination host with credentials and permissions for the same cloud KMS keys. | + +Test a restore on a separate host before relying on the backup. A restore that runs on the original host can succeed even when the keyring configuration is incorrect. + +## Replace or remove a keyring component + +To replace a keyring component: + +1. Stop the `mysqld` process. + +2. Edit the manifest file to reference the replacement component. + +3. Create the configuration file for the replacement component. + +4. Start the `mysqld` process. + +To remove the keyring: + +1. Confirm that no encrypted tablespaces exist on the server. A server with encrypted tablespaces cannot start without a keyring. + +2. Stop the `mysqld` process. + +3. Delete the manifest file. + +4. Start the `mysqld` process. !!! warning - Enable only one keyring plugin or one keyring component at a time for each server instance. Enabling multiple keyring plugins or keyring components or mixing keyring plugins or keyring components is not supported and may result in data loss. \ No newline at end of file + Configure exactly one keyring per server instance. Percona Server does not support multiple keyring plugins, multiple keyring components, or any combination of plugin and component. Such configurations risk data loss.