OADP-7660: Document NonAdminBSL requires long-term credentials #334
+161
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,161 @@ | ||||||
| # NonAdminBSL Credential Requirements: Long-Term Credentials Only | ||||||
|
|
||||||
| _Jira: [OADP-7660](https://redhat.atlassian.net/browse/OADP-7660)_ | ||||||
|
|
||||||
| ## Overview | ||||||
|
|
||||||
| NonAdminBackupStorageLocation (NonAdminBSL) requires non-admin users to provide long-term cloud credentials (access keys, service account JSON keys, client secrets) stored in a Secret in their own namespace. | ||||||
| Short-lived, cloud-native credential mechanisms (AWS STS, GCP Workload Identity Federation, Azure Workload Identity) are **not supported** for NonAdminBSL and **must not be used**. | ||||||
|
|
||||||
| ## How NonAdminBSL Credentials Work Today | ||||||
|
Comment on lines
+1
to
+10
There was a problem hiding this comment. This doc consistently uses the acronym “NonAdminBSL”, but the rest of the repo’s docs and user-facing naming tends to use “NABSL” (matching the CRD shortName |
||||||
|
|
||||||
| The non-admin user creates a Secret in their namespace containing cloud credentials. | ||||||
| The [NonAdminBSL controller](https://github.com/migtools/oadp-non-admin/blob/31a3bfc20310898fae5a438005d0d242240d0fbb/internal/controller/nonadminbackupstoragelocation_controller.go#L736-L841) copies this Secret into the OADP namespace and references the copy in the Velero BSL's `spec.credential` field. | ||||||
| Velero's [`FileStore.Path()`](https://github.com/vmware-tanzu/velero/blob/6e91e72e655568dd6944ca7bb3cf00b6c7fbb3c8/internal/credentials/file_store.go#L66-L88) writes the secret data to a temporary file on disk and passes the file path to the cloud plugin via `config["credentialsFile"]`. | ||||||
|
|
||||||
|
Comment on lines
+10
to
+15
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Description: Verify GitHub permalink URLs return 200 and check if line ranges are valid
echo "Checking GitHub URL accessibility..."
# Check NonAdminBSL controller syncSecrets reference
url1="https://github.com/migtools/oadp-non-admin/blob/31a3bfc20310898fae5a438005d0d242240d0fbb/internal/controller/nonadminbackupstoragelocation_controller.go#L736-L841"
echo "URL 1: $url1"
curl -s -o /dev/null -w "HTTP Status: %{http_code}\n" -L "$url1"
# Check Velero FileStore.Path() reference
url2="https://github.com/vmware-tanzu/velero/blob/6e91e72e655568dd6944ca7bb3cf00b6c7fbb3c8/internal/credentials/file_store.go#L66-L88"
echo "URL 2: $url2"
curl -s -o /dev/null -w "HTTP Status: %{http_code}\n" -L "$url2"
# Check Azure credential reference
url3="https://github.com/vmware-tanzu/velero/blob/6e91e72e655568dd6944ca7bb3cf00b6c7fbb3c8/pkg/util/azure/credential.go#L48-L54"
echo "URL 3: $url3"
curl -s -o /dev/null -w "HTTP Status: %{http_code}\n" -L "$url3"Repository: migtools/oadp-non-admin Length of output: 588 🏁 Script executed: #!/bin/bash
# Verify line ranges are correct by fetching raw file content
echo "=== Checking NonAdminBSL controller.go ==="
raw_url1="https://raw.githubusercontent.com/migtools/oadp-non-admin/31a3bfc20310898fae5a438005d0d242240d0fbb/internal/controller/nonadminbackupstoragelocation_controller.go"
curl -s "$raw_url1" | sed -n '735,850p' | head -20
echo ""
echo "=== Checking Velero FileStore.Path() ==="
raw_url2="https://raw.githubusercontent.com/vmware-tanzu/velero/6e91e72e655568dd6944ca7bb3cf00b6c7fbb3c8/internal/credentials/file_store.go"
curl -s "$raw_url2" | sed -n '60,95p'
echo ""
echo "=== Line count check for controller.go ==="
curl -s "$raw_url1" | wc -l
echo ""
echo "=== Checking context around line 736-841 in controller.go ==="
curl -s "$raw_url1" | sed -n '735p;736p;737p;840p;841p;842p'Repository: migtools/oadp-non-admin Length of output: 2711 🏁 Script executed: #!/bin/bash
# Find the actual end of syncSecrets function
raw_url="https://raw.githubusercontent.com/migtools/oadp-non-admin/31a3bfc20310898fae5a438005d0d242240d0fbb/internal/controller/nonadminbackupstoragelocation_controller.go"
# Get the file and find where syncSecrets starts
echo "=== Finding syncSecrets function boundaries ==="
curl -s "$raw_url" | awk '
/^func \(r \*NonAdminBackupStorageLocationReconciler\) syncSecrets/ {
start = NR
depth = 0
}
start && NR >= start {
for (i = 1; i <= length($0); i++) {
c = substr($0, i, 1)
if (c == "{") depth++
if (c == "}") depth--
}
if (depth == 0 && NR > start) {
print "Function starts at line: " start
print "Function ends at line: " NR
exit
}
}
'
echo ""
echo "=== Lines around 841 ==="
curl -s "$raw_url" | sed -n '838,845p' | cat -n
echo ""
echo "=== Lines around 850 ==="
curl -s "$raw_url" | sed -n '847,856p' | cat -nRepository: migtools/oadp-non-admin Length of output: 1041 Correct the GitHub line range for the NonAdminBSL controller reference. The syncSecrets function in the NonAdminBSL controller actually extends to line 849, not line 841. Update the GitHub permalink from 🤖 Prompt for AI Agents |
||||||
| ## Why Short-Lived Credentials Do Not Work with NonAdminBSL | ||||||
|
|
||||||
| ### 1. Token File Path Is Not Accessible | ||||||
|
|
||||||
| Short-lived credential files reference a projected service account token file on the Velero pod's filesystem: | ||||||
|
|
||||||
| - **AWS STS:** `web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token` | ||||||
| - **GCP WIF:** `"credential_source": {"file": "/var/run/secrets/openshift/serviceaccount/token"}` | ||||||
| - **Azure WI:** `AZURE_FEDERATED_TOKEN_FILE=/var/run/secrets/openshift/serviceaccount/token` | ||||||
|
|
||||||
| This projected token is bound to the `velero` service account in the OADP namespace. | ||||||
| It is only available inside the Velero pod -- it does not exist in the non-admin user's namespace. | ||||||
|
|
||||||
| If a non-admin user creates a credential Secret containing one of these formats, the token file path reference is correct only because it points to the Velero pod's existing projected volume. | ||||||
| The credential would use the **admin-provisioned Velero identity** to authenticate to the cloud provider, not a user-scoped identity. | ||||||
|
|
||||||
| ### 2. No Per-Namespace Cloud Identity Scoping | ||||||
|
|
||||||
| The admin-level STS/WIF/WI credential is configured with a single cloud identity (IAM role, GCP service account, or Azure managed identity) that has broad permissions -- typically access to the entire backup bucket. | ||||||
|
|
||||||
| If a non-admin user's BSL used this credential: | ||||||
| - **All non-admin users would share the same cloud identity** with the same permissions. | ||||||
| - A non-admin user could access or overwrite **any other user's backup data** within the bucket, limited only by their choice of `objectStorage.prefix`. | ||||||
| - There is no cloud-side enforcement of per-namespace isolation. | ||||||
|
|
||||||
| ### 3. Credential File Copied Without Validation | ||||||
|
|
||||||
| The NonAdminBSL controller's [`syncSecrets`](https://github.com/migtools/oadp-non-admin/blob/31a3bfc20310898fae5a438005d0d242240d0fbb/internal/controller/nonadminbackupstoragelocation_controller.go#L736-L841) function copies the user's Secret data verbatim: | ||||||
|
|
||||||
| ```go | ||||||
| for k, v := range sourceNaBSLSecret.Data { | ||||||
| veleroBslSecret.Data[k] = v | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| The controller does not inspect or validate the credential format. | ||||||
| If a user provides a WIF/STS/WI credential file, the controller copies it without recognizing that it references the Velero pod's projected token -- effectively granting the user access via the admin's cloud identity. | ||||||
|
|
||||||
| ### 4. Azure Workload Identity Is Broken for Per-BSL Use | ||||||
|
|
||||||
| Even if per-namespace cloud identities were provisioned, Azure Workload Identity credentials do not work on a per-BSL basis in current Velero. | ||||||
| The [`NewCredential()`](https://github.com/vmware-tanzu/velero/blob/6e91e72e655568dd6944ca7bb3cf00b6c7fbb3c8/pkg/util/azure/credential.go#L48-L54) function reads `AZURE_FEDERATED_TOKEN_FILE`, `AZURE_CLIENT_ID`, and `AZURE_TENANT_ID` exclusively from **environment variables**, ignoring the per-BSL `creds` map entirely. | ||||||
| All Azure BSLs share the same pod-level identity regardless of what is in their credential file. | ||||||
|
|
||||||
| See upstream issue: [vmware-tanzu/velero#9657](https://github.com/vmware-tanzu/velero/issues/9657) | ||||||
|
|
||||||
| ## Required Credential Formats | ||||||
|
|
||||||
| Non-admin users must provide **long-term cloud credentials** in a Secret in their namespace. | ||||||
|
|
||||||
| ### AWS | ||||||
|
|
||||||
| ```yaml | ||||||
| apiVersion: v1 | ||||||
| kind: Secret | ||||||
| metadata: | ||||||
| name: my-backup-credentials | ||||||
| namespace: <user-namespace> | ||||||
| type: Opaque | ||||||
| stringData: | ||||||
| cloud: | | ||||||
| [default] | ||||||
| aws_access_key_id = <ACCESS_KEY_ID> | ||||||
| aws_secret_access_key = <SECRET_ACCESS_KEY> | ||||||
| ``` | ||||||
|
|
||||||
| ### GCP | ||||||
|
|
||||||
| ```yaml | ||||||
| apiVersion: v1 | ||||||
| kind: Secret | ||||||
| metadata: | ||||||
| name: my-backup-credentials | ||||||
| namespace: <user-namespace> | ||||||
| type: Opaque | ||||||
| stringData: | ||||||
| cloud: | | ||||||
| { | ||||||
| "type": "service_account", | ||||||
| "project_id": "<project-id>", | ||||||
| "private_key_id": "<key-id>", | ||||||
| "private_key": "<private-key>", | ||||||
| "client_email": "<sa-email>", | ||||||
| "client_id": "<client-id>", | ||||||
| "auth_uri": "https://accounts.google.com/o/oauth2/auth", | ||||||
| "token_uri": "https://oauth2.googleapis.com/token" | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ### Azure | ||||||
|
|
||||||
| ```yaml | ||||||
| apiVersion: v1 | ||||||
| kind: Secret | ||||||
| metadata: | ||||||
| name: my-backup-credentials | ||||||
| namespace: <user-namespace> | ||||||
| type: Opaque | ||||||
| stringData: | ||||||
| cloud: | | ||||||
| AZURE_SUBSCRIPTION_ID=<subscription-id> | ||||||
| AZURE_TENANT_ID=<tenant-id> | ||||||
| AZURE_CLIENT_ID=<client-id> | ||||||
| AZURE_CLIENT_SECRET=<client-secret> | ||||||
| AZURE_RESOURCE_GROUP=<resource-group> | ||||||
| AZURE_CLOUD_NAME=AzurePublicCloud | ||||||
| ``` | ||||||
|
|
||||||
| ### NonAdminBSL referencing the Secret | ||||||
|
|
||||||
| ```yaml | ||||||
| apiVersion: nac.oadp.openshift.io/v1alpha1 | ||||||
|
There was a problem hiding this comment. The
Suggested change
|
||||||
| kind: NonAdminBackupStorageLocation | ||||||
| metadata: | ||||||
| name: my-bsl | ||||||
| namespace: <user-namespace> | ||||||
| spec: | ||||||
| backupStorageLocationSpec: | ||||||
| provider: aws # or gcp, azure | ||||||
| credential: | ||||||
| name: my-backup-credentials | ||||||
| key: cloud | ||||||
| objectStorage: | ||||||
| bucket: <bucket-name> | ||||||
| prefix: <prefix> | ||||||
| config: | ||||||
| region: <region> | ||||||
| ``` | ||||||
|
|
||||||
| ## Security Recommendations for Administrators | ||||||
|
|
||||||
| 1. **Scope long-term credentials narrowly.** Each non-admin user's cloud credential should have permissions restricted to a specific bucket/prefix. Do not issue credentials with access to the entire backup bucket. | ||||||
|
|
||||||
| 2. **Rotate credentials regularly.** Since non-admin users must use long-term credentials, establish a rotation policy. Revoke and reissue credentials periodically. | ||||||
|
|
||||||
| 3. **Enable BSL approval.** Set `requireApprovalForBSL: true` in the DPA `nonAdmin` section to review each NonAdminBSL before a Velero BSL is created. | ||||||
|
|
||||||
| 4. **Monitor for credential misuse.** Use cloud provider audit logs (CloudTrail, GCP Audit Logs, Azure Activity Logs) to monitor for unexpected access patterns from non-admin credentials. | ||||||
|
|
||||||
| 5. **Do not share credentials across namespaces.** Each non-admin namespace should have its own unique cloud credential scoped to its own storage path. | ||||||
|
|
||||||
| ## Future Work | ||||||
|
|
||||||
| Per-namespace short-lived credentials for NonAdminBSL are being designed in [openshift/oadp-operator#2143](https://github.com/openshift/oadp-operator/pull/2143). | ||||||
| This would allow administrators to pre-provision scoped cloud identities per namespace, with the non-admin controller automatically selecting the correct credential based on namespace mapping. | ||||||
| Until that work is complete, **long-term credentials are the only supported option for NonAdminBSL**. | ||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This new doc doesn’t appear to be linked from README.md or other docs pages, so it’s easy to miss. Consider adding a cross-link from an existing entry point (e.g., README “Using NAC” / NABSL section or docs/non_admin_user.md) so users can discover these credential constraints.