From feab01e717db8e180cd14dfb51f3c3f41d97405e Mon Sep 17 00:00:00 2001 From: baraka-akeyless Date: Sun, 12 Jul 2026 16:20:54 +0300 Subject: [PATCH 1/2] Update Airflow provider docs: cloud auth now supported in Secrets Backend The Secrets Backend now supports aws_iam, gcp, and azure_ad authentication in addition to api_key and uid. This enables seamless integration with managed Airflow services like Amazon MWAA and Google Managed Service for Apache Airflow without requiring static API keys. Changes: - Update capability table and auth methods table to reflect new support - Add cloud auth configuration examples for AWS IAM, GCP, and Azure AD - Add step-by-step Amazon MWAA setup guide - Remove outdated "Secrets Backend limitation" warning - Update troubleshooting section Co-authored-by: Cursor --- .../apache-airflow-provider.md | 88 +++++++++++++++++-- 1 file changed, 80 insertions(+), 8 deletions(-) diff --git a/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md b/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md index d3b80e089..e2e34a1fe 100644 --- a/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md +++ b/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md @@ -26,7 +26,7 @@ The provider is maintained in the [apache/airflow](https://github.com/apache/air | --- | --- | --- | | **Hook** | `airflow.providers.akeyless.hooks.akeyless.AkeylessHook` | Interact with Akeyless directly from Directed Acyclic Graph (DAG) code — fetch static, dynamic, and rotated secrets; create, update, or delete items; list paths. | | **Connection type** | `akeyless` | Airflow connection type identifier. Create a connection with this type in the Airflow UI or environment to supply credentials to the hook. | -| **Secrets Backend** | `airflow.providers.akeyless.secrets.akeyless.AkeylessBackend` | Transparently resolve Airflow Connections, Variables, and Config from Akeyless — no DAG code changes required. Supports `api_key` and `uid` authentication only. | +| **Secrets Backend** | `airflow.providers.akeyless.secrets.akeyless.AkeylessBackend` | Transparently resolve Airflow Connections, Variables, and Config from Akeyless — no DAG code changes required. Supports `api_key`, `uid`, `aws_iam`, `gcp`, and `azure_ad` authentication. | ## Requirements @@ -57,9 +57,9 @@ The provider supports the following Akeyless [Authentication Methods](https://do | `access_type` | Required fields | Supported by | | --- | --- | --- | | `api_key` _default_ | `access_id`, `access_key` | Hook, Secrets Backend | -| `aws_iam` | `access_id` + `cloud_id` extras package | Hook only | -| `gcp` | `access_id` + `cloud_id` extras package; optional: `gcp_audience` | Hook only | -| `azure_ad` | `access_id` + `cloud_id` extras package; optional: `azure_object_id` | Hook only | +| `aws_iam` | `access_id` + `cloud_id` extras package | Hook, Secrets Backend | +| `gcp` | `access_id` + `cloud_id` extras package; optional: `gcp_audience` | Hook, Secrets Backend | +| `azure_ad` | `access_id` + `cloud_id` extras package; optional: `azure_object_id` | Hook, Secrets Backend | | `uid` | `uid_token` | Hook, Secrets Backend | | `jwt` | `access_id`, `jwt` | Hook only | | `k8s` | `access_id`, `k8s_auth_config_name` | Hook only | @@ -187,6 +187,78 @@ export AIRFLOW__SECRETS__BACKEND="airflow.providers.akeyless.secrets.akeyless.Ak export AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_path": "/airflow/connections", "variables_path": "/airflow/variables", "config_path": "/airflow/config", "api_url": "https://api.akeyless.io", "access_id": "", "access_key": "", "access_type": "api_key"}' ``` +#### Cloud-based authentication in the Secrets Backend + +The Secrets Backend supports `aws_iam`, `gcp`, and `azure_ad` authentication, allowing managed Airflow services to authenticate using their workload identity — no static API keys required. + +**AWS IAM** (for [Amazon MWAA](https://aws.amazon.com/managed-workflows-for-apache-airflow/) and EC2/ECS/EKS workloads): + +```text +[secrets] +backend = airflow.providers.akeyless.secrets.akeyless.AkeylessBackend +backend_kwargs = { + "connections_path": "/airflow/connections", + "variables_path": "/airflow/variables", + "config_path": "/airflow/config", + "api_url": "https://api.akeyless.io", + "access_id": "", + "access_type": "aws_iam" + } +``` + +**GCP** (for [Managed Service for Apache Airflow](https://cloud.google.com/composer/docs) and GCE/GKE workloads): + +```text +[secrets] +backend = airflow.providers.akeyless.secrets.akeyless.AkeylessBackend +backend_kwargs = { + "connections_path": "/airflow/connections", + "variables_path": "/airflow/variables", + "config_path": "/airflow/config", + "api_url": "https://api.akeyless.io", + "access_id": "", + "access_type": "gcp", + "gcp_audience": "akeyless.io" + } +``` + +**Azure AD** (for Azure-hosted workloads): + +```text +[secrets] +backend = airflow.providers.akeyless.secrets.akeyless.AkeylessBackend +backend_kwargs = { + "connections_path": "/airflow/connections", + "variables_path": "/airflow/variables", + "config_path": "/airflow/config", + "api_url": "https://api.akeyless.io", + "access_id": "", + "access_type": "azure_ad", + "azure_object_id": "" + } +``` + +> ℹ️ Cloud-based authentication requires the `cloud_id` extras package. See [Installation](#installation). + +##### Using with Amazon MWAA + +1. Upload a `requirements.txt` to your MWAA S3 bucket containing: + + ```text + apache-airflow-providers-akeyless[cloud_id] + ``` + +2. In the MWAA console under **Airflow configuration options**, add: + + | Key | Value | + | --- | --- | + | `secrets.backend` | `airflow.providers.akeyless.secrets.akeyless.AkeylessBackend` | + | `secrets.backend_kwargs` | `{"api_url": "https://api.akeyless.io", "access_id": "", "access_type": "aws_iam"}` | + +3. Ensure the MWAA VPC has outbound HTTPS access to your Akeyless API endpoint (`api.akeyless.io` or your Akeyless Gateway). + +4. Create an Akeyless `aws_iam` Authentication Method associated with the MWAA execution role ARN. + #### Naming Convention Secrets are looked up by joining `/`: @@ -230,9 +302,7 @@ JSON with `conn_uri`: ### Cloud-Based Authentication -For AWS IAM, GCP, or Azure AD, omit `access_key` and set the appropriate `access_type`. The provider uses the workload's cloud identity automatically. - -> ⚠️ **Secrets Backend limitation:** `AkeylessBackend` only supports `api_key` and `uid` authentication. For cloud-based authentication (AWS IAM, GCP, Azure AD) use `AkeylessHook` directly in your DAGs. +For AWS IAM, GCP, or Azure AD, omit `access_key` and set the appropriate `access_type`. The provider uses the workload's cloud identity automatically. Both the **Hook** and the **Secrets Backend** support cloud-based authentication. Example using AWS IAM with the hook: @@ -245,6 +315,8 @@ value = hook.get_secret_value("/my/secret") Set the connection `access_type` extra field to `aws_iam` and install the `cloud_id` extras. The hook authenticates using the workload's AWS IAM identity (EC2 instance profile, ECS task role, and so on) — no static credentials required. +For Secrets Backend cloud authentication configuration examples, see [Cloud-based authentication in the Secrets Backend](#cloud-based-authentication-in-the-secrets-backend). + ## Troubleshooting ### `ImportError: akeyless_cloud_id is required` @@ -257,7 +329,7 @@ pip install apache-airflow-providers-akeyless[cloud_id] ### `ValueError: Unsupported access_type for AkeylessBackend` -`AkeylessBackend` only supports `api_key` and `uid`. For cloud-based authentication in the Secrets Backend, use `AkeylessHook` directly in your DAGs instead. +`AkeylessBackend` supports `api_key`, `uid`, `aws_iam`, `gcp`, and `azure_ad`. Other authentication types (`jwt`, `k8s`, `certificate`) are only available through `AkeylessHook`. ### Secret not found when using Secrets Backend From 33074b8c2bcd0b96205e15f904dfa2fc41c10ebe Mon Sep 17 00:00:00 2001 From: baraka-akeyless Date: Sun, 12 Jul 2026 16:26:58 +0300 Subject: [PATCH 2/2] Address CodeRabbit review feedback - Split cloud_id out of Required fields into its own Extras package column - Expand MWAA step 4 into two steps: ARN binding + Access Role assignment - Use absolute URL for cross-reference link Co-authored-by: Cursor --- .../apache-airflow-provider.md | 26 ++++++++++--------- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md b/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md index e2e34a1fe..698554385 100644 --- a/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md +++ b/docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md @@ -54,16 +54,16 @@ pip install apache-airflow-providers-akeyless[cloud_id] The provider supports the following Akeyless [Authentication Methods](https://docs.akeyless.io/docs/access-and-authentication-methods): -| `access_type` | Required fields | Supported by | -| --- | --- | --- | -| `api_key` _default_ | `access_id`, `access_key` | Hook, Secrets Backend | -| `aws_iam` | `access_id` + `cloud_id` extras package | Hook, Secrets Backend | -| `gcp` | `access_id` + `cloud_id` extras package; optional: `gcp_audience` | Hook, Secrets Backend | -| `azure_ad` | `access_id` + `cloud_id` extras package; optional: `azure_object_id` | Hook, Secrets Backend | -| `uid` | `uid_token` | Hook, Secrets Backend | -| `jwt` | `access_id`, `jwt` | Hook only | -| `k8s` | `access_id`, `k8s_auth_config_name` | Hook only | -| `certificate` | `access_id`, `certificate_data`, `private_key_data` | Hook only | +| `access_type` | Required fields | Extras package | Supported by | +| --- | --- | --- | --- | +| `api_key` _default_ | `access_id`, `access_key` | — | Hook, Secrets Backend | +| `aws_iam` | `access_id` | `cloud_id` | Hook, Secrets Backend | +| `gcp` | `access_id`; optional: `gcp_audience` | `cloud_id` | Hook, Secrets Backend | +| `azure_ad` | `access_id`; optional: `azure_object_id` | `cloud_id` | Hook, Secrets Backend | +| `uid` | `uid_token` | — | Hook, Secrets Backend | +| `jwt` | `access_id`, `jwt` | — | Hook only | +| `k8s` | `access_id`, `k8s_auth_config_name` | — | Hook only | +| `certificate` | `access_id`, `certificate_data`, `private_key_data` | — | Hook only | > ⚠️ **Unsupported authentication methods:** The following Akeyless authentication methods are **not** supported by this provider: OCI IAM, Kerberos, LDAP, SAML, OIDC, and Email. @@ -257,7 +257,9 @@ backend_kwargs = { 3. Ensure the MWAA VPC has outbound HTTPS access to your Akeyless API endpoint (`api.akeyless.io` or your Akeyless Gateway). -4. Create an Akeyless `aws_iam` Authentication Method associated with the MWAA execution role ARN. +4. In Akeyless, create an `aws_iam` [Authentication Method](https://docs.akeyless.io/docs/aws-iam-auth-method) bounded to the MWAA execution role ARN. + +5. Assign the Authentication Method to an Akeyless [Access Role](https://docs.akeyless.io/docs/rbac#access-roles) that has **read** permissions on the configured secret paths (for example, `/airflow/connections/*`, `/airflow/variables/*`). #### Naming Convention @@ -315,7 +317,7 @@ value = hook.get_secret_value("/my/secret") Set the connection `access_type` extra field to `aws_iam` and install the `cloud_id` extras. The hook authenticates using the workload's AWS IAM identity (EC2 instance profile, ECS task role, and so on) — no static credentials required. -For Secrets Backend cloud authentication configuration examples, see [Cloud-based authentication in the Secrets Backend](#cloud-based-authentication-in-the-secrets-backend). +For Secrets Backend cloud authentication configuration examples, see [Cloud-based authentication in the Secrets Backend](https://docs.akeyless.io/docs/apache-airflow-provider#cloud-based-authentication-in-the-secrets-backend). ## Troubleshooting