Update Airflow provider: cloud auth now supported in Secrets Backend - #319
Conversation
…kend The Secrets Backend now supports aws_iam, gcp, and azure_ad authentication in addition to api_key and uid. This enables seamless integration with managed Airflow services like Amazon MWAA and Google Managed Service for Apache Airflow without requiring static API keys. Changes: - Update capability table and auth methods table to reflect new support - Add cloud auth configuration examples for AWS IAM, GCP, and Azure AD - Add step-by-step Amazon MWAA setup guide - Remove outdated "Secrets Backend limitation" warning - Update troubleshooting section Co-authored-by: Cursor <cursoragent@cursor.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThe Apache Airflow provider documentation now states that AkeylessBackend supports AWS IAM, GCP, and Azure AD authentication, adds configuration examples and Amazon MWAA guidance, and updates cross-references and troubleshooting content. ChangesAirflow cloud authentication
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/Integrations` & Plugins/plugins-overview/apache-airflow-provider.md:
- Line 260: Update the checklist step describing the Akeyless aws_iam
Authentication Method to separately require binding it to the MWAA
execution-role ARN and assigning it to an Akeyless Access Role with read
permission for the configured secret paths.
- Around line 318-319: Update the “Cloud-based authentication in the Secrets
Backend” link in the documentation to use the page’s canonical absolute URL
while preserving the existing section anchor.
- Around line 60-62: The integrations table incorrectly lists cloud_id as a
required connection field. Update the aws_iam, gcp, and azure_ad rows so the
required-fields column contains access_id and any auth-specific optional fields,
while moving cloud_id to a dependency or package-extra note/column.
- Around line 256-258: Update the MWAA Akeyless configuration example around
secrets.backend_kwargs to include the Gateway-specific api_url value or an
explicit placeholder for the user’s Akeyless Gateway URL, while retaining the
SaaS endpoint example for the standard path. Ensure the surrounding instructions
clearly indicate that Gateway users must replace api_url with their Gateway URL.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 39f82489-04f5-4d9f-a144-7c668757064c
📒 Files selected for processing (1)
docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md
| | `secrets.backend_kwargs` | `{"api_url": "https://api.akeyless.io", "access_id": "<Access ID>", "access_type": "aws_iam"}` | | ||
|
|
||
| 3. Ensure the MWAA VPC has outbound HTTPS access to your Akeyless API endpoint (`api.akeyless.io` or your Akeyless Gateway). |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the relevant section with line numbers.
sed -n '230,280p' 'docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md'Repository: akeylesslabs/technical-documentation
Length of output: 1779
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Search for the Akeyless Airflow provider connection docs referenced by the comment.
rg -n "api_url|Gateway|secrets.backend_kwargs|access_type" 'docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md' 'docs' -g '!**/*.png'Repository: akeylesslabs/technical-documentation
Length of output: 50393
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# If the provider docs exist in-repo, inspect the relevant guidance for backend_kwargs / api_url.
fd -a 'connections.md' .Repository: akeylesslabs/technical-documentation
Length of output: 174
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the exact snippet context from the review target file with numbers.
nl -ba 'docs/Integrations & Plugins/plugins-overview/apache-airflow-provider.md' | sed -n '248,265p'Repository: akeylesslabs/technical-documentation
Length of output: 214
🌐 Web query:
airflow.apache.org docs akeyless provider connections api_url gateway backend_kwargs
💡 Result:
The apache-airflow-providers-akeyless package allows you to connect to the Akeyless Vault Platform using a dedicated akeyless connection type or as a secrets backend [1][2][3]. Akeyless Connection To configure an Akeyless connection, use the akeyless connection type [1][3]. The api_url field is used to specify the Akeyless API endpoint (e.g., https://api.akeyless.io) or your specific Akeyless Gateway URL [1][3]. Authentication details are provided via the login (Access ID), password (Access Key), and an Extra JSON field [1][3]. The Extra field supports various authentication methods defined by the access_type parameter (e.g., api_key, aws_iam, gcp, azure_ad, uid, jwt, k8s, certificate) [1][3]. Akeyless Secrets Backend When configuring Akeyless as a secrets backend in airflow.cfg or via environment variables, you use backend_kwargs to pass necessary configuration [4][5]. Example backend_kwargs configuration in airflow.cfg: [secrets] backend = airflow.providers.akeyless.secrets.akeyless.AkeylessBackend backend_kwargs = { "connections_path": "/airflow/connections", "variables_path": "/airflow/variables", "api_url": "https://api.akeyless.io", "access_id": "p-xxxxxxxxx", "access_key": "your-access-key", "access_type": "api_key" } Key parameters for backend_kwargs include [4][5]: - api_url: The Akeyless API endpoint (default: https://api.akeyless.io) [4][5]. - access_id: Your Akeyless Access ID [4][5]. - access_key: Your Akeyless Access Key (required for api_key authentication) [4][5]. - access_type: The authentication method (default: api_key) [4][5]. - connections_path, variables_path, config_path: Path prefixes in Akeyless for retrieving these Airflow objects [4][5]. Note: The Secrets Backend implementation supports api_key and uid authentication methods [3]. Additionally, in the connection settings, the field previously known as jwt was renamed to jwt_token to ensure proper masking in logs [6].
Citations:
- 1: https://airflow.apache.org/docs/apache-airflow-providers-akeyless/stable/connections.html
- 2: https://airflow.apache.org/docs/apache-airflow-providers-akeyless/stable/index.html
- 3: https://docs.akeyless.io/docs/apache-airflow-provider
- 4: https://airflow.apache.org/docs/apache-airflow-providers-akeyless/stable/secrets-backend.html
- 5: https://airflow.apache.org/docs/apache-airflow-providers-akeyless/stable/_api/airflow/providers/akeyless/secrets/akeyless/index.html
- 6: https://airflow.apache.org/docs/apache-airflow-providers-akeyless/stable/changelog.html
Add a Gateway-specific api_url example. The MWAA snippet still points to https://api.akeyless.io; if users follow the Gateway path, they need to replace api_url with their Gateway URL or the backend will keep using the SaaS endpoint.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/Integrations` & Plugins/plugins-overview/apache-airflow-provider.md
around lines 256 - 258, Update the MWAA Akeyless configuration example around
secrets.backend_kwargs to include the Gateway-specific api_url value or an
explicit placeholder for the user’s Akeyless Gateway URL, while retaining the
SaaS endpoint example for the standard path. Ensure the surrounding instructions
clearly indicate that Gateway users must replace api_url with their Gateway URL.
- Split cloud_id out of Required fields into its own Extras package column - Expand MWAA step 4 into two steps: ARN binding + Access Role assignment - Use absolute URL for cross-reference link Co-authored-by: Cursor <cursoragent@cursor.com>
2 participants
Summary
Updates the Apache Airflow provider documentation to reflect that the Secrets Backend now supports
aws_iam,gcp, andazure_adauthentication (in addition toapi_keyanduid).This corresponds to the upstream Airflow PR: apache/airflow#69772
Changes
aws_iam,gcp,azure_adfrom "Hook only" to "Hook, Secrets Backend"ValueErrorentry to list all supported auth typesMade with Cursor
Summary by CodeRabbit
aws_iam,gcp, andazure_adalongside existingapi_keyanduid.airflow.cfg/backend_kwargsexamples.