Test: Add IBM Watsonx Data (AMS) auth support#1263
Draft
edwinjosechittilappilly wants to merge 40 commits intomainfrom
Draft
Test: Add IBM Watsonx Data (AMS) auth support#1263edwinjosechittilappilly wants to merge 40 commits intomainfrom
edwinjosechittilappilly wants to merge 40 commits intomainfrom
Conversation
Introduce support for IBM Watsonx Data (AMS) embedded authentication via the ibm-lh-console-session cookie. Add IBM_AUTH_ENABLED and IBM_JWT_PUBLIC_KEY_URL settings and .env.example entries; pre-fetch and cache IBM JWT public key at startup. Implement JWT validation and key rotation handling in new auth.ibm_auth module. Wire IBM auth into dependencies (extract/validate cookie, set request.state.user), main service initialization, auth_service (prevent Google OAuth login when IBM auth is active and expose ibm_auth_mode in user info), and session_manager (use IBM JWT as-is). Update frontend auth context and login page to recognize IBM auth mode and disable local login/logout and redirect accordingly. Adjust logout endpoint to no-op when IBM auth is enabled.
Update IBM public key fetching to expect a JSON payload with a "public_key" field instead of using raw response content. Adds validation and logging when the key is missing, encodes the key if returned as a string, and continues to cache the loaded PEM public key. Raises a ValueError on missing key to surface the error earlier.
init_oauth() raises ValueError when IBM auth mode is enabled, but /auth/init currently catches all exceptions and returns HTTP 500. This makes a normal “login not supported in IBM mode” case look like a server error. Prefer raising an HTTPException(status_code=400/409, ...) here, or update auth_init to translate ValueError into a 4xx response.
create_user_opensearch_client() now calls jwt_token.startswith(...), which will raise if jwt_token is None. There are existing call sites that pass jwt_token=None (e.g., API key auth / internal tasks), and while SessionManager.get_effective_jwt_token() often fills this in, it can still return None for unknown users. Consider handling None explicitly (e.g., raise a clear error or default to Bearer wrapping only when jwt_token is a non-empty string) to avoid an unexpected AttributeError.
decode_ibm_jwt() only catches jwt.DecodeError, but expired tokens (and many other invalid token conditions) raise ExpiredSignatureError / InvalidTokenError and will currently bubble up, likely turning a normal “not authenticated” case into a 500. Consider catching jwt.InvalidTokenError (or jwt.PyJWTError) and returning None consistently.
When IBM_AUTH_ENABLED, return a "partial_logout" response that clears IBM-related browser cookies but warns that the server-side IBM/AMS session may still be active and should be terminated via the identity provider. The default "logged_out" response is now returned only when IBM auth is not enabled. Also added a comment about not requiring secure=True for local development when setting the "ibm-auth-basic" cookie in ibm_login.
Add support for Traefik/AMS header-based IBM authentication and persist per-user Basic credentials.
- Frontend: remove the IBM username/password form and ensure login redirect also considers IBM auth mode.
- Config: add IBM_CREDENTIALS_HEADER setting (default X-IBM-LH-Credentials).
- Connectors: add ConnectionManager.upsert_ibm_credentials to store/update ibm_credentials connector entries in connections.json.
- Dependencies: make _get_ibm_user async and handle a configurable credentials header (decode Basic token, derive username, persist credentials via connector_service), then short-circuit and return a constructed User; update get_current_user and get_optional_user to await the IBM auth helper.
- Session manager / User model: add opensearch_username and opensearch_credentials fields; when IBM auth is enabled, always create a fresh OpenSearch client (do not reuse cached client) to handle per-request rotating credentials.
These changes enable production deployments where Traefik injects IBM credentials, allow background processes to reuse those credentials, and align session/client behavior for IBM AMS auth.
Backend:
- src/config/settings.py — Added IBM_CREDENTIALS_HEADER = os.getenv("IBM_CREDENTIALS_HEADER", "X-IBM-LH-Credentials")
- src/session_manager.py — Added opensearch_username and opensearch_credentials fields to User; updated get_user_opensearch_client to skip the cache in IBM mode so credential rotation is always reflected
- src/connectors/connection_manager.py — Added upsert_ibm_credentials() async method to store/update per-user IBM credentials in connections.json with connector_type="ibm_credentials"
- src/dependencies.py — Made _get_ibm_user, get_current_user, get_optional_user all async; added Option 0 (highest priority) that reads the configurable IBM_CREDENTIALS_HEADER, decodes the base64 credentials, persists to
connections.json, and sets jwt_token to "Basic <b64>" so OpenSearch gets the right Authorization header
Frontend:
- frontend/app/login/page.tsx — Removed IbmLoginForm component; added isIbmAuthMode to both the redirect useEffect and early-return guard so IBM users are immediately bounced to /chat without seeing any login form
Add handling for IBM Basic credentials and propagate them into Langflow/MCP global variables. Introduces extract_ibm_credentials() in auth/ibm_auth.py to decode "Basic <b64>" strings and reuse decoded username/password. dependencies.py now uses this helper and stores opensearch_username/opensearch_credentials on the User when present. Pass jwt_token through ChatService and LangflowFileService to add_provider_credentials_to_headers, which now accepts jwt_token and builds IBM OpenSearch vars via build_ibm_opensearch_vars(). build_mcp_global_vars_from_config also accepts jwt_token and injects OPENSEARCH_USERNAME, OPENSEARCH_PASSWORD and JWT when IBM_AUTH_ENABLED and a Basic credential is provided. This enables Langflow and MCP components to authenticate OpenSearch in IBM deployments.
Decode IBM JWT cookie early to extract user_id, email, and name and reuse those values across auth flows. Persist Basic (lh) credentials only when a user_id is available. When the Basic header is missing, attempt to load stored Basic credentials from the connector service for the JWT-identified user and derive OpenSearch username/credentials from them. This reduces duplicate JWT decoding, ensures connector upserts use a stable user_id, and consolidates user construction for header- and cookie-based authentication paths.
Change default UI and flow settings and normalize token/credentials handling: - Set bearer_prefix default to false in OpenSearch multimodal component to avoid adding a client-side "Bearer " prefix. - Flip several flow boolean defaults from true to false in ingestion_flow.json, openrag_agent.json, openrag_nudges.json, and openrag_url_mcp.json (disable the previously enabled flags). - Simplify dependencies: forward lh_credentials directly as opensearch_credentials when building the User object. - Prefix generated JWT with "Bearer " in session_manager so returned tokens are ready for Authorization headers. These changes ensure consistent auth header formatting and adjust default telemetry/boolean flags across flows.
…r IBM Users Create and propagate an optional per-user OpenSearch client and add IBM authentication header validation. - settings.onboarding: when IBM auth is enabled and a user JWT exists, build a user-specific OpenSearch client and pass it into index initialization. - main: make wait_for_opensearch, init_index and init_index_when_ready accept an optional opensearch_client, use the provided client or fall back to the global client, and use a local os_client in init_index for index operations. Preserve admin/global client usage for alerting security. - dependencies: add checks that the IBM credentials header exists and is a Basic credential (return 401 otherwise); small formatting/whitespace cleanups and clearer dataclasses.replace formatting when attaching effective JWT to the request user. These changes allow initializing and operating OpenSearch using per-user credentials when applicable and improve IBM AMS auth error handling.
Remove early warnings and 401 responses for missing/invalid IBM credentials header so other auth paths (cookies or configured connections) can be used. Move validation later: require the X-IBM-LH-Credentials header before extracting Basic credentials and raise a clearer 401 message if it's not present or not Basic-formatted. This reduces noisy logs and allows alternative credential sources to be consulted first.
Introduce a new /unauthorized client page to show an authentication-required UI (with IBM-specific messaging and a retry button). Update ProtectedRoute to read isIbmAuthMode from auth context, adjust loading logic, and redirect IBM-authenticated flows to /unauthorized instead of /login; keep previous behavior for no-auth and normal auth modes. Also refine effect dependencies and early returns to avoid unnecessary redirects while loading.
Insert warning logs into _get_ibm_user to output request headers and cookies for debugging the IBM authentication flow. Includes a TODO to remove these logs after testing; note this can expose sensitive data and should be removed or sanitized before production.
Stop raising a 401 when IBM LH Basic credentials are absent and instead prefer a JWT fallback. Introduce opensearch_username/opensearch_credentials variables and set jwt_token to Bearer {ibm_token} by default; if a Basic header is present, use it for both jwt_token and opensearch credentials. Add warnings when falling back to JWT or when a JWT exists but no user_id can be extracted. Ensure the created User object reflects the correct jwt_token and opensearch_credentials values.
Add IBM_AUTH_ENABLED gating to startup tasks so OpenSearch connection/index initialization is skipped when IBM auth mode is active. When not enabled, preserves previous behavior: wait_for_opensearch(), optionally ensure index if DISABLE_INGEST_WITH_LANGFLOW, and ensure index exists after onboarding (with improved logging and error handling). configure_alerting_security is now executed only in the non-IBM-auth path. Raises on failures as before.
Propagate a jwt_token through the default document ingestion pipeline to support IBM auth. settings.py now supplies the user's jwt_token during onboarding when IBM_AUTH_ENABLED. Multiple ingestion functions in main.py (ingest_openrag_docs_when_ready, ingest_default_documents_when_ready, _ingest_default_documents_langflow, _ingest_default_documents_url_langflow, _ingest_default_documents_url, _ingest_default_documents_openrag) accept an optional jwt_token and pass it to downstream processors (including DocumentFileProcessor and Langflow ingestion calls). Anonymous JWT handling was adjusted to prefer a provided jwt_token and otherwise let the session_manager create and expose an anonymous JWT.
Allow running with IBM-managed OpenSearch auth by using no static http_auth when IBM_AUTH_ENABLED, and relax the OPENSEARCH_PASSWORD requirement for Langflow client initialization in that mode. Also add a readiness short-circuit in the OpenSearch health endpoint: when IBM_AUTH_ENABLED is set the endpoint returns ready immediately with a note indicating per-request auth handling. These changes enable deployments that use IBM token-based authentication instead of static user/password credentials.
#1177) settings.py calls current_config.providers.any_configured() but ProvidersConfig had no such method, causing an AttributeError that surfaces as a 500 error during onboarding when configuring watsonx.ai or ollama providers. Closes #1163 Made-with: Cursor Co-authored-by: themavik <themavik@users.noreply.github.com>
Makefile: When generating the test JWT for OpenSearch, strip the leading 'Bearer ' prefix from the printed token using Python's removeprefix so the token can be used directly. src/config/settings.py: Add an async close() alias that calls cleanup() on AppClients for convenience and clearer intent when closing clients.
Allow IBM credentials to be provided as either 'Basic <base64>' or raw '<base64>' and also accept Bearer JWTs. Updated extract_ibm_credentials to decode raw base64 or Basic-prefixed input. Adjusted dependency logic to treat any non-empty lh_credentials as valid, and normalize jwt_token to include the 'Basic ' prefix when appropriate. Clarified session_manager docs to indicate opensearch_credentials store raw base64 (no 'Basic ' prefix). Updated langflow_headers to rename param to credentials and to emit OPENSEARCH_USERNAME/OPENSEARCH_PASSWORD only for Basic credentials while always exporting JWT (supports Bearer tokens as well). Changes apply to src/auth/ibm_auth.py, src/dependencies.py, src/session_manager.py, and src/utils/langflow_headers.py.
Introduce APIKeyService._get_opensearch_client(jwt_token) to centralize OpenSearch client selection. When IBM auth is enabled and a JWT is provided (and session_manager is present), the service will create a user-authenticated OpenSearch client; otherwise it falls back to the default admin client. Replace previous inline imports/usages of clients.opensearch across key creation, lookup, listing and revocation to use the helper. Also update the /keys endpoint to pass user.jwt_token into list_keys so per-user clients can be used.
github-actions
bot
added
frontend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce support for IBM Watsonx Data (AMS) embedded authentication via the ibm-lh-console-session cookie. Add IBM_AUTH_ENABLED and IBM_JWT_PUBLIC_KEY_URL settings and .env.example entries; pre-fetch and cache IBM JWT public key at startup. Implement JWT validation and key rotation handling in new auth.ibm_auth module. Wire IBM auth into dependencies (extract/validate cookie, set request.state.user), main service initialization, auth_service (prevent Google OAuth login when IBM auth is active and expose ibm_auth_mode in user info), and session_manager (use IBM JWT as-is). Update frontend auth context and login page to recognize IBM auth mode and disable local login/logout and redirect accordingly. Adjust logout endpoint to no-op when IBM auth is enabled.