fix: added OpenSearch security calls under feature flag#1325
Open
lucaseduoli wants to merge 12 commits intomainfrom
Open
fix: added OpenSearch security calls under feature flag#1325lucaseduoli wants to merge 12 commits intomainfrom
lucaseduoli wants to merge 12 commits intomainfrom
Conversation
… security script on opensearch
* Added IBM security calls * Made opensearch security calls be universal, not only with IBM * remove init index when ready * updated docker files to copy security files to backend and to not run security script on opensearch * added index name to index patterns dynamically * remove comment to be able to build opensearch * Revert changes to docker copose and dockerfile for security * re-added image to opensearch and backend * comment out opensearch build * Add cloud_securityconfig and IBM auth support Add a cloud-specific OpenSearch security config and wiring for IBM-managed deployments. - Add cloud_securityconfig/* YAML files (action_groups, config, internal_users, nodes_dn, roles, roles_mapping, tenants) to define OIDC, DLS roles, and tenant defaults for cloud environments. - Update Dockerfile and Dockerfile.backend to copy cloud_securityconfig into the image and set ownership. - Update src/utils/opensearch_utils.py to read IBM_AUTH_ENABLED from settings, choose between securityconfig and cloud_securityconfig, log the chosen config dir, and improve all_access rolesmapping handling by: - preserving existing all_access users when merging, - defaulting backend_roles to avoid unsafe "all_access" role in backend_roles, - removing any "all_access" backend_role to prevent IBM API key users from gaining super-admin privileges and bypassing DLS. - Move test.sh into scripts/test.sh (rename). Purpose: support IBM/cloud authentication mode while preserving intended DLS and admin mappings in OpenSearch and avoid granting unintended super-admin access. * Update test.sh * Merge hosts & backend_roles into all_access When preserving existing all_access mapping, also merge hosts and backend_roles in addition to users. Handle None values by defaulting users/hosts/backend_roles to empty lists, filter out the literal all_access backend role from existing backend_roles before merging, and add debug logging for preserved hosts and backend_roles. This ensures existing OpenSearch security mappings are retained safely when updating. * Update roles.yml * Pin onboarding admin in OpenSearch security Propagate the onboarding user's OpenSearch username into index/security initialization and pin them into the all_access role mapping. Added an optional admin_username parameter to init_index and setup_opensearch_security, passed from onboarding when IBM_AUTH_ENABLED. setup_opensearch_security now appends the provided admin user to the all_access "users" list (if missing) and logs the action so the onboarding user retains admin access after DLS/backend_roles changes. * Use user_id for OpenSearch admin username When initializing the OpenSearch index after onboarding, pass user.user_id as the admin_username if IBM_AUTH_ENABLED. This replaces the previous use of user.opensearch_username (which may be missing or inconsistent) so the init_index call receives the correct identifier. --------- Co-authored-by: Lucas Oliveira <lucas.edu.oli@hotmail.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Grant additional index and cluster permissions for openrag_user_role in both securityconfig and cloud_securityconfig: add indices:admin/refresh and indices:data/write/delete|update/byquery, introduce an api_keys (api_keys*) entry with CRUD and mappings permissions constrained by a DLS on user_id, and allow CRUD on .opendistro-alerting-config. Also add cluster:monitor/* to securityconfig and update .gitignore to ignore /opensearch-data2.
github-actions
bot
added
backend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new OpenSearch security configuration for cloud deployments, improves document-level security (DLS) role definitions, and adds a comprehensive test script for validating ACL and DLS functionality. It also updates Dockerfiles to include the new configuration files and makes a minor fix in the onboarding process for OpenSearch index initialization.
Cloud OpenSearch Security Configuration:
cloud_securityconfigdirectory with configuration files for OpenSearch security, including action groups, roles, roles mapping, internal users, tenants, and node DNs. These files define DLS rules, OpenID authentication, and role mappings tailored for cloud environments. [1] [2] [3] [4] [5] [6] [7]Dockerfile and Build Updates:
DockerfileandDockerfile.backendto copy bothsecurityconfigand the newcloud_securityconfigdirectories into the images, and set correct ownership permissions. This ensures the new security configuration is available in both OpenSearch and backend containers. [1] [2] [3]Role and Permission Enhancements:
openrag_user_rolein bothsecurityconfig/roles.ymlandcloud_securityconfig/roles.ymlto include additional cluster and index permissions, DLS filters, and support for API key indices. This strengthens document access controls and enables fine-grained security. [1] [2] [3]Testing and Validation:
scripts/test.sh, a robust shell script that tests OpenSearch ACLs and DLS by verifying user roles, index mappings, document visibility, and application-level ACL filters for multiple users. This script helps ensure that security rules are enforced as expected.OpenSearch Index Initialization Fix:
src/api/settings.pyto use the correctinit_indexfunction and pass the appropriate admin username, improving reliability when initializing OpenSearch indices after onboarding. [1] [2]