diff --git a/stps/sig-iuo/CNV-63822-role-aggregation-opt-out.md b/stps/sig-iuo/CNV-63822-role-aggregation-opt-out.md new file mode 100644 index 0000000..724064c --- /dev/null +++ b/stps/sig-iuo/CNV-63822-role-aggregation-opt-out.md @@ -0,0 +1,318 @@ +# Openshift-virtualization-tests Test plan + +## **Role Aggregation Opt-Out - Quality Engineering Plan** + +### **Metadata & Tracking** + +- **Enhancement(s):** [Kubevirt VEP](https://github.com/kubevirt/enhancements/issues/160) +- **Feature Tracking:** [CNV-50792](https://issues.redhat.com/browse/CNV-50792) +- **Epic Tracking:** [CNV-63822](https://issues.redhat.com/browse/CNV-63822) +- **Feature Maturity:** + - DP: 4.22 + - TP: 4.23/5.0 + - GA: 5.1 +- **QE Owner(s):** Ramon Lobillo (@rlobillo), Alex Barker (@albarker-rh) +- **Owning SIG:** sig-iuo (Install, Upgrade, Operators) +- **Participating SIGs:** sig-ui + +**Document Conventions (if applicable):** N/A — no feature-specific terms required. + +### **Feature Overview** + +By default, all project administrators, editors, and viewers automatically receive access to +OpenShift Virtualization resources. Role Aggregation Opt-Out allows cluster administrators to +disable this automatic access and instead grant virtualization permissions explicitly per user +and namespace, enabling fine-grained control in multi-tenant environments. +This STP covers testing for the Tech Preview phase (4.23/5.0). + +--- + +### **I. Motivation and Requirements Review (QE Review Guidelines)** + +This section documents the mandatory QE review process. The goal is to understand the feature's value, +technology, and testability before formal test planning. + +#### **1. Requirement & User Story Review Checklist** + +- [x] **Review Requirements** + - *List the key D/S requirements reviewed:* Cluster admins can limit the access to virtualization components + +- [x] **Understand Value and Customer Use Cases** + - *Describe the feature's value to customers:* Enables tenant isolation by requiring + explicit virtualization access grants instead of automatic role aggregation. + - *List the customer use cases identified:* + - As a cluster administrator, I want to disable automatic virtualization access + so that only explicitly authorized users can use and interact with virtualization resources + +- [x] **Testability** + - *Note any requirements that are unclear or untestable:* All requirements are testable + through standard API and RBAC validation. + +- [x] **Acceptance Criteria** + - *List the acceptance criteria:* + - A cluster administrator can control the role aggregation strategy and any change to the setting takes effect on the virtualization deployment + - When opt-out is enabled, users with project admin, edit, or view roles in a namespace are forbidden from performing actions on virtualization resources + - When opt-out is disabled after being enabled, automatic access is restored for users who were previously blocked + - *Note any gaps or missing criteria:* None. Defined in CNV-63822 epic. + +- [x] **Non-Functional Requirements (NFRs)** + - *List applicable NFRs and their targets:* + - Security: RBAC hardening — users blocked without explicit grant + - Backward Compatibility: default unchanged + - UI: console changes tracked under CNV-80935 + - Docs: upstream docs available; downstream docs planned for 4.22 + - *Note any NFRs not covered and why:* + - Performance: N/A — negligible RBAC overhead + - Monitoring: N/A — no new metrics/alerts, uses standard Kubernetes RBAC + - Scalability: N/A — scales with Kubernetes natively + - Observability: N/A — standard audit logging covers RBAC decisions + +#### **2. Known Limitations** + +The limitations are documented to ensure alignment between development, QA, and product teams. +The following are confirmed product constraints accepted before testing begins. + +None — reviewed and confirmed with [Name/Date — TBD] that no feature limitations apply for +this release. + +#### **3. Technology and Design Review** + +- [x] **Developer Handoff/QE Kickoff** + - *Key takeaways and concerns:* We agreed on the testing strategy and configuration requirements. + +- [x] **Technology Challenges** + - *List identified challenges:* N/A + - *Impact on testing approach:* N/A + +- [x] **API Extensions** + - *List new or modified APIs:* New cluster-level configuration field to control role + aggregation behavior (default: enabled, opt-out: manual). + - *Testing impact:* Tests must validate config changes and downstream RBAC effects. + +- [x] **Test Environment Needs** + - *See environment requirements in Section II.3 and testing tools in Section II.3.1* + +- [x] **Topology Considerations** + - *Describe topology requirements:* Feature is cluster-scoped and topology-independent. + - *Impact on test design:* Works on all topologies (standard, SNO, compact). + +### **II. Software Test Plan (STP)** + +This STP serves as the **overall roadmap for testing**, detailing the scope, approach, resources, +and schedule. + +#### **1. Scope of Testing** + +**Testing Goals** + +- **[P0]** Verify a cluster administrator can enable role aggregation opt-out and the setting is applied to the virtualization deployment +- **[P0]** Verify a cluster administrator can switch the aggregation mode and the change propagates to the virtualization deployment +- **[P0]** Verify that removing the aggregation configuration resets the virtualization deployment to its original unconfigured state +- **[P0]** Verify that when opt-out is enabled, an unprivileged user with a project admin role cannot perform virtualization admin actions (receives Forbidden error) +- **[P0]** Verify that when opt-out is enabled, an unprivileged user with an edit role cannot perform virtualization edit actions (receives Forbidden error) +- **[P0]** Verify that when opt-out is enabled, an unprivileged user with a view role cannot perform virtualization view actions (receives Forbidden error) +- **[P0]** Verify that disabling opt-out after it was enabled restores automatic access for users who were previously blocked + +**Regression Goals** + +- **[P0]** Verify existing RBAC and migration functionality is not broken by the new feature — tier 2 regression suites run on the feature cluster, including migration rights tests + +**Out of Scope (Testing Scope Exclusions)** + +The following items are explicitly Out of Scope for this test cycle and represent intentional +exclusions. No verification activities will be performed for these items, and any related issues +found will not be classified as defects for this release. + +- **Testing OpenShift RBAC infrastructure itself** + - *Rationale:* Core RBAC evaluation is the responsibility of the OCP platform team; no duplication of their test effort + - *PM/Lead Agreement:* [Name/Date — TBD] + +- **Testing all individual permission rules within virtualization roles** + - *Rationale:* Individual role rules are not affected by this feature; this feature controls whether roles are aggregated, not the content of the roles themselves + - *PM/Lead Agreement:* [Name/Date — TBD] + +- **External IdP compatibility (LDAP, Active Directory)** + - *Rationale:* RBAC logic is IdP-agnostic; HTPasswd testing validates the core permission logic + - *PM/Lead Agreement:* [Name/Date — TBD] + +- **Multi-tenant cluster scale testing (100+ users)** + - *Rationale:* RBAC evaluation overhead is negligible; functional correctness at smaller scale is sufficient + - *PM/Lead Agreement:* [Name/Date — TBD] + +**Test Limitations** + +None — reviewed and confirmed that no test limitations apply for this release. +*Sign-off:* [Name/Date — TBD] + +#### **2. Test Strategy** + +**Functional** + +- [x] **Functional Testing** — Validates that the feature works according to specified requirements and user stories + - *Details:* Core focus: verify opt-out configuration, RBAC enforcement, and default behavior preservation. + +- [x] **Automation Testing** — Confirms test automation plan is in place for CI and regression coverage (all tests are expected to be automated) + - *Details:* All tier 1 and tier 2 tests automated; tier 1 validates configuration, tier 2 validates end-to-end user workflows. + +- [x] **Regression Testing** — Verifies that new changes do not break existing functionality + - *Details:* Migrate role aggregation is already covered by existing tier 2 regression tests. + +**Non-Functional** + +- [ ] **Performance Testing** — Validates feature performance meets requirements (latency, throughput, resource usage) + - *Details:* N/A — feature adds no performance-sensitive operations; RBAC evaluation overhead is negligible. + +- [ ] **Scale Testing** — Validates feature behavior under increased load and at production-like scale (e.g., large number of VMs, nodes, or concurrent operations) + - *Details:* N/A — Kubernetes RBAC scales natively; feature does not introduce new scalability concerns. + +- [x] **Security Testing** — Verifies security requirements, RBAC, authentication, authorization, and vulnerability scanning + - *Details:* Feature is a security enhancement; tests verify users are correctly blocked when opt-out is enabled for all 3 role levels. + +- [ ] **Usability Testing** — Validates user experience and accessibility requirements + - *Details:* UI changes tracked under [CNV-80935](https://issues.redhat.com/browse/CNV-80935); UI team (sig-ui) owns console testing for opt-out configuration. + +- [ ] **Monitoring** — Does the feature require metrics and/or alerts? + - *Details:* No new metrics or alerts required; feature uses standard Kubernetes RBAC. + +**Integration & Compatibility** + +- [x] **Compatibility Testing** — Ensures feature works across supported platforms, versions, and configurations + - *Details:* Default behavior unchanged; backward compatibility with previous API versions maintained. + +- [x] **Upgrade Testing** — Validates upgrade paths from previous versions, data migration, and configuration preservation + - *Details:* Verify default behavior preserved across z-stream upgrades; verify opt-out config persists after upgrade. + +- [ ] **Dependencies** — Blocked by deliverables from other components/products. Identify what we need from other teams before we can test. + - *Details:* No blocking dependencies; upstream and downstream implementations are complete. + +- [x] **Cross Integrations** — Does the feature affect other features or require testing by other teams? Identify the impact we cause. + - *Details:* UI team (sig-ui) needs to implement and test console changes ([CNV-80935](https://issues.redhat.com/browse/CNV-80935)). + +**Infrastructure** + +- [ ] **Cloud Testing** — Does the feature require multi-cloud platform testing? Consider cloud-specific features. + - *Details:* N/A — feature is RBAC-based and platform-independent; no cloud-specific behavior. + +#### **3. Test Environment** + +- **Cluster Topology:** Standard or SNO — feature works on all topologies; multi-node preferred + +- **OCP & OpenShift Virtualization Version(s):** OCP 4.22 with OpenShift Virtualization 4.22 + +- **CPU Virtualization:** N/A — not relevant for RBAC testing + +- **Compute Resources:** Minimum per worker node: 4 vCPUs, 16GB RAM + +- **Special Hardware:** N/A — no special hardware required + +- **Storage:** Any RWX storage class (e.g., ocs-storagecluster-ceph-rbd-virtualization) + +- **Network:** OVN-Kubernetes, IPv4 — no special network requirements + +- **Required Operators:** OpenShift Virtualization (standard installation) + +- **Platform:** Any supported platform (bare metal, AWS, Azure, GCP — no platform-specific behavior) + +- **Special Configurations:** HTPasswd identity provider — REQUIRED: Must have HTPasswd IdP with unprivileged user + +#### **3.1. Testing Tools & Frameworks** + +- **Test Framework:** Standard + +- **CI/CD:** N/A + +- **Other Tools:** N/A + +#### **4. Entry Criteria** + +The following conditions must be met before testing can begin: + +- [ ] Requirements and design documents are **approved and merged** +- [ ] Test environment can be **set up and configured** (see Section II.3 - Test Environment) +- [x] Upstream implementation **merged** (role aggregation opt-out support) +- [x] Downstream implementation **complete** (configuration field available in cluster settings) +- [x] Developer Handoff/QE Kickoff meeting completed + +#### **5. Risks** + +**Timeline/Schedule** + +- **Risk:** N/A + - **Mitigation:** Feature implementation is complete upstream and downstream; no schedule risk. + +**Test Coverage** + +- **Risk:** Cannot exhaustively test all RBAC role combinations and permission permutations. + - **Mitigation:** Focus on the 3 critical role levels (admin, edit, view) covering acceptance criteria; individual permission rules within roles are unaffected by this feature. + - *Areas with reduced coverage:* Individual permission rules within each virtualization role; only role-level access is validated. + - *Sign-off:* [Name/Date — TBD] + +**Test Environment** + +- **Risk:** N/A + - **Mitigation:** Standard OCP cluster with HTPasswd IdP is sufficient; no special hardware or infrastructure required. + +**Untestable Aspects** + +- **Risk:** Cannot test with production identity providers (LDAP, Active Directory, OAuth) in the lab. + - **Mitigation:** RBAC logic is IdP-agnostic; HTPasswd validation covers the enforcement path regardless of IdP. + - *Alternative validation approach:* Functional validation with HTPasswd covers the RBAC enforcement path regardless of IdP. + - *Sign-off:* [Name/Date — TBD] + +**Resource Constraints** + +- **Risk:** N/A + - **Mitigation:** Feature testing scope is manageable with assigned QE resources; no staffing or capacity constraints. + +**Dependencies** + +- **Risk:** UI changes ([CNV-80935](https://issues.redhat.com/browse/CNV-80935)) are pending; console configuration interface may not be ready for testing. + - **Mitigation:** Track progress with UI team (sig-ui); API-level testing can proceed independently of UI. + - *Dependent teams or components:* sig-ui — console configuration interface for opt-out + - *Sign-off:* [Name/Date — TBD] + + +--- + +### **III. Test Scenarios & Traceability** + +- **[CNV-63822]** — As a cluster admin, I want to control the role aggregation strategy for virtualization resources + - *Test Scenario:* [Tier 1] Verify that enabling opt-out mode applies the setting to the virtualization deployment + - *Priority:* P0 + + - *Test Scenario:* [Tier 1] Verify that changing the aggregation mode propagates the updated setting to the virtualization deployment + - *Priority:* P0 + + - *Test Scenario:* [Tier 1] Verify that removing the aggregation configuration resets the virtualization deployment to its original state + - *Priority:* P0 + +- **[CNV-63822]** — As a cluster admin, I want to enable opt-out so unprivileged users cannot access virtualization resources + - *Test Scenario:* [Tier 2] Verify an unprivileged user with project admin role cannot perform virtualization admin actions when opt-out is enabled (receives Forbidden error) + - *Priority:* P0 + + - *Test Scenario:* [Tier 2] Verify an unprivileged user with edit role cannot perform virtualization edit actions when opt-out is enabled (receives Forbidden error) + - *Priority:* P0 + + - *Test Scenario:* [Tier 2] Verify an unprivileged user with view role cannot perform virtualization view actions when opt-out is enabled (receives Forbidden error) + - *Priority:* P0 + +- **[CNV-63822]** — As a cluster admin, I want to disable opt-out to restore default behavior + - *Test Scenario:* [Tier 2] Verify that disabling opt-out restores automatic access for previously blocked users + - *Priority:* P0 + +--- + +### **IV. Sign-off and Approval** + +This Software Test Plan requires approval from the following stakeholders: + +* **Reviewers:** + - QE Lead / @rnester + - sig-iuo representative / @orenc1 @hmeir @OhadRevah @rlobillo + - sig-ui representative / @upalatucci + +* **Approvers:** + - QE Lead / @rnetser + - sig-iuo representative / @hmeir + - QE Manager / @kmajcher-rh + - Product Manager / Ronen Sde-Or @ronensdeor