Split StorageNodeSet into StorageNode + StorageNodeOps CRDs

[noctarius]

Today StorageNodeSet (storage.simplyblock.io/v1alpha1) is overloaded. A single CRD carries three unrelated concerns:

1. Grouping / fleet management: adding multiple storage nodes to a cluster at once, and rolling restarts/updates across them.
2. Per-node configuration: SPDK settings, PCIe filters, NUMA layout, device selection, resources, etc.
3. Imperative actions: shutdown, restart, suspend, resume, remove driven through the mutable spec.action field.

Mixing declarative per-node config with imperative, one-shot actions on the same object is the core problem. Actions require mutating a long-lived spec (spec.action + spec.nodeUUID), which fights kubectl apply/GitOps, has no natural completion/history, and forces the set-level reconciler to branch between "manage the fleet" and "run an action on one node." Per-node config also can't diverge from the set without awkward overrides.

This issue proposes splitting the responsibilities into three CRDs:

• StorageNode: owns all storage-node-specific configuration and observed state for a single node.
• StorageNodeOps: a one-shot, imperative action targeting a single StorageNode (like the existing VolumeMigration pattern).
• StorageNodeSet: stays, but purely as a grouping/template resource that provisions and rolls updates/restarts across a set of StorageNodes.

Motivation

• Declarative vs. imperative separation. Actions become immutable, auditable, self-completing objects instead of a mutated field on a live spec — the same model VolumeMigration already uses successfully.
• GitOps-friendly. A StorageNodeSet / StorageNode in git no longer flip-flops because someone triggered a restart via spec.action.
• History & retries. Each StorageNodeOps is a discrete record with its own status, generation, and lifecycle. Today status.actionStatus only keeps the latest action.
• Per-node divergence. A StorageNode can hold node-specific overrides (device names, PCIe lists, NUMA sockets) that don't fit a single fleet-wide spec.
• Simpler reconcilers. The set controller stops branching on spec.action. A dedicated ops controller owns action state machines.

Proposed design

StorageNode (new)

The declarative unit of a single storage node. Owns configuration and observed status. Created/owned by a StorageNodeSet, but may also be authored directly for one-off nodes.

The following items should be available in StorageNodeSetSpec and StorageNode:

• ClusterImage
• SpdkImage
• SpdkProxyImage
• MaxLogicalVolumeCount
• MaxSize
• MgmtIfname
• Partitions
• JournalManagerSpec
• CorePercentage
• PcieAllowList
• PcieDenyList
• PcieModel
• DriveSizeRange
• SocketsToUse
• NodesPerSocket
• DataIfname
• DeviceNames
• UbuntuHost
• SkipKubeletConfiguration
• ForceFormat4K
• EnableCpuTopology
• ReservedSystemCPU
• SpdkSystemMemory
• ContainerResources
• InitContainerResources
• ImagePullPolicy
• Tolerations

Values in StorageNode override values from StorageNodeSet. StorageNode has full precedence to ensure clean overridability for all properties.

Furthermore, the StorageNode holds a reference to its ClusterName and its target worker node (the former per-node WorkerNode).

Moves from StorageNodeSetStatus (per-node observed state) to StorageNode:

• NodeStatus → the single node's status (UUID, health, status, CPU, memory, volumes, ports, devices, uptime, hostname, mgmtIp, postedAt).
• NodeLatencyMetrics → this node's fio baseline latency data. (volume rebalancing branch)

StorageNodeOps (new)

A one-shot imperative operation against one StorageNode, mirroring VolumeMigration.

StorageNodeOpsSpec:

• nodeRef: target StorageNode (replaces nodeUUID + workerNode lookups).
• action (from spec.action): enum shutdown | restart | suspend | resume | remove | migrate (let's be explicit about the node migration and not call it restart).
• reattachVolume (from spec.reattachVolume): for restart flows.
• force (from spec.force): forced execution.

StorageNodeOpsStatus (from today's ActionStatus):

• state (pending | running | success | failed), message, updatedAt, observedGeneration, triggered, and ideally a phase lifecycle like VolumeMigration (Pending → Validating → Running → Completed/Failed).

The object is immutable once created and is not re-run on spec edits.

StorageNodeSet (retained, narrowed)

Stays as the grouping + rollout resource. Responsibilities:

• Provision N StorageNodes across workerNodes (fleet add).
• Act as a template for the shared per-node config (fields above are defaulted onto the StorageNodes it creates; individual StorageNodes may override).
• Own rolling restart/update across the set (this is explicitly kept per the request), including drain coordination.

Keeps from StorageNodeSetSpec:

• ClusterName
• WorkerNodes
• MaxParallelNodeAdds
• OpenShiftCluster
• OpenShiftMachineConfigPool
• and the shared config template block used to stamp out StorageNodes.

Keeps from StorageNodeSetStatus (fleet-level orchestration state):

• DrainCoordination ([]NodeDrainState): rolling-upgrade drain state per worker.
• PendingNodeAdds: node-add POST timestamps.
• SchedulingFailedWorkers: workers with FailedScheduling events.
• An aggregate/rollup of child StorageNode states (replacing the inlined status.nodes).

Removed from StorageNodeSet:

• spec.action
• spec.nodeUUID
• spec.workerNode
• spec.reattachVolume
• spec.force
• status.actionStatus