Skip to content

Add support for Failure Domain Definitions #307

Description

@noctarius

Failure domains let the control plane spread erasure-coding chunks across independent fault groups. They are enabled in two steps and both must be threaded through to the ../sbcli control plane:

  1. Cluster creation: opt in at the StorageCluster level.
  2. Storage node creation: each node must declare which failure-domain group it belongs to.

Step 1: StorageCluster opt-in

Add a cluster-level toggle to StorageClusterSpec, alongside the other fault-tolerance/topology flags (MaxFaultTolerance, StrictNodeAntiAffinity):

  • EnableFailureDomains *bool → json:"enableFailureDomains,omitempty"
  • Immutable: mark with a kubebuilder CEL rule like the existing immutable spec fields. Dailure-domain mode can't be flipped on a live cluster.

Thread it into cluster create:

  • Add enable_failure_domain to utils.ClusterAddParams
  • Call create_cluster(enable_failure_domain=...)

⚠️ Naming to confirm: the request text says --enable-failure-domains (plural), but the sbcli function param is enable_failure_domain (singular). The Go struct field can be plural, but the JSON tag sent to the control plane must match whatever the /api/v2/clusters/ endpoint actually accepts. Verify the wire key before wiring it up.

Step 2: per-node failure-domain group

When the cluster has failure domains enabled, every storage node must be assigned to a failure-domain group (an integer). This is a per-node property, so it belongs on the new StorageNode CRD:

  • FailureDomain *int32 → json:"failureDomain,omitempty" on StorageNodeSpec.

    • StorageNodeSet (as the template) may set a default/strategy, but the concrete value lives per StorageNode — different nodes belong to different groups, which is exactly why this per-node field needs the split.

    Thread it into node add:

    • postStorageNodeSet() (→ the new StorageNode provisioning controller) in simplyblockstoragenodeset_controller.go (params built ~L317–336, POSTed to /api/v2/clusters/{uuid}/storage-nodes).
    • Add to utils.StorageNodeSetAddParams (internal/utils/types.go) — maps to sbcli add_node(failure_domain=...) (simplyblock_core/storage_node_ops.py), CLI --failure-domain N.

Validation

Mirror the control-plane rules (storage_node_ops.py::add_node) in the operator (webhook/validation) so failures surface as CRD validation, not a backend 4xx.

  • If StorageCluster.enableFailureDomains == true → the node's failureDomain is required and must be a valid group integer.
  • If disabled → failureDomain must be unset (control plane defaults to -1).
  • If disabled → a failure domain setting must cause a CRD validation error on StorageNodeSet or StorageNode
  • Validate the failure domain id is >0 on CRD validation

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions