design: fork genesis bootstrap with per-node Jobs

[bdchatham]

Summary

• Formalizes how forked networks bootstrap into their own genesis state using per-node bootstrap Jobs
• Solves PVC multiplicity (each SeiNode needs its own PVC populated independently)
• Adds resource isolation for multi-GB fork genesis initialization (separate Job resource limits, no liveness probes)
• Enables halt-height control (exportHeight + 1) for coordinated fork launches
• Reuses existing bootstrap Job infrastructure (buildBootstrapPlan, GenerateBootstrapJob, teardown/handoff)

Key Design Decisions

1. Fork genesis ceremony nodes route through bootstrap Job when ForkExportHeight > 0 on GenesisCeremonyNodeConfig
2. Single new CRD field: forkExportHeight on GenesisCeremonyNodeConfig (set by SeiNodeDeployment controller)
3. Group-level plan unchanged -- exporter lifecycle, assembly, peer collection all stay the same
4. Per-node plan gains bootstrap phases: deploy Job/Service → genesis ceremony tasks on Job sidecar → seid processes fork genesis with halt-height → teardown → post-bootstrap config on StatefulSet

Open Items from Tide Team Review

The kubernetes-specialist and platform-engineer identified findings that need discussion:

Critical (must resolve)

• NeedsBootstrap predicate returns false for fork genesis nodes → StatefulSet races bootstrap Job for RWO PVC
• deployBootstrapJobExecution.Execute errors on nil SnapshotSource → need dispatch logic for fork genesis
• Missing MaxRetries on ConfigureGenesis in buildForkGenesisPlan → first S3 poll failure kills ceremony

Warnings (should address)

• Bootstrap resources too low for multi-GB genesis (need fork-specific resource tier or override)
• Assembler artifact completeness: does sidecar validate all node gentxs present before assembling?
• Post-bootstrap ConfigureGenesis redundantly re-downloads multi-GB genesis from S3
• Bootstrap seid image: should fork genesis use source chain binary or new chain binary?
• PVC recovery on failed bootstrap: partial state needs cleanup before retry
• forkGenesisParamsForTaskType should reuse genesisParamsForTaskType directly

Test plan

• Review design for correctness and completeness
• Resolve critical findings before implementation
• Discuss warning-level trade-offs

🤖 Generated with Claude Code