Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: CI

on:
pull_request:
branches: [main]
push:
branches: [main]

jobs:
# ── Job 1: Helm lint & render ──────────────────────────────────────────────
helm-lint:
name: Helm lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install Helm
uses: azure/setup-helm@v4
with:
version: "latest"

- name: Lint openbao-init (local values)
run: helm lint apps/openbao-init/ -f apps/openbao-init/values-local.yaml

- name: Lint openbao-init (scaleway values)
run: helm lint apps/openbao-init/ -f apps/openbao-init/values-scaleway.yaml

- name: Render bootstrap chart (local)
run: helm template boot bootstrap/ --set env=local --set revision=main

- name: Render clusters/local chart
run: helm template loc clusters/local/ --set revision=main

- name: Render clusters/scaleway chart
run: helm template loc clusters/scaleway/ --set revision=main

# ── Job 2: Infisical absent ────────────────────────────────────────────────
infisical-absent:
name: Infisical absent
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Assert no Infisical references in active GitOps manifests
run: |
if git grep -ri infisical apps/ platform/ clusters/ bootstrap/ 2>/dev/null; then
echo "ERROR: Infisical references found in GitOps manifests"
exit 1
fi
echo "OK: No Infisical references found"
21 changes: 21 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# OS / editor
.DS_Store
Thumbs.db
*.swp
*.swo
.vscode/
.idea/

# Secrets (never commit)
*.secret.yaml
*.secret.yml
kubeconfig*
*.key
*.pem
secrets/

# Helm
*.tgz

# Temp
tmp/
3 changes: 3 additions & 0 deletions .specify/feature.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"feature_directory": "specs/001-replace-infisical-oss-secrets"
}
44 changes: 43 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ minikube status # cluster run
kubectl --context minikube get pods -n argocd # ArgoCD healthy?
kubectl --context minikube get applications -n argocd # apps synced?
kubectl --context minikube get pods -n ingress-nginx # ingress up?
kubectl --context minikube get clustersecretstore openbao # OpenBao + ESO ready?
```

## Validation commands
Expand All @@ -40,16 +41,21 @@ Always target the local cluster explicitly to avoid accidental execution against
```bash
# Lint a Helm chart
helm lint apps/demo/ -f apps/demo/values-local.yaml
helm lint apps/openbao-init/ -f apps/openbao-init/values-local.yaml
helm lint apps/openbao-init/ -f apps/openbao-init/values-scaleway.yaml

# Render an app chart to inspect output
helm template demo apps/demo/ -f apps/demo/values-local.yaml
helm template openbao-init apps/openbao-init/ -f apps/openbao-init/values-local.yaml

# Render the app-of-apps charts (confirm revision propagation)
helm template boot bootstrap/ --set env=local --set revision=<branch>
helm template loc clusters/local/ --set revision=<branch>
helm template loc clusters/local/ --set revision=<branch> --set repoURL=<repo>
helm template loc clusters/scaleway/ --set revision=<branch> --set repoURL=<repo>

# Validate a platform Application manifest
kubectl --context minikube apply --dry-run=client -f platform/local/kube-prometheus-stack.yml
kubectl --context minikube apply --dry-run=client -f platform/local/openbao.yml
```

## Adding things
Expand All @@ -63,3 +69,39 @@ kubectl --context minikube apply --dry-run=client -f platform/local/kube-prometh
4. `bootstrap/templates/staging.yaml` guarded by `{{- if eq .Values.env "staging" }}` as the entry point for external provisioners

**New platform tool:** Add a file to `platform/<env>/` following `platform/local/ingress-nginx.yml`.

<!-- SPECKIT START -->
## Active Feature

**Feature**: Replace Infisical with OSS Secrets Backend
**Plan**: [specs/001-replace-infisical-oss-secrets/plan.md](specs/001-replace-infisical-oss-secrets/plan.md)
**Tasks**: [specs/001-replace-infisical-oss-secrets/tasks.md](specs/001-replace-infisical-oss-secrets/tasks.md)
**Status**: Tasks generated — ready for `/speckit-implement`
**Tool**: OpenBao (MPL 2.0) — decided 2026-06-21

### Quick reference

| Artifact | Path |
|----------|------|
| Spec | `specs/001-replace-infisical-oss-secrets/spec.md` |
| Plan | `specs/001-replace-infisical-oss-secrets/plan.md` |
| Research | `specs/001-replace-infisical-oss-secrets/research.md` |
| Data model | `specs/001-replace-infisical-oss-secrets/data-model.md` |
| Contracts | `specs/001-replace-infisical-oss-secrets/contracts/` |
| Quickstart | `specs/001-replace-infisical-oss-secrets/quickstart.md` |
| Tasks | `specs/001-replace-infisical-oss-secrets/tasks.md` |

### MVP scope (US1, Phases 1–3, tasks T001–T020)

1. Scaffold `apps/openbao-init/` Helm chart (T001–T004)
2. Write `platform/<env>/openbao.yml` + `external-secrets.yml` (T005–T009)
3. Write init Job, RBAC, policies, ClusterSecretStore, cluster templates, sync waves (T010–T018)
4. Validate: `helm lint apps/openbao-init/` + local Scenario 1 smoke test (T019–T020)

### Key constraints

- `scaleway-s3-credentials` Secret created by infra Terraform, not this repo — init Job reads it
- S3 backend: no HA on Scaleway (no DynamoDB locking) → single replica
- Init Job must be idempotent; root token revoked after first init
- All external Helm chart versions must be pinned explicitly
<!-- SPECKIT END -->
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,33 @@ The `platform` App-of-Apps picks it up automatically.

---

## Local development workflow

**Smoke test** (destroy + rebuild everything from scratch):
```bash
# from the infrastructure repo
mise run reset && mise run dev
```

ArgoCD takes a few minutes to reconcile all apps — be patient. The difference between "still syncing" and "actually stuck" is visible in:
```bash
kubectl --context minikube get applications -n argocd
```

**Important:** before running, check `infrastructure/02-cluster/local/nico.auto.tfvars` — it contains the `gitops_revision` variable that controls which branch of this repo ArgoCD will deploy from. If you're testing a feature branch, set it there:
```hcl
gitops_revision = "feat/your-branch"
```
If it points to `main` and your changes aren't merged yet, ArgoCD will deploy the old code.

**Validate OpenBao is up:**
```bash
kubectl --context minikube get clustersecretstore openbao
# READY=True → OpenBao running, unsealed, ESO connected
```

---

## Accessing the demo app (local)

After `mise run reset && mise run dev` in the infrastructure repo:
Expand Down
71 changes: 71 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# TODO

## OpenBao — ce qui reste à faire

### 1. Sortir du mode `dev`

`platform/local/openbao.yml` a encore `dev: enabled: true`.
Ce mode est in-memory, sans Raft, sans seal — inutilisable en prod.

- [ ] Passer en mode standalone avec Raft réel (supprimer `dev: enabled: true`, garder la config `standalone.config` déjà présente)

---

### 2. Auto-unseal — **Option C choisie : AWS KMS natif**

Scaleway KMS n'est pas supporté nativement par OpenBao (pas d'API AWS KMS-compatible, pas de PKCS#11). On part sur AWS KMS natif (hors Scaleway).

- [x] **Terraform KMS** — `infrastructure/03-backup/scaleway/kms.tf` : clé KMS symétrique (rotation auto, `prevent_destroy`), user IAM dédié single-purpose (Encrypt/Decrypt/DescribeKey via key policy, aucune policy IAM), access key statique. Outputs : `openbao_unseal_kms_key_id`, `openbao_unseal_aws_region`, `openbao_unseal_access_key_id`, `openbao_unseal_secret_access_key`.
- ⚠️ **Apply = admin/local uniquement** (`aws sso login`). Le rôle CI backup est S3-only et la permissions-boundary interdit les users IAM + access keys → un push touchant `kms.tf` fait échouer le CI `backup_scaleway`.
- [ ] **Côté gitops** — ajouter le stanza `seal "awskms"` dans `standalone.config` de `platform/<env>/openbao.yml` (`region`, `kms_key_id`) + injecter `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` dans le pod OpenBao via un Secret (`extraSecretEnvironmentVars` ou env). Chicken-and-egg : ce Secret ne peut PAS venir d'OpenBao lui-même → source externe (bootstrap cluster terraform, ou Infisical→ESO pendant la transition).
- [ ] **Init** — avec auto-unseal, `bao operator init` génère toujours des recovery keys (Shamir de secours) : les stocker hors-cluster.

---

### 3. Snapshot agent — prérequis OpenBao

La config S3 est posée (`platform/local/openbao.yml`), mais le CronJob ne peut pas s'authentifier à OpenBao sans :

- [ ] Activer le Kubernetes auth method dans OpenBao :
```bash
bao auth enable kubernetes
bao write auth/kubernetes/config kubernetes_host=https://kubernetes.default.svc
```
- [ ] Créer la policy `snapshot` (droit sur `sys/storage/raft/snapshot`)
- [ ] Créer le role `snapshot` lié au ServiceAccount du CronJob :
```bash
bao write auth/kubernetes/role/snapshot \
bound_service_account_names=openbao-snapshot-agent \
bound_service_account_namespaces=openbao \
policies=snapshot \
ttl=1h
```

---

### 4. Restauration au démarrage (non natif) — **implémenté, en attente de validation E2E**

Le snapshot agent fait des backups, mais ne restaure pas au boot.
Pour un cluster éphémère, il faut un job d'init qui :

- [x] Vérifie si OpenBao est fraîchement initialisé (`bao operator init -status`)
- [x] Télécharge le dernier snapshot depuis `s3://backup-dev-id`
- [x] Exécute `bao operator raft snapshot restore <snapshot>`

Réalisé via `apps/openbao-init/` : Job-hook ArgoCD (`hook: Sync`,
`hook-delete-policy: BeforeHookCreation`) qui miroir le script de backup upstream
dans la même image `openbao-snapshot-agent` (`bao` + `s3cmd`). Idempotent : ne touche
jamais un cluster déjà initialisé. Sur cluster vide → `bao operator init` (token root
jetable, juste le temps du restore) puis `snapshot restore`. **Local uniquement**
(scaleway hors scope, pas encore de snapshots).

- [ ] Smoke test : `mise run reset && mise run dev`, vérifier qu'un secret témoin
survit au reset. Vérifier aussi que le nœud se re-unseal seul après restore
(même clé KMS) — sinon redémarrer `openbao-0`.

---

## Ce qui est fait

- [x] `snapshotAgent` configuré dans `platform/local/openbao.yml` — pousse vers `s3://backup-dev-id` (Scaleway fr-par) toutes les 5 min
- [x] Credentials S3 injectés via Secret `scaleway-s3-credentials` (créé par Terraform, pas dans git)
6 changes: 6 additions & 0 deletions apps/openbao-init/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: v2
name: openbao-init
description: Restores the latest OpenBao snapshot from S3 on a fresh (empty) cluster, via an ArgoCD sync hook
type: application
version: 0.1.0
appVersion: "0.3.0"
63 changes: 63 additions & 0 deletions apps/openbao-init/templates/restore-configmap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
{{- if .Values.restore.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
name: openbao-init-restore
namespace: {{ .Release.Namespace }}
annotations:
# Normal resource (see serviceaccount.yaml). Wave 0 → exists before the Job.
argocd.argoproj.io/sync-wave: "0"
data:
restore.sh: |
#!/usr/bin/env sh
# Restore the latest OpenBao snapshot from S3 on a fresh (empty) cluster.
# Mirrors the upstream backup script (bao-snapshot.sh), same image.
# Idempotent: never touches an already-initialized cluster.
set -eu

# 1. Wait for the OpenBao API. `openbao` syncs in parallel (separate
# Application); its pod comes up 0/1 (uninitialized) but the `openbao`
# Service publishes not-ready addresses, so the API is reachable. `bao
# status` returns rc 1 on a connection error, rc 2 when reachable-but-
# sealed/uninitialized. Budget ~10 min for a cold start.
i=0
while true; do
# Capture the real exit code: `|| rc=$?` both shields `set -e` and avoids
# the `if`-without-`else` trap that would reset $? to 0. rc 0 = unsealed,
# rc 2 = reachable but sealed/uninitialized — both mean the API is up.
rc=0
bao status >/dev/null 2>&1 || rc=$?
if [ "$rc" = "0" ] || [ "$rc" = "2" ]; then break; fi
i=$((i + 1))
if [ "$i" -gt 120 ]; then echo "timeout waiting for OpenBao API"; exit 1; fi
echo "waiting for OpenBao API..."
sleep 5
done

# 2. Idempotence guard: exit 0 of `init -status` means already initialized.
# Never clobber a live cluster (protects against selfHeal re-syncs).
if bao operator init -status >/dev/null 2>&1; then
echo "OpenBao already initialized — nothing to restore."
exit 0
fi

# 3. Locate the latest snapshot. Names sort chronologically.
LATEST=$(s3cmd ls "$S3_URI/" --host="$S3_HOST" --host-bucket="$S3_BUCKET" $S3CMD_EXTRA_FLAG \
| sort | tail -1 | awk '{print $4}')
if [ -z "$LATEST" ]; then
echo "No snapshot found in S3 — leaving node uninitialized for manual bootstrap."
exit 0
fi
echo "Latest snapshot: $LATEST"

# 4. Throwaway init: works on empty storage and yields a root token valid long
# enough to perform the restore. The restore replaces all storage (auth,
# policies, secrets) so this token becomes void afterwards — by design.
ROOT=$(bao operator init | grep -i 'Initial Root Token' | awk '{print $NF}')
export BAO_TOKEN="$ROOT"

s3cmd get "$LATEST" /tmp/restore.snapshot \
--host="$S3_HOST" --host-bucket="$S3_BUCKET" $S3CMD_EXTRA_FLAG
bao operator raft snapshot restore /tmp/restore.snapshot
echo "Restored $LATEST (throwaway root token is now void)."
{{- end }}
73 changes: 73 additions & 0 deletions apps/openbao-init/templates/restore-job.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
{{- if .Values.restore.enabled }}
apiVersion: batch/v1
kind: Job
metadata:
name: openbao-init-restore
namespace: {{ .Release.Namespace }}
annotations:
# Normal resource, wave 1 (after SA + ConfigMap). On a fresh cluster the Job
# doesn't exist → OutOfSync → ArgoCD creates and runs it. Re-runs on a live
# cluster are prevented by the idempotence guard in restore.sh.
argocd.argoproj.io/sync-wave: "1"
# A Job's pod template is immutable; Replace lets ArgoCD recreate it cleanly
# when the spec (e.g. restore.sh) changes, instead of failing on the patch.
argocd.argoproj.io/sync-options: Replace=true
spec:
backoffLimit: 2
template:
metadata:
labels:
app.kubernetes.io/name: openbao-init
component: restore
spec:
serviceAccountName: openbao-init
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 100
runAsGroup: 1000
fsGroup: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: restore
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: ["sh", "/scripts/restore.sh"]
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
env:
- name: BAO_ADDR
value: {{ .Values.baoAddr | quote }}
- name: S3_HOST
value: {{ .Values.s3.host | quote }}
- name: S3_BUCKET
value: {{ .Values.s3.bucket | quote }}
- name: S3_URI
value: {{ .Values.s3.uri | quote }}
- name: S3CMD_EXTRA_FLAG
value: {{ .Values.s3.extraFlag | quote }}
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: {{ .Values.s3CredentialsSecret }}
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ .Values.s3CredentialsSecret }}
key: AWS_SECRET_ACCESS_KEY
volumeMounts:
- name: scripts
mountPath: /scripts
- name: tmp
mountPath: /tmp
volumes:
- name: scripts
configMap:
name: openbao-init-restore
- name: tmp
emptyDir: {}
{{- end }}
Loading
Loading