feat(shared-storage): resolve NFS Service ClusterIP at install time

[oren-openteams]

Closes #106.

Problem

The shared-storage PV's spec.nfs.server is hardcoded to the in-cluster NFS Service's FQDN:

nfs:
  server: {{ include "nebari-data-science-pack.fullname" . }}-nfs.{{ .Release.Namespace }}.svc.cluster.local

The kubelet performs NFS mounts in the host's network namespace, where /etc/resolv.conf typically points at the upstream DNS provider (the VPC resolver on managed Kubernetes), not the cluster's CoreDNS. As a result, mounts that reference the Service FQDN fail with:

mount.nfs: Failed to resolve server <name>-nfs.<ns>.svc.cluster.local: Name or service not known

on most cloud-managed clusters (EKS, GKE without node-local DNS, AKS without DNS forwarders). Singleuser pods stay stuck at the volume-attach step and JupyterHub spawns never complete.

Why legacy nebari didn't hit this

Legacy nebari deliberately computed the NFS Service's ClusterIP at terraform-apply time and baked the IP into the PV — see nfs-server/output.tf:

output "endpoint_ip" {
  value = kubernetes_service.main.spec.0.cluster_ip
}

and the wiring at jupyterhub.tf:

nfs_endpoint = var.jupyterhub-shared-endpoint == null
  ? module.kubernetes-nfs-server.0.endpoint_ip
  : var.jupyterhub-shared-endpoint

That design got lost in the helm-chart refactor. This PR ports it forward.

Change

templates/shared-pvc.yaml — the PV's nfs.server is now resolved at install time via helm lookup:

{{- $svcName := printf "%s-nfs" (include "nebari-data-science-pack.fullname" .) }}
{{- $svcFQDN := printf "%s.%s.svc.cluster.local" $svcName .Release.Namespace }}
{{- $svcIP := "" }}
{{- $svcLookup := (lookup "v1" "Service" .Release.Namespace $svcName) }}
{{- if $svcLookup }}
{{-   $svcIP = (default "" $svcLookup.spec.clusterIP) }}
{{- end }}
nfs:
  server: {{ default $svcFQDN $svcIP | quote }}
  path: /exports

kube-proxy programs host-side iptables for Service IPs, so the mount works without any host-DNS dependency once the IP is in the PV.

Backward compatibility

When lookup returns nil — which happens on the very first helm install (Service doesn't exist yet) and during helm template-only renders (no live API) — the PV falls back to the FQDN. That keeps the chart working on clusters where host DNS DOES forward cluster.local to CoreDNS (k3s, microk8s with custom resolvers, which is likely why this regression wasn't caught in upstream testing).

Caveat: PV is immutable

PersistentVolume.spec.persistentvolumesource is immutable, so a fresh install that lands the FQDN cannot be patched into the IP form via helm upgrade — the API server rejects the change. Recovery path on affected clusters: delete the PV (Retain reclaim policy preserves the backing storage on the NFS server's RWO PVC) and re-sync. Subsequent installs in the same namespace will see the existing Service and bake the IP directly.

A follow-up could add an optional sharedStorage.nfsServer.endpoint chart value as an explicit operator override (mirroring legacy's jupyterhub-shared-endpoint) to make first-install on managed Kubernetes clusters work without the delete-and-resync dance. Out of scope here; happy to add it if reviewers prefer the combined approach.

Tests

tests/unit/test_shared_pvc_nfs_server.py adds two regression tests:

1. FQDN-fallback path — helm template has no live API, so lookup always returns nil. The PV's nfs.server must fall back to the cluster-local FQDN. Pins the contract that backward-compatible rendering still works.
2. YAML-renders-without-errors smoke test — the -}} / }} whitespace-trimming choices around the lookup block are easy to break. Without this guard, a stray -}} collapses persistentVolumeReclaimPolicy: Retain and nfs: onto one line and produces an inscrutable mapping values not allowed in this context error.

The "live Service returns the ClusterIP" path is exercised end-to-end by any helm install against a real cluster, so it's not covered by helm template-based unit tests. Could add an integration test against a kind cluster if that's the project's preference.

How I tested

helm template test-release . --namespace test-ns \
    --set keycloak.hostname=keycloak.example.com \
    --set sharedStorage.enabled=true \
    --set sharedStorage.nfsServer.enabled=true \
    | grep -A 25 "kind: PersistentVolume$"

→ Renders cleanly with server: "test-release-nebari-data-science-pack-nfs.test-ns.svc.cluster.local" (FQDN fallback path, since no Service exists in helm template context).

pytest tests/unit/test_shared_pvc_nfs_server.py -v
# → 2 passed