Found while testing #117 live on an AWS Nebari cluster (Helm/ArgoCD install).
On a Helm install, every served LLMModel fails to reconcile:
reconciling EPP Role: roles "<model>-epp" is forbidden: user
"system:serviceaccount:nebari-llm-serving-system:nebari-llm-serving-operator"
is attempting to grant RBAC permissions not currently held:
inference.networking.x-k8s.io: inferenceobjectives, inferencemodelrewrites, inferencepools, inferencepoolimports (get/list/watch)
The per-model EPP Role rendered by operator/internal/controller/reconcilers/inferencepool.go grants get/list/watch on those inference.networking.x-k8s.io resources, but the operator's own ClusterRole (charts/nebari-llm-serving/templates/operator-clusterrole.yaml) does not hold them. Kubernetes privilege-escalation prevention then blocks the operator from creating the Role, and the LLMModel reconcile aborts before it creates the model's SecurityPolicy and AIGatewayRoute too.
make deploy on kind runs the operator with a broader role, which is why the kind validation did not catch it.
Impact: no served model can come up on a Helm/ArgoCD install.
Fix: add the matching inference.networking.x-k8s.io get/list/watch rule to the operator ClusterRole. Done on the #117 branch (commit 0640fb2); verified the operator can create the EPP Role after the change (it failed with the escalation error before).
Found while testing #117 live on an AWS Nebari cluster (Helm/ArgoCD install).
On a Helm install, every served
LLMModelfails to reconcile:The per-model EPP Role rendered by
operator/internal/controller/reconcilers/inferencepool.gogrants get/list/watch on thoseinference.networking.x-k8s.ioresources, but the operator's own ClusterRole (charts/nebari-llm-serving/templates/operator-clusterrole.yaml) does not hold them. Kubernetes privilege-escalation prevention then blocks the operator from creating the Role, and the LLMModel reconcile aborts before it creates the model's SecurityPolicy and AIGatewayRoute too.make deployon kind runs the operator with a broader role, which is why the kind validation did not catch it.Impact: no served model can come up on a Helm/ArgoCD install.
Fix: add the matching
inference.networking.x-k8s.ioget/list/watch rule to the operator ClusterRole. Done on the #117 branch (commit 0640fb2); verified the operator can create the EPP Role after the change (it failed with the escalation error before).