Skip to content

[Bug] ClusterRole missing events patch/create permissions for cross-namespace TemporalWorkerDeployment resources (v1.4.0) #277

@gabriel-yahav

Description

@gabriel-yahav

Description

Starting in v1.4.0, the controller attempts to patch Kubernetes events for
TemporalWorkerDeployment objects. However, the ClusterRole provisioned by
the Helm chart does not include create/patch verbs on the events resource,
causing the controller to continuously log permission errors and fail to record
events.

Setup

  • Controller chart version: 1.4.0
  • Controller namespace: X
  • TemporalWorkerDeployment namespace: Y

The controller is deployed in a different namespace than the TemporalWorkerDeployment
CRs it manages. This is a common and supported pattern (deploying the controller
once cluster-wide and managing CRs in application namespaces).

Error

Server rejected event (will not retry!)" err="events \"dbtworker.18a6745739983ba1\" is forbidden: User \"system:serviceaccount:X:temporal-worker-controller-service-account\" cannot patch resource \"events\" in │
│  API group \"\" in the namespace \"Y\""

Root Cause

The ClusterRole created by the Helm chart is missing events permissions.
The controller-runtime library used by the controller (sigs.k8s.io/controller-runtime@v0.21.0)
records events against the namespace where the watched resource lives (in this case X),
not where the controller itself is deployed.

Since a ClusterRole + ClusterRoleBinding is used (as expected for a cluster-wide
controller), the fix is straightforward — add events to the ClusterRole rules.

Fix

Add to the controller's ClusterRole:

- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch

Workaround

Manually patching the ClusterRole to add the above rule resolves the issue
until a chart fix is released.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions