What is the problem?
Effect Cluster entity server spans currently include the concrete entityId in the span name. In packages/cluster/src/internal/entityManager.ts, the entity RPC server is created with a span prefix shaped like:
spanPrefix: `${entity.type}(${address.entityId})`,That produces server-side operation names like:
Chefy/WorkflowSerialGate/SyncOrganizationMetrics(SyncOrganizationMetrics:v7fh2im4s28uz5bdewi7c9he).Run
For entity IDs that contain domain keys, workflow execution IDs, tenant IDs, queue keys, etc., this turns span.name / operation into high-cardinality data. Backends such as Axiom, Jaeger, and span-metrics pipelines group by span name, so each entity ID becomes a distinct operation.
Expected behavior
The span name should stay low cardinality, for example:
Chefy/WorkflowSerialGate/SyncOrganizationMetrics.Run
The concrete entity ID should remain available as an attribute. The current implementation already sets:
"entity.id": address.entityId
"entity.type": entity.type
That seems like the right place for the dynamic value.
Actual behavior
The dynamic entityId is part of the exported span name because it is embedded in the RPC server spanPrefix:
${entity.type}(${address.entityId}).${rpcName}
Why this matters
OpenTelemetry span names are expected to be low cardinality. This issue can create large numbers of unique operations for otherwise identical entity RPCs, especially for persisted/durable entities where the entity ID is often a workflow execution key or serial-lane key.
We are working around this in our collector by normalizing names matching:
^Chefy/Workflow(Type|Serial)Gate/[^()]+\(.+\)\.Run$
but this is a downstream normalization workaround. It would be better if Effect Cluster either:
- omitted
entityId from the default server span prefix, or
- provided an annotation/config hook to customize the entity server span name / span prefix, similar in spirit to HTTP client span-name generators.
Suggested direction
A low-cardinality default could be:
or possibly:
spanPrefix: `${entity.type}.server`with entity.id retained as a span attribute for drilling into a specific entity instance.
If changing the default is too breaking for existing users, an entity/protocol annotation for server span prefix generation would let applications opt into stable operation names without collector-side rewrites.
What is the problem?
Effect Cluster entity server spans currently include the concrete
entityIdin the span name. Inpackages/cluster/src/internal/entityManager.ts, the entity RPC server is created with a span prefix shaped like:That produces server-side operation names like:
For entity IDs that contain domain keys, workflow execution IDs, tenant IDs, queue keys, etc., this turns
span.name/ operation into high-cardinality data. Backends such as Axiom, Jaeger, and span-metrics pipelines group by span name, so each entity ID becomes a distinct operation.Expected behavior
The span name should stay low cardinality, for example:
The concrete entity ID should remain available as an attribute. The current implementation already sets:
That seems like the right place for the dynamic value.
Actual behavior
The dynamic
entityIdis part of the exported span name because it is embedded in the RPC serverspanPrefix:Why this matters
OpenTelemetry span names are expected to be low cardinality. This issue can create large numbers of unique operations for otherwise identical entity RPCs, especially for persisted/durable entities where the entity ID is often a workflow execution key or serial-lane key.
We are working around this in our collector by normalizing names matching:
but this is a downstream normalization workaround. It would be better if Effect Cluster either:
entityIdfrom the default server span prefix, orSuggested direction
A low-cardinality default could be:
or possibly:
spanPrefix: `${entity.type}.server`with
entity.idretained as a span attribute for drilling into a specific entity instance.If changing the default is too breaking for existing users, an entity/protocol annotation for server span prefix generation would let applications opt into stable operation names without collector-side rewrites.