Skip to content

feat(jobs): facet Consumer spans by environment.id in Datadog - #795

Open
shyim wants to merge 1 commit into
mainfrom
cursor/consumer-span-environment-id-b191
Open

feat(jobs): facet Consumer spans by environment.id in Datadog#795
shyim wants to merge 1 commit into
mainfrom
cursor/consumer-span-environment-id-b191

Conversation

@shyim

@shyim shyim commented Aug 12, 2026

Copy link
Copy Markdown
Member

Summary

Copy environment.id (and a tiny outcome enum) onto go-queue Consumer entry spans for environment-scoped jobs so Datadog can filter/facet EnvironmentScrape process / SitespeedScrape process without opening child Internal spans.

Before / after

Before After
Consumer span attrs messaging.message.* only + environment.id, outcome (ok/error)
Facet by environment Only on child environment.scrape On entry span (EnvironmentScrape process, etc.)
High-cardinality fields Still omitted (no URL, emails, extension name lists)

Approach

go-queue’s otel middleware has no message→attribute mapping. Handlers already receive the Consumer span in ctx (trace.SpanFromContext). We annotate before scrape services start their Internal child spans:

  • EnvironmentScrapeenvironment.id + outcome
  • SitespeedScrape → same
  • Other jobs either have no environment id (*Cleanup, ShopwareChangelogSync) or high-cardinality payloads (StoreExtensionSync.Names) — left untouched

Helper: runEnvironmentJob in api/internal/jobs/consumer_span.go.

Test plan

  • Unit tests for runEnvironmentJob (ok/error outcome; attributes stay on parent Consumer span)
  • mise run lint (local) + CI Lint
  • CI Tests (integration suite)
Open in Web Open in Cursor 

Annotate go-queue otel Consumer spans for EnvironmentScrape and
SitespeedScrape with low-cardinality environment.id (and outcome) so
Datadog can facet process spans without opening child Internal spans.

Co-authored-by: Soner <github@shyim.de>
@shyim
shyim marked this pull request as ready for review August 12, 2026 08:13
@greptile-apps

greptile-apps Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds environment and outcome facets to Consumer spans for environment-scoped jobs. The implementation preserves handler errors and avoids placing high-cardinality payload fields on entry spans, but returned failures should also be recorded on the active span.

  • Adds a shared Consumer-span annotation helper.
  • Wraps EnvironmentScrape and SitespeedScrape handlers.
  • Adds unit coverage for successful, failed, and parent-versus-child span annotation.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking observability issue around recording returned job errors on the active span.

The handler behavior and returned errors are preserved, but the new helper only assigns an outcome attribute rather than recording failures with the repository-required OTel error metadata.

Files Needing Attention: api/internal/jobs/consumer_span.go

Important Files Changed

Filename Overview
api/internal/jobs/consumer_span.go Adds shared Consumer-span annotations, but returned job errors are not explicitly recorded on the span.
api/internal/jobs/handlers.go Routes environment and sitespeed scrape handlers through the annotation helper while preserving their return values.
api/internal/jobs/consumer_span_test.go Verifies success/error attributes and confirms annotations remain on the parent Consumer span.

Sequence Diagram

sequenceDiagram
    participant Q as go-queue Consumer span
    participant H as runEnvironmentJob
    participant S as Scrape service
    Q->>H: Invoke handler(ctx, environmentID)
    H->>Q: Set environment.id
    H->>S: Scrape(ctx, environmentID)
    S-->>H: error or nil
    H->>Q: Set outcome
    H-->>Q: Return error or nil
Loading

Fix All in Codex

Reviews (1): Last reviewed commit: "feat(jobs): set environment.id on Consum..." | Re-trigger Greptile

Comment on lines +38 to +40
err := run(ctx)
annotateConsumerOutcome(ctx, err)
return err

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Job errors lack span records

When an environment or sitespeed scraper returns an error, runEnvironmentJob sets only outcome="error" before returning it. The background-job observability convention also requires recording the error and setting the span status, otherwise the Consumer span lacks the expected diagnostic error event and status.

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants