[FEATURE]: Support origin discovery via entity-id or container-id

[kristjanvalur]

Describe the goal of the feature

Looking at dd-trace-python, I see that library performs container discovery by examining the /proc/ filesystem, and populates the Datadog-Entity-Id and Datadog-Container-Id headers accordingly.

Perhaps discovering these is out of scope of a C implementation, but would not at least looking for environment variables, DD_CONTAINER_ID and/or DD_ENTITY_ID be helpful?

(Getting container id is not guaranteed to work in cgroups2, depending on namespace, but the cgroup inode number apparently does...)

I also see the python lib populate the Datadog-External-Env with the DD_EXTERNAL_ENV environment variable, although I don't know what that is supposed to do...

Is your feature request related to a problem?

No response

Describe alternatives you've considered

No response

Additional context

No response

[dmehala]

Hey @kristjanvalur
Thank you for raising this issue and for contributing.

To better understand your needs and provide an appropriate solution, could you please clarify why support for the entity-id and container-id is required in your use case? This will help us assess how we can best address your request.

[kristjanvalur]

Hi there. I'm publishing a PR #196 where I explain this better.

Basically, ENTITY_ID (and container id) are used by the datadog agent to apply tags to data coming from a running container. This simplifies tagging by not having to push individual configuration variables into the container. The Datadog Adminssion Controller already sets DD_ENTITY_ID and other variables for this purpose. CONTAINER_ID is useful in cases when an app is running in a container not controlled by the Admission Controller.

Other implementations, such as dd-agent-py already support these semantics, it seems that they have not yet been added to the C implementation. Even though e.g. dd-agent-py also supportgs its own origin discovery, it will defer to DD_ENTITY_ID or DD_CONTAINER_ID if provided.

Concrete use case: We are using dd-agent-cpp as part of an UnrealEngine server running in a kubernetes cluster and having the trace agent support the same environment variables as e.g. DogStatsD simplifies deployment.

[dmehala]

I now have a clearer understanding of your requirements. If I recall correctly, using Unix Domain Socket (UDS) instead of HTTP automatically resolves the container ID and entity ID.

I will look for the relevant documentation and test UDS to determine whether it can address your issue. I'll keep you updated on my findings.

EDIT: About missing container tags:

If your Kubernetes cluster is using cgroup v2 we recommend to upgrade to at least 7.40.0 of the Datadog Agent, take advantage of Unix-Domain-Socket (UDS) to send traces, and ensure you have the hostPID: true setting in the Agent pod.

This method works in both cgroup v1 and v2, so it can be done in advance.

The trace-agent can receive traces over both the port and socket at the same time - in case you need to migrate your application pods one at a time.

Can you give a shot to UDS? Link to our public documentation

[kristjanvalur]

I can see if it possible with my kubernetes people. But is there a particular reason why the -cpp version of dd-trace should not have the same feature set as the other language libraries? Are you moving away from "DD_EXTERNAL_ENV" and "DD_ENTITY_ID"? Surely there is value in having feature parity across the implementations? Anyway, my PR has been submitted so it works either way, we already work with a fork of this repo.

[dmehala]

I completely agree that dd-trace-cpp—along with all tracers—should have feature parity. Unfortunately, that’s not currently the case, and the C++ tracer has been particularly affected by this gap. This is something my team and I are actively working to improve.

Your request to support the new origin discovery feature is absolutely valid. While your contribution does work, I believe it may not fully comply with the specifications, if I recall correctly. Good news is I start from your PR and do my best to support it as soon as possible.

In the meantime, my responsibility is to recommend the official solution currently available, which is to use UDS until this feature is fully implemented. I also encourage opening a feature request to track this properly.

TL;DR: I'm going to implement the feature.

[kristjanvalur]

Cheers! I'll see if I can persuade my people to use Unix sockets.

[bartugozet]

Hey @dmehala we are using istio proxy to send the traces to datadog and we are trying to implement a solution where we can use UDS. How can we configure the tracer to achieve that?