Feature request: explicit attribution labels and confidence for SFT exports

[vincentwuxi]

PR Requirement: Explicit Attribution Labels and Confidence for SFT Exports

Status: requirement draft for maintainer review.
Optional implementation spike: #179

Title

Add explicit attribution labels and confidence metadata for gateway / passive capture, and allow SFT exports to filter ambiguous-source trajectories.

Background

Heron can reconstruct LLM calls and agent turns from several capture paths:

• eBPF SSL uprobes, where Heron can attach native OS process attribution such as pid, comm, and executable path.
• Passive / gateway / post-TLS-termination capture, where Heron can reconstruct plaintext HTTP calls and agent turns, but cannot infer the originating local OS process unless the capture point exposes an identity.

This distinction matters when users export captured production traffic into SFT / fine-tuning trajectories. A reconstructed call may be semantically correct, but if the source identity is ambiguous, users should be able to exclude it from training datasets rather than silently training on mislabeled examples.

The request was motivated by this gateway-capture scenario:

For the gateway case: can I hand Heron an explicit identity key, such as a header or mTLS SAN promoted by the gateway, to use as the attribution label? Does the reconstructed record carry an attribution-confidence flag, so SFT export can filter out ambiguous-source calls instead of silently training on mislabeled turns?

Problem

Today Heron has process attribution for eBPF-capable capture, and raw HTTP headers may be stored with spans. However, Heron does not provide a first-class, persisted attribution label/confidence model that:

1. Lets operators explicitly mark a trusted request header as the attribution identity for gateway/passive capture.
2. Distinguishes high-confidence attribution from ambiguous attribution.
3. Persists attribution metadata through storage and API read paths.
4. Exposes attribution metadata in SFT export metadata.
5. Allows SFT export to exclude turns whose source attribution is below a requested confidence threshold.

Without this, downstream dataset generation can accidentally mix calls from indistinguishable agents, tenants, workspaces, or services.

Goals

• Preserve Heron's existing eBPF process attribution semantics.
• Add a generic way to promote trusted gateway metadata into an attribution label.
• Persist attribution label, source, and confidence on each reconstructed span.
• Surface attribution metadata in span/detail/trace read APIs.
• Add an export-time filter so users can build SFT datasets from only high-confidence attributed turns.
• Keep default behavior backward compatible: if no attribution config is set, capture and export should continue to work.

Non-Goals

• Do not claim Heron can recover OS process identity from passive packet capture when that identity is not visible.
• Do not directly parse mTLS SAN from TLS sessions in this change.
• Do not add automatic PII / secret redaction in this change.
• Do not introduce RBAC or access-control changes in this change.
• Do not claim support for Go crypto/tls or Rust rustls plaintext capture through the existing SSL_read / SSL_write uprobe path.

Proposed Behavior

Attribution Confidence Values

Introduce a small confidence vocabulary:

• high: the attribution label came from a native eBPF process attribution or a configured trusted identity key visible in HTTP metadata.
• medium: reserved for future correlation strategies where identity is probable but not native / explicit.
• ambiguous: Heron reconstructed the call, but no reliable identity label was available.

Initial implementation may only emit high and ambiguous.

Attribution Source Values

Each span should carry a source string explaining where the attribution came from:

• ebpf_process
• request_header:<header-name>
• unknown

Future source values may include Kubernetes metadata, gateway route identity, probe identity, or other explicit capture-point identities.

Attribution Label

Each span should carry an optional label:

• eBPF path: executable path when available, otherwise process comm.
• Gateway/passive path: first configured non-empty request header value.
• No identity: null.

eBPF process attribution should take precedence over configured request headers when both are present, because it is native OS-level attribution.

Configuration

Add a top-level config section:

[attribution]
request_headers = []

Example:

[attribution]
request_headers = ["x-agent-id", "x-workspace-id"]

Behavior:

• Header matching should be case-insensitive.
• Header priority should follow the configured order.
• The first configured header with a non-empty value becomes the attribution label.
• For mTLS gateways, operators should configure the gateway to promote the client identity / SAN into a trusted HTTP header before Heron captures the plaintext HTTP traffic.
• If no configured header is present and no eBPF process attribution exists, attribution should be ambiguous.

Data Model

Add attribution metadata to LlmCall:

pub attribution: AttributionInfo

Suggested type shape:

pub struct AttributionInfo {
    pub label: Option<String>,
    pub source: String,
    pub confidence: AttributionConfidence,
}

pub enum AttributionConfidence {
    High,
    Medium,
    Ambiguous,
}

The default should be:

{
  "label": null,
  "source": "unknown",
  "confidence": "ambiguous"
}

Storage

Persist attribution metadata on spans in both DuckDB and ClickHouse:

attribution_label VARCHAR / Nullable(String)
attribution_source VARCHAR NOT NULL DEFAULT 'unknown' / String DEFAULT 'unknown'
attribution_confidence VARCHAR NOT NULL DEFAULT 'ambiguous' / String DEFAULT 'ambiguous'

Migration requirements:

• Existing databases should be upgraded idempotently on startup.
• Existing rows should default to no label, unknown source, and ambiguous confidence.
• Fresh schemas and migrated schemas should have the same read/write behavior.

API

Expose attribution fields in span read models:

• Span list item.
• Span detail.
• Trace span item / turn detail child spans.

Suggested JSON fields:

{
  "attribution_label": "agent-a",
  "attribution_source": "request_header:x-agent-id",
  "attribution_confidence": "high"
}

If no label is available:

{
  "attribution_label": null,
  "attribution_source": "unknown",
  "attribution_confidence": "ambiguous"
}

SFT Export

Add an optional query parameter to both single and batch trajectory export endpoints:

min_attribution_confidence=high|medium|ambiguous

Expected behavior:

• If omitted, keep current export behavior.
• If set to high, export only turns whose relevant spans meet high-confidence attribution.
• For batch export, skip turns that do not meet the threshold and count them in the existing skipped export count.
• For single export, return a user-facing validation error explaining that attribution confidence is below the requested minimum.
• Include attribution metadata in the exported trajectory _meta object.

Suggested _meta shape:

{
  "attribution": {
    "label": "agent-a",
    "source": "request_header:x-agent-id",
    "confidence": "high"
  }
}

Turn-Level Filtering Rule

For turn-level SFT export, filtering should be conservative:

• If any span in the exported turn is below the requested confidence threshold, skip the turn.
• This avoids silently mixing ambiguous-source calls into a training sample.

Compatibility

Default behavior should remain unchanged:

• Existing config files still load.
• Existing DuckDB / ClickHouse databases migrate automatically.
• Existing capture pipelines continue to run.
• SFT export without min_attribution_confidence behaves as before, except the _meta object may include the new attribution metadata.

Suggested Implementation Areas

• h-common: shared attribution types and config.
• h-llm: derive attribution while converting HTTP exchanges into LlmCall.
• h-storage: extend query structs.
• h-storage-duckdb: schema, migration, write path, read path.
• h-storage-clickhouse: schema, migration, insert rows, read path.
• h-api: export query params and _meta output.
• app/heron: pass attribution config into the pipeline.
• docs: config reference and schema design notes.

Test Requirements

At minimum:

• Unit test: eBPF process attribution takes precedence over configured headers.
• Unit test: configured request header matching is case-insensitive.
• Unit test: no process and no configured header produces ambiguous.
• API/export test: min_attribution_confidence parses valid values and rejects invalid values.
• DuckDB tests: fresh schema write/read round-trip includes attribution metadata.
• DuckDB migration test: legacy DB gains attribution columns with default ambiguous values.
• Workspace compile check: all LlmCall fixture constructors are updated.

Suggested validation commands:

cargo check --workspace
cargo test -p h-llm attribution
cargo test -p h-api export
cargo test -p h-storage-duckdb
cargo test --workspace --no-run
git diff --check

Documentation Requirements

Update configuration docs with:

• [attribution] section.
• Example request_headers.
• Explanation that mTLS SAN must be promoted by the gateway into visible HTTP metadata.
• Explicit statement that passive capture cannot infer OS process attribution without visible identity metadata.
• SFT export example using min_attribution_confidence=high.

Update schema docs with:

• attribution_label.
• attribution_source.
• attribution_confidence.
• Default values for historical rows.

Acceptance Criteria

The feature is complete when:

• A gateway operator can configure a trusted header such as x-agent-id.
• Spans reconstructed from gateway/passive plaintext HTTP carry attribution_label=<header value>, attribution_source=request_header:<header name>, and attribution_confidence=high.
• eBPF spans continue to use native process attribution and are marked high confidence.
• Spans without either eBPF process identity or configured header identity are marked ambiguous.
• SFT batch export can exclude ambiguous-source turns using min_attribution_confidence=high.
• The implementation is backwards compatible with existing configs and databases.
• Tests cover attribution derivation, export filtering, and DuckDB migration.

Suggested Maintainer Response / PR Summary

This change makes Heron's attribution boundary explicit. eBPF capture still provides native process attribution. For gateway or passive capture, Heron can now accept a trusted identity key exposed in HTTP metadata, persist that attribution with confidence, and let SFT exports filter out ambiguous-source turns. This prevents training pipelines from silently using reconstructed but ambiguously attributed traffic.

[github-actions]

Hey Vincent — thanks for this writeup, it's one of the more thoroughly-scoped feature requests I've triaged in a while. The motivation (don't let ambiguous-source turns silently leak into SFT datasets) is exactly right, and the eBPF-vs-gateway distinction is a real gap in Heron today.

I dug into the code to size it. LlmCall lives in server/h-llm/src/model.rs and already carries process: Option<ProcessInfo> for the eBPF path, but there's no label/source/confidence triplet anywhere. Adding it means: a new [attribution] block in h-common/src/config.rs (AppConfig), deriving the label in h-llm/src/processor.rs where the LlmCall literal is built, adding three columns plus a new migration phase to h-storage-duckdb/src/schema.rs and the mirror h-storage-clickhouse/src/schema.rs, extending the span read models in h-storage/src/query.rs, threading the fields through h-api/src/routes/export.rs (both ExportTrajectoryParams and ExportBatchParams), and updating roughly 15–20 LlmCall { ... } fixture sites across h-llm and h-turn tests. The export filter and _meta enrichment then sit on top of all of that.

That's the rub: it touches seven crates plus schema migrations on two backends plus the API surface — and the spike PR (#179) clocks in at ~620 lines across 47 files, which lines up with what I see. That's well beyond what I'd hand to an unattended agent without a human steering. A single autopilot pass across two DB migrations and ~20 fixture sites is exactly where silent regressions hide.

I'd suggest splitting it into reviewable slices, each of which a human (or an agent with you watching) can land safely:

1. Types + config + derivation — AttributionInfo, [attribution] config, populate it in h-llm. Unit tests for eBPF-precedence, case-insensitive header match, ambiguous fallback. No storage yet.
2. DuckDB schema + migration + write/read — phase-9 migration, round-trip test, legacy-DB migration test.
3. ClickHouse mirror — same shape, separate PR.
4. API surface — expose on span detail / trace / list, with response tests.
5. SFT export filter — min_attribution_confidence on both endpoints, _meta enrichment, turn-level skip rule.

Happy to help scope #1 into something concrete if you'd like — it's the natural first PR and unblocks the rest. And thanks again for filing this with both a spec and a reference implementation; that's genuinely useful.

Maintainer controls

This isn't auto-queued for the dev agent. A maintainer can add agent:try to have it attempted anyway, or agent:skip to mute re-triage. After editing the issue, re-add agent:assess to run triage again.

[vaderyang]

It triggers too many changes regarding SFT export, it will be considered in future release.