Refactor IntermediateStep to ATIFConverter

examples/advanced_agents/alert_triage_agent/src/nat_alert_triage_agent/telemetry_metrics_analysis_agent.py

@@ -16,11 +16,13 @@
 from pydantic import Field
 
 from nat.builder.builder import Builder
+from nat.builder.context import Context
 from nat.builder.framework_enum import LLMFrameworkEnum
 from nat.builder.function_info import FunctionInfo
 from nat.cli.register_workflow import register_function
 from nat.data_models.component_ref import LLMRef
 from nat.data_models.function import FunctionBaseConfig
+from nat.data_models.intermediate_step import TraceMetadata
 
 from . import utils
 from .prompts import TelemetryMetricsAnalysisAgentPrompts
@@ -88,7 +90,14 @@ def telemetry_metrics_analysis_agent(state: MessagesState):
 
         # Execute analysis and get response
         input_message = f"Host to investigate: {host_id}. Alert type: {alert_type}"
-        response = await agent_executor.ainvoke({"messages": [HumanMessage(content=input_message)]})
+        delegation_metadata = TraceMetadata(provided_metadata={"is_subagent_delegation": True})
+        with Context.get().push_active_function("telemetry_metrics_analysis_subagent_call",
+                                                input_data={
+                                                    "host_id": host_id,
+                                                    "alert_type": alert_type,
+                                                },
+                                                metadata=delegation_metadata):
+            response = await agent_executor.ainvoke({"messages": [HumanMessage(content=input_message)]})
 
         conclusion = response["messages"][-1].content
 

packages/nvidia_nat_atif/intermediate-step-to-atif-mapping.md

@@ -18,72 +18,97 @@ limitations under the License.
 
 # IntermediateStep to ATIF Mapping
 
-This document explains how IntermediateStep event streams are mapped to ATIF trajectories.
-
-It is intentionally generic and applies to the current conversion path used by the toolkit.
-
-## ID Mappings
+This document defines the current conversion contract from NAT `IntermediateStep`
+events to ATIF trajectories.
+
+The conversion is intentionally split into two passes:
+
+1. **Pass 1: build execution structure**
+   - Determine root context ownership and delegated child context ownership.
+   - Build subagent linkage metadata from explicit delegation markers.
+2. **Pass 2: project contexts to ATIF**
+   - Convert each context into ATIF steps using deterministic step window rules.
+   - Emit flat tool call and observation rows with lineage metadata for nesting reconstruction.
+
+The converter code implementing this model lives in:
+`packages/nvidia_nat_core/src/nat/utils/atif_converter.py`, with key entry points:
+`_pass1_build_execution_structure()` and `_pass2_project_context_to_steps()`.
+
+## Pass 1: Execution Structure Rules
+
+- Events are ordered by `event_timestamp`.
+- Context ownership is computed from callable lineage and explicit delegation markers.
+- Delegation boundaries are explicit only:
+  - marker source: `metadata.provided_metadata["is_subagent_delegation"] == true`
+  - no implicit delegation inference in the clean mapping path.
+- Pass 1 outputs:
+  - `root_events`
+  - `child_events_by_session`
+  - `subagent_ref_by_call_id`
+
+## Pass 2: Context Projection Rules
+
+- Each context is projected independently into ATIF steps.
+- `WORKFLOW_START` can emit a `source="user"` step for the root context.
+- Step windows are anchored by terminal events:
+  - `LLM_END`
+  - `TOOL_END`
+  - `FUNCTION_END`
+  - `WORKFLOW_END` (terminal output only when not redundant)
+- Tool rows are flat by design:
+  - nested tool calls are kept in `tool_calls`, `observation.results`,
+    `extra.tool_ancestry`, and `extra.tool_invocations`
+  - subscribers reconstruct nesting using ancestry `function_id` and `parent_id`.
+- Chunk events (`LLM_NEW_TOKEN`, `SPAN_CHUNK`) do not emit standalone ATIF steps.
+
+## Subagent Trajectory Rules
+
+- Parent observation rows include `subagent_trajectory_ref` only for explicitly
+  marked delegating calls.
+- Child trajectories are built from delegated context events only.
+- Embedded children are stored under:
+  - `trajectory.extra["subagent_trajectories"]`
+- Reference resolution precedence:
+  1. `subagent_trajectory_ref.trajectory_path`
+  2. `trajectory.extra.subagent_trajectories[session_id]`
+
+## ID and Lineage Mapping
 
 | IntermediateStep | ATIF | Mapping Rule | Notes |
 |---|---|---|---|
 | `payload.UUID` | `tool_calls[*].tool_call_id` | `tool_call_id = "call_" + payload.UUID` | Invocation occurrence identity |
-| `payload.UUID` | `observation.results[*].source_call_id` | `source_call_id = tool_call_id` | Observation links to invocation |
-| `payload.UUID` | `extra.tool_invocations[*].invocation_id` | `invocation_id = tool_call_id` | Timing row identity for same invocation |
+| `payload.UUID` | `observation.results[*].source_call_id` | `source_call_id = tool_call_id` | Observation-to-call link |
+| `payload.UUID` | `extra.tool_invocations[*].invocation_id` | `invocation_id = tool_call_id` | Invocation timing row identity |
 | `function_ancestry.function_id` | `extra.tool_ancestry[*].function_id` | Direct match | Callable lineage node identity |
-| `function_ancestry.parent_id` | `extra.tool_ancestry[*].parent_id` | Direct match | Parent callable lineage node identity |
-| `function_ancestry.function_id` (step context) | `extra.ancestry.function_id` | Direct match | Step-level callable context |
-| Not applicable | `step_id` | Generated ATIF sequence counter | Not derived from IntermediateStep UUID |
+| `function_ancestry.parent_id` | `extra.tool_ancestry[*].parent_id` | Direct match | Lineage parent node identity |
+| `function_ancestry.function_id` (step anchor) | `extra.ancestry.function_id` | Direct match | Step context identity |
+| Not applicable | `step_id` | Generated sequentially from 1 | Not derived from UUID |
 
-## Name Mappings
+## Name Mapping
 
 | IntermediateStep | ATIF | Mapping Rule | Notes |
 |---|---|---|---|
-| `payload.name` (tool, function, or LLM by event type) | `tool_calls[*].function_name` or `model_name` | Context dependent | IntermediateStep name is polymorphic by event type |
-| `function_ancestry.function_name` | `extra.tool_ancestry[*].function_name` | Direct match | Callable lineage node name |
-| `function_ancestry.parent_name` | `extra.tool_ancestry[*].parent_name` | Direct match | Parent callable lineage node name |
-| `function_ancestry.function_name` (step context) | `extra.ancestry.function_name` | Direct match | Step-level lineage name |
-
-## Event-to-Step Mapping
-
-- `WORKFLOW_START` maps to an ATIF user step (`source = "user"`).
-- `LLM_END` starts an ATIF agent turn candidate step.
-- `TOOL_END` and `FUNCTION_END` are accumulated into the pending ATIF step as observed invocations.
-- `WORKFLOW_END` may emit a terminal ATIF agent step when final output is present and not redundant.
-- `LLM_NEW_TOKEN` and other non-terminal chunk events are not directly emitted as standalone ATIF steps.
-
-## Identity Semantics
-
-- Invocation identity and callable identity are intentionally different:
-  - Invocation identity: `tool_call_id`, `source_call_id`, `invocation_id`
-  - Callable identity: `function_id`, `parent_id`
-- Correct lineage interpretation requires both:
-  - use invocation IDs for per-call correlation,
-  - use callable IDs for hierarchy and repeated-call disambiguation.
-
-## Timing Mappings
-
-- IntermediateStep end events commonly use:
-  - `event_timestamp` as end timestamp,
-  - `span_event_timestamp` as start timestamp.
-- ATIF timing is represented in:
-  - `extra.invocation.start_timestamp` and `extra.invocation.end_timestamp` (step-level),
-  - `extra.tool_invocations[*].start_timestamp` and `extra.tool_invocations[*].end_timestamp` (per invocation).
-
-## Practical Validation Checklist
-
-- Verify `tool_call_id == source_call_id == invocation_id` for each ATIF invocation row.
-- Verify `tool_call_id == "call_" + payload.UUID` for mapped tool or function end events.
-- Verify callable lineage consistency:
-  - `function_ancestry.function_id <-> extra.tool_ancestry[*].function_id`
-  - `function_ancestry.parent_id <-> extra.tool_ancestry[*].parent_id`
-- Verify name consistency:
-  - `function_ancestry.function_name <-> extra.tool_ancestry[*].function_name`
-  - `function_ancestry.parent_name <-> extra.tool_ancestry[*].parent_name`
-
-## Additional Identifiers Worth Tracking
-
-- IntermediateStep structural parent linkage: `parent_id`
-- Event semantics: `payload.event_type`
-- Time surfaces: `event_timestamp`, `span_event_timestamp`, ATIF `timestamp`
-- Session scope: ATIF `session_id`
-- Framework or provider run IDs when present in metadata (for example, model framework trace IDs)
+| `payload.name` on `LLM_END` | `model_name` | Direct match | Model name for agent step |
+| `payload.name` on `TOOL_END` or `FUNCTION_END` | `tool_calls[*].function_name` | Direct match | Invocation display name |
+| `function_ancestry.function_name` | `extra.tool_ancestry[*].function_name` | Direct match | Callable lineage name |
+| `function_ancestry.parent_name` | `extra.tool_ancestry[*].parent_name` | Direct match | Parent lineage name |
+
+## Timing Mapping
+
+- `span_event_timestamp` maps to start timestamp surfaces when present.
+- `event_timestamp` maps to end timestamp surfaces.
+- ATIF timing fields:
+  - `extra.invocation.start_timestamp`
+  - `extra.invocation.end_timestamp`
+  - `extra.tool_invocations[*].start_timestamp`
+  - `extra.tool_invocations[*].end_timestamp`
+
+## Validation Checklist
+
+- `tool_call_id == source_call_id == invocation_id` for each projected invocation row.
+- `tool_call_id == "call_" + payload.UUID` for mapped tool and function end events.
+- Lineage consistency:
+  - `function_ancestry.function_id` equals `extra.tool_ancestry[*].function_id`
+  - `function_ancestry.parent_id` equals `extra.tool_ancestry[*].parent_id`
+- Step IDs are sequential and start from 1.
+- Every emitted `subagent_trajectory_ref.session_id` resolves by precedence rules.