Skip to content

Missing Human-in-the-Loop Tracing Support #401

@prajwalun

Description

@prajwalun

Title: Missing tracing support for human-in-the-loop agent workflows

Description:
My resume agent pauses for human input when it detects potential fabrication risks or needs clarification. The current tracing API doesn't handle these pause/resume scenarios properly, making it difficult to track the complete workflow.

Problem:
When my agent pauses for user clarification, the traces break or become disconnected:

if fabrication_risk['needs_clarification']:
    # How do we trace this pause for user input?
    request_clarification()
    continue  # Resume processing after user input

This creates fragmented traces that are hard to debug and monitor.

Expected Behavior:
Tracing should support pause/resume patterns for human-in-the-loop workflows:

# Option 1: Pause/resume context managers
with judgment.pause_for_human_input(reason="clarification_needed"):
    user_response = await get_user_clarification()
    
with judgment.resume_from_human_input(previous_span_id=span_id):
    continue_processing(user_response)

# Option 2: Session continuation
session_id = judgment.get_current_session_id()
# ... pause for user input ...
judgment.resume_session(session_id, user_input=user_response)

Current Behavior:

  • Traces become fragmented when agent execution pauses for human input
  • No way to correlate spans before and after human interaction
  • Difficult to debug and monitor complete workflows
  • Session continuity is lost

Impact:
This makes it impossible to properly monitor and debug human-in-the-loop agent workflows, which are common in production systems.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions