Skip to content

feat: add privacy-safe raw control, integration tests, and docs for trajectory detail page - #430

Open
Hpward-Ling wants to merge 2 commits into
inclusionAI:mainfrom
Hpward-Ling:first
Open

feat: add privacy-safe raw control, integration tests, and docs for trajectory detail page#430
Hpward-Ling wants to merge 2 commits into
inclusionAI:mainfrom
Hpward-Ling:first

Conversation

@Hpward-Ling

Copy link
Copy Markdown

Motivation

Resolves #267.

The agentic trajectory detail page needs a privacy-safe default for raw sample exposure, integration test coverage for the trajectory detail endpoint, and user-facing documentation.

What this PR does

1. Privacy-safe raw sample control (areno/dashboard/server.py)

  • trajectory_detail() now accepts include_raw: bool = False keyword argument
  • Default behavior changed: raw sample dict is no longer attached by default
  • Pass include_raw=True or query param ?include_raw=true to explicitly request the raw sample under the raw key
  • Updated build_trajectory_detail docstring to reflect that raw is not included by default

Safe default: existing behavior of listing trajectory events, token counts, training mask summary, and end reason is preserved. The only change is that the full raw sample (including token sequences and loss mask arrays) is no longer exposed unless explicitly requested.

2. Integration tests (tests/test_dashboard_trajectory.py)

Added TestTrajectoryDetailIntegration class with 7 new test cases:

  • test_resolves_sample_from_registered_job — success path: registered job → valid trajectory detail
  • test_job_not_found_returns_error — error path: nonexistent job_id
  • test_sample_not_found_returns_error — error path: valid job, nonexistent step/prompt/sample indices
  • test_raw_omitted_by_default — privacy default: raw key absent without include_raw
  • test_raw_included_when_requestedinclude_raw=True attaches raw sample
  • test_non_agentic_sample_in_job_returns_invalid — non-agentic sample in a job returns valid: False
  • test_large_trace_does_not_crash — 50-turn tool call trace does not crash

Tests use a _registered_job fixture that directly manipulates the global STATE, requiring no HTTP server or external database.

3. Documentation (docs/cli/observability.rst)

Added "Dashboard trajectory detail page" section covering:

  • Feature overview (events timeline, final answer, token counts, training-mask, end reason)
  • Security guarantees (read-only display, no tool execution, collapsed long output)
  • API endpoint reference with query parameters and response field table
  • Copyable curl example with JSON response
  • Limitations

Testing

pytest tests/test_dashboard_trajectory.py -v

chengyan.ts added 2 commits July 30, 2026 14:40
…rajectory detail

- trajectory_detail() now defaults to omitting the raw sample dict; pass
  include_raw=True (or ?include_raw=true query param) to attach it
- HTTP route parses include_raw query parameter for the /api/jobs/<id>/trajectory
  endpoint
- Add TestTrajectoryDetailIntegration covering job/sample resolution, error
  paths, include_raw behavior, and large-trace robustness (7 new tests)
- Add 'Dashboard trajectory detail page' section to docs/cli/observability.rst
  with API reference, response field table, and curl example

Closes inclusionAI#267
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an agentic trajectory detail page to the dashboard

1 participant