fix(result): normalize result artifact timestamps

[bingran-you]

Summary

Closes #528.

This keeps agent_result as the canonical result metrics object while preserving the existing top-level n_tool_calls, n_skill_invocations, and n_prompts fields as compatibility mirrors. Both native rollout and hosted-env writers now source those mirrors from agent_result, so the old flat fields cannot drift independently.

It also normalizes artifact timestamps at the result/config serialization boundary:

• result.json.started_at and result.json.finished_at now serialize as timezone-aware UTC ISO 8601 strings with a Z suffix.
• native config.json.started_at and hosted-env config.json.started_at use the same formatter.
• aware non-UTC started_at inputs are handled without naive/aware datetime arithmetic failures.

CI security follow-up

Fresh CI hit two transitive pip-audit advisories after the first push:

• click 8.3.1 -> 8.4.2
• cryptography 46.0.7 -> 48.0.1

cryptography>=48.0.1 requires moving the exact litellm[proxy] pin from 1.89.0 to 1.91.0, the first stable release in this range whose proxy extra permits the fixed cryptography line. LiteLLM-focused tests and the full suite pass on the updated lock.

Why

Issue #528 called out two related artifact-shape problems:

• consumers had to choose between duplicated top-level metrics and agent_result metrics;
• result timestamps were emitted via str(datetime), producing space-separated values with no timezone while other artifact fields already used ISO 8601.

Removing the top-level metric mirrors in one shot would break existing contract tests and likely downstream consumers, so this PR makes agent_result the source of truth first and leaves removal of compatibility mirrors as a deliberate later schema break.

Validation

• uv sync --extra dev --extra sandbox-daytona --locked
• uv run --extra dev python -m pytest tests/test_verify.py::TestResultJson tests/test_hosted_env_rollout_contract.py::test_hosted_env_writes_contract_result_json -q -> 7 passed
• uv run --extra dev python -m pytest tests/test_verify.py tests/test_hosted_env_rollout_contract.py tests/test_sdk_internals.py::TestBuildResult tests/test_config_redaction.py tests/test_usage_litellm.py tests/test_native_acp_usage.py tests/test_metrics.py -q -> 133 passed
• uv run --extra dev python -m pytest tests/test_litellm_config.py tests/test_litellm_hardening.py tests/test_litellm_logging.py tests/test_litellm_smoke.py tests/test_usage_litellm.py tests/test_agent_registry.py -q -> 134 passed, 2 skipped
• uv run --extra dev python -m pytest tests/test_trace_import_cli.py::test_tasks_generate_rejects_removed_short_options -q -> 4 passed
• uv run --extra dev python -m pytest tests/ -q -> 4855 passed, 16 skipped, 7 deselected
• uv export --locked --extra dev --extra sandbox-daytona --extra sandbox-modal --format requirements-txt --no-hashes --output-file /tmp/req-audit && uv run --with pip-audit pip-audit -r /tmp/req-audit --no-deps --disable-pip -> no known vulnerabilities found
• uv run ruff format --check pyproject.toml src/benchflow/_utils/timestamps.py src/benchflow/rollout/_results.py src/benchflow/hosted_env.py tests/test_verify.py tests/test_hosted_env_rollout_contract.py tests/test_trace_import_cli.py
• uv run ruff check .
• uv run ty check src/

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a8baaead0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Convert naive local times before writing UTC artifacts

When BenchFlow runs on a host whose local timezone is not UTC, native rollouts still create started_at with _init_rollout()'s local naive datetime.now() (src/benchflow/rollout/_setup.py:272). This branch then simply attaches UTC to that wall time, so result.json/config.json claim e.g. 10:00 local is 10:00Z instead of converting to the actual UTC instant, corrupting artifact ordering and cross-run comparisons outside UTC environments.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Name the guarded commit in timestamp regression tests

For these new regression tests, the docstrings cite only issue #528; /workspace/benchflow/AGENTS.md requires regression tests to name the PR or commit they guard. Please include this commit hash or the eventual PR number here and in the similar timestamp regression docstring below so future maintainers can trace the intended regression boundary.

Useful? React with 👍 / 👎.

[bingran-you]

Daily scan follow-up (2026-07-13): pushed d285ba940 to clear the fresh pip-audit failure on this PR.

What changed:

• click lock entry updated from 8.3.1 to 8.4.2 for PYSEC-2026-2132.
• cryptography lock entry updated from 46.0.7 to 48.0.1 for GHSA-537c-gmf6-5ccf.
• litellm[proxy] exact pin moved from 1.89.0 to 1.91.0 because 1.89.0 requires cryptography<47, while 1.91.0 permits cryptography>=48.0.1.
• The Click 8.4 error-message formatting regression test now asserts the removed option is rejected without pinning Click's old punctuation.

Local validation after the dependency update:

• exact CI pip-audit command -> no known vulnerabilities found
• LiteLLM/proxy-focused tests: 134 passed, 2 skipped
• removed-short-option CLI regression: 4 passed
• full unit suite: 4855 passed, 16 skipped, 7 deselected
• repo-wide ruff check . and ty check src/ passed

Waiting on refreshed GitHub checks for head d285ba940.

[bingran-you]

CI follow-up (2026-07-13): refreshed GitHub checks are now green on head d285ba940.

Passed checks:

• test
• pip-audit
• manifest-parity / parity
• integration-light / rollout-smoke

Labels moved back to status:ready + review:pending; remaining gate is non-author human review before merge.

[bingran-you]

Users Simulation automation review (2026-07-13): ready by simulation on head d285ba9401aa06918201d55c1463463672b1bfb1.

Scope checked: result/config timestamp normalization, agent_result as the source of truth for compatibility metric mirrors, hosted-env result shape, dependency audit follow-up, and broad CLI/SDK regression risk.

Evidence:

• Shared base smoke passed: uv sync --extra dev --extra sandbox-daytona --locked, ruff check ., ty check src, bench --help, bench agent list, bench tasks check --help, bench eval run --help, SDK Scene.single(agent="oracle") + RolloutConfig.
• PR worktree checks passed: uv sync --extra dev --extra sandbox-daytona --locked; pytest tests/test_verify.py tests/test_hosted_env_rollout_contract.py tests/test_sdk_internals.py::TestBuildResult tests/test_config_redaction.py tests/test_usage_litellm.py tests/test_native_acp_usage.py tests/test_metrics.py tests/test_trace_import_cli.py::test_tasks_generate_rejects_removed_short_options -q -> 137 passed; focused ruff; ty check src; CLI + SDK smoke.
• GitHub current-head checks are green: test, pip-audit, manifest-parity / parity, and integration-light / rollout-smoke passed.

Artifact health:

• Downloaded run 29236955437 and validated it with benchflow-experiment-review: 1/1 healthy, result.json, nested and rollout-level results.jsonl, trajectory/acp_trajectory.jsonl, and trajectory/llm_trajectory.jsonl present.
• The model rollout has 27 ACP events, 18/18 LLM exchanges with provider usage, one training-ready row, reward 1.0, 309,996 total tokens, 16 tool calls, and timing total 296.1s.
• result.json.started_at, result.json.finished_at, and config.json.started_at are UTC ISO timestamps with Z; top-level n_tool_calls / n_skill_invocations / n_prompts mirror agent_result.
• Caveat: GLM user-endpoint cost accounting is still unpriced (agent_result.cost_usd=null, price_source=null) even though token/timing telemetry is complete.

Thermo-nuclear review: no blocker. The timestamp helper is small and canonical, the compatibility mirrors now read from agent_result, and no changed production file crosses the 1k-line threshold. Keeping status:ready + review:pending; remaining gate is non-author human review.