Skip to content

Docs/observability surfacedocs: document the current observability surface - #447

Open
qsxns wants to merge 8 commits into
inclusionAI:mainfrom
qsxns:docs/observability-surface
Open

Docs/observability surfacedocs: document the current observability surface#447
qsxns wants to merge 8 commits into
inclusionAI:mainfrom
qsxns:docs/observability-surface

Conversation

@qsxns

@qsxns qsxns commented Aug 3, 2026

Copy link
Copy Markdown

Closes #45
Closes #122

Summary

Documents the current observability surface of AReno — what training
emits through console logs, train_stats, TensorBoard scalar metrics,
and agentic trajectory diagnostics, and how to inspect each during
debugging.

This is a docs-only change. No code, dependencies, or CLI surface are
modified.

What's documented

docs/reference/environment-variables.rst

  • ARENO_LOG_LEVEL — controls the areno logger level (default INFO).
    Set to DEBUG for per-request decode progress and fine-grained engine
    diagnostics.
  • ARENO_LOG_COMPLETIONS — number of decoded rollout samples persisted per
    training step (default 1), written to rollout_samples.{pid}.jsonl.

docs/cli/observability.rst

  • Console logs section now covers log-level control via ARENO_LOG_LEVEL.
  • New "Reading metrics during debugging" section with an
    EventAccumulator code example for reading TensorBoard scalars
    programmatically.
  • New "Dashboard" section covering areno dashboard --start/--stop, the
    default port, and the metrics-directory artifacts
    (dashboard_state.{pid}.json, rollout_samples.{pid}.jsonl,
    areno_run_config.{pid}.json / .txt).

The existing content already cited areno.api.metrics, --metrics-log-dir,
trainer logging paths, and the rollout/*, train/*, time/* namespaces,
and explicitly notes that AReno does not yet provide a built-in wandb
integration. All of that is preserved.

Acceptance criteria

  • Cites areno.api.metrics, --metrics-log-dir, and trainer logging paths
  • Does not claim wandb support yet
  • Includes at least one example of reading a metric during debugging
  • Documents logs, train_stats, TensorBoard, and agentic trajectory diagnostics
  • Covers rollout/*, train/*, and time/* metric namespaces
  • Explains how to launch TensorBoard for a run

Verification

  • All 17 acceptance-criteria checks pass.
  • RST heading structure validated — no underline-length errors.
  • Every factual claim cross-checked against the source
    (areno/cli/dashboard.py, areno/cli/train.py, areno/api/metrics.py,
    areno/api/defaults.py).
  • The dashboard CLI syntax in the docs was corrected to match the real
    interface (--start/--stop, no --no-browser).
  • Sphinx is not installed locally, so make html was not run; the RST
    follows existing patterns in the same file.

Notes

  • docs/cli/observability.rst remains an orphan page (not linked from the
    toctree) — consistent with the existing cross-references from
    docs/reference/cli.rst and docs/cli/training.rst.

秦亮 and others added 8 commits August 3, 2026 17:17
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…fact reference

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…face

areno dashboard uses --start/--stop flags, not start/stop subcommands, and
has no --no-browser option. The server starts in the background and prints
the URL rather than opening a browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The design spec and implementation plan under docs/superpowers/ are
agent-workflow artifacts, not user-facing documentation. Remove them so
the PR contains only the observability doc changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qsxns

qsxns commented Aug 3, 2026

Copy link
Copy Markdown
Author

#45

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document the current observability surface

1 participant