Skip to content

Add CI artifacts for sanitized benchmark and profile outputs #43

@tony

Description

@tony

Summary

Keep CI artifact upload separate from the local observability work on streamline-02. The local profiler and benchmarker should land first; this issue can add CI upload support afterward without mixing workflow design into the profiler branch.

CI runners do not have representative local agent-history stores, so uploaded artifacts should be treated as harness-smoke evidence unless they are generated from explicit sanitized fixtures. Real bottleneck evidence should still come from local profiles.

References

Proposed work

  • Add a dedicated workflow or manual workflow dispatch for sanitized benchmark/profile artifacts.
  • Consider a small reusable composite action in this repository if multiple jobs need the same upload policy.
  • Use fixture-only profiler or benchmark payloads in CI; do not read real user agent stores.
  • Upload JSON/Markdown artifacts with explicit names that include the component and GitHub run metadata.
  • Use if-no-files-found: error for required artifacts and keep include-hidden-files: false unless there is a reviewed reason to change it.
  • Set a short retention period appropriate for diagnostic artifacts.

Non-goals

  • Do not treat CI timings as representative bottleneck evidence.
  • Do not upload prompt text, conversation text, raw argv, local absolute paths, secrets, or hidden files.
  • Do not make this part of the streamline-02 local profiler branch.

Acceptance criteria

  • CI can upload sanitized profiler/benchmark artifacts from fixture-backed runs.
  • Artifact names and descriptions make limits and components visible.
  • The workflow documents that local profiles remain the source of real bottleneck evidence.
  • Any reusable action is small, repo-local, and wraps actions/upload-artifact rather than reimplementing upload behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions