Skip to content

Integrate torc for benchmark orchestration (optional feature) #159

@pesap

Description

@pesap

Description

Add optional torc integration to `arco-bench` for distributed benchmark orchestration. This enables:

  • Parallel benchmark execution across multiple nodes
  • SQLite-based result storage with full provenance (git commit, hardware info, timestamps)
  • Change detection (only re-run benchmarks for changed code)
  • REST API to query historical performance data
  • Slurm integration for HPC benchmark runs

Implementation

Add `torc` feature flag to `arco-bench`:

When `torc` feature enabled:

  • Benchmarks defined as KDL workflow files (similar to torc workflows)
  • Submit to torc server via HTTP API
  • Poll for completion with resource metrics
  • Store results in torc's SQLite database

When `torc` feature disabled (default):

  • Run benchmarks locally with current behavior
  • JSONL output for CI comparison

Usage

Context

Depends on #139 (Locked benchmark suite). The benchmark suite should be implemented first with local execution, then this adds the optional torc layer for distributed/tracked runs.

Acceptance Criteria

  • `arco-bench` compiles with and without `torc` feature
  • Benchmark workflows can be defined in KDL format
  • CI job demonstrates torc-based benchmark execution
  • Documentation for setting up local torc server for benchmarks

Target: v0.5 (optional enhancement to benchmark suite)
Priority: Medium (blocks on #139)

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarksPerformance testing and regressionenhancementNew feature or requestv0.5Version 0.5: Benchmarks + Logging

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions