Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/rulesets/branches/default-branch.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"parameters": {
"required_approving_review_count": 1,
"dismiss_stale_reviews_on_push": true,
"required_reviewers": [],
"require_code_owner_review": false,
"require_last_push_approval": false,
"required_review_thread_resolution": true,
Expand All @@ -40,7 +41,8 @@
"do_not_enforce_on_create": false,
"required_status_checks": [
{
"context": "call-tool-crucible-ci / tool-crucible-ci-complete"
"context": "crucible-ci-complete",
"integration_id": 15368
}
]
}
Expand Down
52 changes: 43 additions & 9 deletions .github/workflows/crucible-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,35 @@ name: crucible-ci
on:
pull_request:
branches: [ master ]
paths-ignore:
- LICENSE
- '**.md'
- '.github/rulesets/**'
- .github/workflows/run-crucible-tracking.yaml
- .github/workflows/crucible-merged.yaml
- .github/workflows/faux-crucible-ci.yaml
- 'docs/**'
workflow_dispatch:

concurrency:
group: ${{ github.ref }}/crucible-ci
cancel-in-progress: true

jobs:
call-tool-crucible-ci:
changes:
runs-on: ubuntu-latest
outputs:
only-docs: ${{ steps.filter.outputs.only_changed }}
steps:
- uses: actions/checkout@v4
- id: filter
uses: tj-actions/changed-files@v47
with:
files: |
LICENSE
*.md
**/*.md
.github/rulesets/**
.github/workflows/run-crucible-tracking.yaml
.github/workflows/crucible-merged.yaml
.github/workflows/crucible-ci.yaml
docs/**

call-real-tool-crucible-ci:
needs: changes
if: ${{ github.event_name == 'workflow_dispatch' || needs.changes.outputs.only-docs != 'true' }}
uses: perftool-incubator/crucible-ci/.github/workflows/tool-crucible-ci.yaml@main
with:
ci_target: "sysstat"
Expand All @@ -23,3 +40,20 @@ jobs:
secrets:
ci_registry_auth: ${{ secrets.CRUCIBLE_CI_ENGINES_REGISTRY_AUTH }}
quay_oauth_token: ${{ secrets.CRUCIBLE_QUAYIO_OAUTH_TOKEN }}

call-faux-tool-crucible-ci:
needs: changes
if: ${{ github.event_name != 'workflow_dispatch' && needs.changes.outputs.only-docs == 'true' }}
uses: perftool-incubator/crucible-ci/.github/workflows/faux-tool-crucible-ci.yaml@main

crucible-ci-complete:
needs: [ call-real-tool-crucible-ci, call-faux-tool-crucible-ci ]
if: always()
runs-on: ubuntu-latest
steps:
- name: Check Results
if: >-
contains(needs.*.result, 'failure') ||
contains(needs.*.result, 'cancelled')
run: exit 1
- run: echo "crucible-ci complete"
17 changes: 0 additions & 17 deletions .github/workflows/faux-crucible-ci.yaml

This file was deleted.

26 changes: 26 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Sysstat Tool

## Purpose
Collects system performance metrics using Linux sysstat utilities (mpstat, sar, iostat, pidstat) during benchmark execution and post-processes the raw output into crucible's canonical metric format.

## Languages
- Bash: collection scripts (`sysstat-start`, `sysstat-stop`)
- Perl: post-processor (`sysstat-post-process`)

## Key Files
| File | Purpose |
|------|---------|
| `sysstat-start` | Launches configured subtools with `--subtools` and `--interval` parameters |
| `sysstat-stop` | Kills running collectors, compresses output with xz |
| `sysstat-post-process` | Parses raw sysstat output into crucible metrics (uses `toolbox::metrics`, `toolbox::json` from `$TOOLBOX_HOME/perl`) |
| `rickshaw.json` | Rickshaw integration: endpoint allow/block lists, file deployment, post-process script |
| `workshop.json` | Engine image build: compiles sysstat v12.5.1 from source |

## Configuration
- `--subtools <list>` — Comma-separated subtools to run (default: `mpstat,sar,iostat,pidstat`)
- `--interval <seconds>` — Collection interval (default: `3`)

## Conventions
- Primary branch is `master`
- Runs as a profiler tool on master/worker/profiler roles, blocked on client/server
- Standard Bash/Perl modelines and 4-space indentation
29 changes: 26 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,28 @@
# tool-sysstat
Scripts to help run sysstat tools and post-process its data
[![CI Actions Status](https://github.com/perftool-incubator/tool-sysstat/workflows/crucible-ci/badge.svg)](https://github.com/perftool-incubator/tool-sysstat/actions)

#### rickshaw.json
A config file to describe how to run and post-process sysstat, as well as where sysstat should and should not run.
System performance metric collection and post-processing for the [crucible](https://github.com/perftool-incubator/crucible) framework using the Linux [sysstat](https://github.com/sysstat/sysstat) utilities.

## Subtools

Sysstat runs one or more subtools in parallel during test execution. The default set is all four:

| Subtool | Data Collected |
|---------|---------------|
| mpstat | Per-CPU utilization statistics |
| sar | Memory (paging, swapping, utilization), network device statistics |
| iostat | Disk I/O throughput and latency |
| pidstat | Per-process CPU, memory, and I/O statistics |

## Configuration

The start script accepts two parameters:
- `--subtools <list>` — Comma-separated list of subtools to run (default: `mpstat,sar,iostat,pidstat`)
- `--interval <seconds>` — Collection interval in seconds (default: `3`)

## Integration

Sysstat runs as a profiler tool on endpoint nodes. It is allowed on profiler, master, and worker collector roles but blocked on client and server roles. The post-processor (`sysstat-post-process`) converts raw sysstat output into crucible's canonical metric format.

### rickshaw.json
Defines how sysstat integrates with rickshaw: which files to deploy to engines, which endpoint/collector-type combinations are allowed or blocked, and the post-processing script.
10 changes: 1 addition & 9 deletions rickshaw.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@
"endpoint": "remotehosts",
"collector-types": [ "client", "server" ]
},
{
"endpoint": "k8s",
"collector-types": [ "client", "server" ]
},
{
"endpoint": "kube",
"collector-types": [ "client", "server" ]
Expand All @@ -42,13 +38,9 @@
"endpoint": "remotehosts",
"collector-types": [ "profiler" ]
},
{
"endpoint": "k8s",
"collector-types": [ "master", "worker" ]
},
{
"endpoint": "kube",
"collector-types": [ "master", "worker" ]
"collector-types": [ "profiler" ]
}
],
"start": "sysstat-start",
Expand Down
Loading