Skip to content

Commit cf93e24

Browse files
SunrisepeakSunrisepeak
committed
ci: 手动触发可切换缓存 + 全量跑完给出每个成员的耗时排名
三件事,都先在本地跑通再接进 CI。 ── 1. tests/run_members.sh —— 本地与 CI 用同一份 ──────────────────────────── 把内联在 workflow 里的循环抽成脚本。理由不是整洁:**只存在于 CI 的耗时表没法用来 决定优化什么**,而与 CI 不一致的本地脚手架量的是另一回事。 bash tests/run_members.sh --all bash tests/run_members.sh --all --shard 3/8 bash tests/run_members.sh --all --cache local bash tests/run_members.sh opencv-module protobuf 本地实测发现一个 CI 上才会炸的 bug:`--all` 从 mcpp.toml 里 grep 成员名,而该文件 第 3 行的**散文**也写了 `tests/examples/`(「tests/examples/ — each consumes...」), 去掉前缀后是**空字符串** —— CI 上会变成 `mcpp test -p ""`。现在同时过滤空名和 不存在的目录(注释里被讨论到的 `tests/examples/asio-ssl` 也因此不会变成幽灵成员)。 过滤后 59 个,与目录一致。 失败不吞:任一成员失败则退出码非零,但**耗时表照常打印** —— 跑挂了的那次恰恰最 需要知道时间花在哪。 ── 2. workflow_dispatch 可选缓存模式 ─────────────────────────────────────── cache: global(默认)| local `local` 让每个成员各自重编全部依赖,是耗时表的对照基准。非 dispatch 事件时该 input 为空字符串,而 mcpp 的 `resolve_cache_mode` 只在**非空**时才认这个环境变量, 所以天然回落到默认的 global,不需要额外分支。 ── 3. 耗时排名 ──────────────────────────────────────────────────────────── 分片把成本藏起来了:八个 runner 各报各的,没人看得见到底谁在吃时间。每片把 `<秒>\t<成员>\t<ok|FAIL>` 传成 artifact,新增 `timings` job 按平台合并,排名写进 run summary(带占比)。`always()` —— 失败的那次正是最该读它的时候。 汇总的合并/排序/占比逻辑已在本地用构造数据验证过。 表里的 total 是**各片之和**,墙钟是**最慢那片** —— 这两个数不是一回事,summary 里写明了,免得下次拿总和当墙钟看。
1 parent 7218bef commit cf93e24

2 files changed

Lines changed: 183 additions & 7 deletions

File tree

‎.github/workflows/validate.yml‎

Lines changed: 76 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ on:
1212
# nightly full regression — exercises every workspace member regardless of diff
1313
- cron: "0 6 * * *"
1414
workflow_dispatch:
15+
inputs:
16+
cache:
17+
description: "Package build cache — 'local' rebuilds every dependency per member, which is what the timing table should be read against when comparing"
18+
type: choice
19+
options: [global, local]
20+
default: global
1521

1622
env:
1723
# 2026.8.5.3 carries two things this workflow depends on:
@@ -553,18 +559,27 @@ jobs:
553559
# into this runner's actual member names, so `mcpp test --workspace`
554560
# — which would ignore the sharding and rebuild everything here — is
555561
# gone.
562+
#
563+
# tests/run_members.sh is the SAME script you run locally. A timing
564+
# table that only exists in CI cannot be used while deciding what to
565+
# optimise, and a local harness that differs from CI measures
566+
# something else.
556567
if [ -z "$MEMBERS" ]; then
557568
echo "No workspace member affected by this change — nothing to test."
558569
else
559-
rc=0
560-
for m in $MEMBERS; do
561-
echo "::group::mcpp test -p $m"
562-
"$MCPP" test -p "$m" || rc=1
563-
echo "::endgroup::"
564-
done
565-
exit $rc
570+
MCPP_TIMINGS="$PWD/timings.tsv" bash tests/run_members.sh $MEMBERS
566571
fi
567572
573+
# Per-shard timings, merged by the `timings` job below. `always()`: a
574+
# run that failed is exactly when knowing where the time went matters.
575+
- name: Upload this shard's timings
576+
if: always() && hashFiles('timings.tsv') != ''
577+
uses: actions/upload-artifact@v4
578+
with:
579+
name: timings-${{ matrix.platform }}-${{ matrix.shard }}
580+
path: timings.tsv
581+
retention-days: 14
582+
568583
# install()-driven packages (openssl, openblas) build through their own
569584
# Make/Configure system, whose output xim's interface mode swallows; a
570585
# failed hook surfaces only as `E_INTERNAL: [<pkg>] failed:`. Each writes
@@ -583,3 +598,57 @@ jobs:
583598
echo "::endgroup::"
584599
done < <(find tests/examples "$HOME/.mcpp/registry" -name 'mcpp_*_build.log' 2>/dev/null)
585600
[ "$found" = 1 ] || echo "no install() build logs found"
601+
602+
# ── Where the time went ───────────────────────────────────────────────
603+
# Sharding hides the cost: eight runners each report their own slice, and
604+
# nobody can see which members actually dominate. This merges them into one
605+
# ranking per platform, in the run summary, so the next optimisation starts
606+
# from measurement instead of a guess.
607+
#
608+
# `always()` — a failed run is exactly when this is worth reading.
609+
timings:
610+
needs: [select, workspace]
611+
if: always() && needs.select.outputs.members != ''
612+
runs-on: ubuntu-latest
613+
steps:
614+
- uses: actions/download-artifact@v4
615+
with:
616+
pattern: timings-*
617+
path: timings
618+
continue-on-error: true
619+
- name: Rank members by wall-clock
620+
shell: bash
621+
run: |
622+
shopt -s nullglob
623+
files=(timings/*/timings.tsv)
624+
if [ ${#files[@]} -eq 0 ]; then
625+
echo "no timing data (every shard skipped or failed before testing)" \
626+
>> "$GITHUB_STEP_SUMMARY"
627+
exit 0
628+
fi
629+
630+
# Artifact name carries the platform: timings-<platform>-<shard>.
631+
for plat in linux macos windows; do
632+
rows=$(mktemp)
633+
for f in timings/timings-$plat-*/timings.tsv; do
634+
[ -f "$f" ] && cat "$f" >> "$rows"
635+
done
636+
[ -s "$rows" ] || { rm -f "$rows"; continue; }
637+
638+
total=$(awk -F'\t' '{s += $1} END {print s+0}' "$rows")
639+
count=$(wc -l < "$rows")
640+
{
641+
echo "### $plat — ${count} member(s), ${total}s of member wall-clock"
642+
echo
643+
echo "| rank | seconds | share | member | result |"
644+
echo "|---:|---:|---:|---|---|"
645+
sort -rn "$rows" | awk -F'\t' -v tot="$total" '
646+
{ pct = tot > 0 ? ($1 * 100 / tot) : 0
647+
printf "| %d | %s | %.1f%% | `%s` | %s |\n", NR, $1, pct, $2, $3 }'
648+
echo
649+
} >> "$GITHUB_STEP_SUMMARY"
650+
rm -f "$rows"
651+
done
652+
653+
echo "_Total is the SUM across shards; wall-clock is the slowest shard._" \
654+
>> "$GITHUB_STEP_SUMMARY"

‎tests/run_members.sh‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
#!/usr/bin/env bash
2+
# run_members.sh — run workspace members one by one, timing each.
3+
#
4+
# The same script CI runs and you run locally, on purpose: a timing table that
5+
# only exists in CI cannot be used while deciding what to optimise, and a local
6+
# harness that differs from CI measures something else.
7+
#
8+
# bash tests/run_members.sh --all
9+
# bash tests/run_members.sh opencv-module protobuf
10+
# bash tests/run_members.sh --all --shard 3/8
11+
# bash tests/run_members.sh --all --cache local # bypass the package cache
12+
#
13+
# Env:
14+
# MCPP path to the mcpp binary (default: `mcpp` on PATH)
15+
# MCPP_TIMINGS where to append `<seconds>\t<member>\t<ok|FAIL>` rows
16+
#
17+
# Exit status is non-zero if any member failed. The timing table is printed
18+
# regardless — a slow run is worth measuring even when it breaks.
19+
set -u
20+
21+
MCPP="${MCPP:-mcpp}"
22+
timings="${MCPP_TIMINGS:-}"
23+
cache=""
24+
shard=""
25+
members=()
26+
all=0
27+
28+
while [ $# -gt 0 ]; do
29+
case "$1" in
30+
--all) all=1; shift ;;
31+
--shard) shard="$2"; shift 2 ;;
32+
--cache) cache="$2"; shift 2 ;;
33+
--timings) timings="$2"; shift 2 ;;
34+
-h|--help) sed -n '2,20p' "$0"; exit 0 ;;
35+
-*) echo "unknown option: $1" >&2; exit 2 ;;
36+
*) members+=("$1"); shift ;;
37+
esac
38+
done
39+
40+
# `--all` reads the workspace manifest rather than the directory, so a member
41+
# that exists on disk but is not registered is not silently tested.
42+
#
43+
# Both filters below are load-bearing. mcpp.toml's PROSE mentions the path too
44+
# — line 3 says "tests/examples/ — each consumes this repo's own packages",
45+
# which this grep matches with an empty tail, and `mcpp test -p ""` is not a
46+
# useful thing to run. Requiring a real directory also means a name that only
47+
# appears in a comment (`tests/examples/asio-ssl` is discussed in one) cannot
48+
# turn into a phantom member.
49+
if [ "$all" = 1 ]; then
50+
while IFS= read -r m; do
51+
[ -n "$m" ] || continue
52+
[ -d "tests/examples/$m" ] || continue
53+
members+=("$m")
54+
done < <(grep -o 'tests/examples/[A-Za-z0-9._-]*' mcpp.toml \
55+
| sed 's|tests/examples/||' | sort -u)
56+
fi
57+
58+
if [ "${#members[@]}" -eq 0 ]; then
59+
echo "no members selected — pass names or --all" >&2
60+
exit 2
61+
fi
62+
63+
# --shard N/M keeps every M-th member starting at N. Round-robin by position,
64+
# which is what separates adjacent expensive members (opencv-module,
65+
# -dnn, -unifont) onto different runners.
66+
if [ -n "$shard" ]; then
67+
idx=${shard%%/*}
68+
cnt=${shard##*/}
69+
picked=()
70+
i=0
71+
for m in "${members[@]}"; do
72+
[ $((i % cnt)) -eq "$idx" ] && picked+=("$m")
73+
i=$((i + 1))
74+
done
75+
members=("${picked[@]+"${picked[@]}"}")
76+
echo "shard $idx/$cnt -> ${#members[@]} member(s)"
77+
fi
78+
79+
[ -n "$cache" ] && export MCPP_BUILD_CACHE="$cache"
80+
echo "cache mode: ${MCPP_BUILD_CACHE:-global (default)}"
81+
82+
rows=$(mktemp)
83+
trap 'rm -f "$rows"' EXIT
84+
rc=0
85+
86+
for m in "${members[@]}"; do
87+
echo "::group::mcpp test -p $m"
88+
t0=$(date +%s)
89+
if "$MCPP" test -p "$m"; then status=ok; else status=FAIL; rc=1; fi
90+
t1=$(date +%s)
91+
echo "::endgroup::"
92+
printf '%s\t%s\t%s\n' "$((t1 - t0))" "$m" "$status" >> "$rows"
93+
printf ' %-34s %5ss %s\n' "$m" "$((t1 - t0))" "$status"
94+
done
95+
96+
[ -n "$timings" ] && cat "$rows" >> "$timings"
97+
98+
echo
99+
echo "── slowest members ──────────────────────────────────────────"
100+
total=$(awk -F'\t' '{s += $1} END {print s+0}' "$rows")
101+
sort -rn "$rows" | head -15 | awk -F'\t' -v tot="$total" '
102+
{ pct = tot > 0 ? ($1 * 100 / tot) : 0
103+
printf " %6ss %5.1f%% %-34s %s\n", $1, pct, $2, $3 }'
104+
echo " ────────"
105+
printf ' %6ss total across %s member(s)\n' "$total" "${#members[@]}"
106+
107+
exit "$rc"

0 commit comments

Comments
 (0)