ci: bound every job at 120 minutes - #6005
Conversation
No TAP job had a timeout. TEST_TAP_TIMEOUT also defaults to 0 (no per-test timeout), so a single hung test ran until GitHub's 6-hour ceiling. That happened on #5991: CI-mysql84-g9 sat for hours on test_ssl_fast_forward-3_libmariadb-t and produced no useful signal. It is worse than slow. GitHub marks a run killed by the 6h ceiling as 'cancelled', not 'failure', so every archive step guarded by `failure() && !cancelled()` is skipped -- the run that most needs its logs is the one that uploads none. Adds `timeout-minutes: 120` to all 83 jobs across 64 workflows. 120 is well above the slowest observed successful group (CI-legacy-g4, ~58 min) and matches the value ci-pg-compat.yml already uses. Not changed: - ci-pg-compat.yml (already 120) - ci-repltests.yml, ci-shuntest.yml (already 45, deliberately tighter) Jobs that only call a reusable workflow are untouched: `timeout-minutes` is not a valid key on a `uses:` job, which is why this has to live on the callee side here rather than in the CI-*.yml callers on v3.0. All 64 files re-parse as valid YAML.
|
Warning Review limit reached
Next review available in: 13 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughThe pull request adds 120-minute job timeouts and selected 90-minute test-step timeouts to GitHub Actions workflows. The trigger watcher receives a 240-minute timeout and explanatory comments. ChangesCI timeout standardization
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 98324909ce
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| jobs: | ||
| tests: | ||
| runs-on: ubuntu-22.04 | ||
| timeout-minutes: 120 |
There was a problem hiding this comment.
When Run mysql84-g9 tests hangs for 120 minutes, this job-level deadline terminates the entire tests job rather than merely failing that test step, so the later cleanup and Archive artifacts logs steps at lines 185–218 cannot run; the archive guard also explicitly excludes cancellation. The same pattern exists across the TAP workflows changed here, so the motivating timeout scenario still produces no diagnostic logs. Put a shorter timeout/watchdog on the test step while retaining a longer job timeout so failure handling has time to execute.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci-3p-aiomysql.yml:
- Line 26: Adjust the job timeout strategy so cleanup steps remain runnable
after tests time out: apply the 120-minute timeout to the test/run step, or make
job-level cancellation cleanup and artifact-upload steps cancellation-safe with
bounded execution. Make this change in .github/workflows/ci-3p-aiomysql.yml:26,
.github/workflows/ci-3p-django-framework.yml:26,
.github/workflows/ci-3p-laravel-framework.yml:26,
.github/workflows/ci-3p-mariadb-connector-c.yml:26,
.github/workflows/ci-3p-mysql-connector-j.yml:26,
.github/workflows/ci-3p-pgjdbc.yml:26,
.github/workflows/ci-3p-php-pdo-mysql.yml:26,
.github/workflows/ci-3p-php-pdo-pgsql.yml:26,
.github/workflows/ci-3p-postgresql.yml:26,
.github/workflows/ci-3p-sqlalchemy.yml:27, and
.github/workflows/ci-unittests.yml:16; preserve baseline checking, artifact
uploads, and Run unit-tests-g1 permission repair in the reported logs.
In @.github/workflows/ci-basictests.yml:
- Line 84: Update the artifact-preservation conditions to run under always()
when a job fails or is cancelled, replacing failure() && !cancelled() so
permission fixes, chmod/copy, and uploads still execute after timeouts. Apply
the permission-fix and log/archive condition changes in
.github/workflows/ci-basictests.yml (84-84),
.github/workflows/ci-legacy-clickhouse-g1.yml (16-16),
.github/workflows/ci-legacy-g1.yml (84-84),
.github/workflows/ci-legacy-g2-genai.yml (16-16),
.github/workflows/ci-pgsql-socket-g1.yml (16-16),
.github/workflows/ci-set_parser_algorithm_3-g1.yml (16-16), and
.github/workflows/ci-taptests-pgsql-cluster.yml (63-63); update the build
archive upload condition in .github/workflows/ci-builds.yml (54-54) and the
permission-fix and artifact archive conditions in
.github/workflows/ci-selftests.yml (16-16). Verify one deliberately timed-out
job per workflow family.
In @.github/workflows/ci-legacy-g2.yml:
- Line 84: Update the post-test artifact permission-fixing and upload conditions
to use a cancellation-safe policy such as failure() || cancelled(), ensuring
logs are preserved when the 120-minute timeout cancels a job. Apply this
consistently in .github/workflows/ci-legacy-g2.yml:84-84,
ci-legacy-g3.yml:84-84, ci-legacy-g4.yml:84-84, ci-legacy-g5.yml:84-84,
ci-legacy-g6.yml:84-84, ci-mysql84-g7.yml:16-16, ci-mysql84-g8.yml:16-16,
ci-mysql84-g9.yml:16-16, ci-mysql84-gr-g1.yml:16-16, ci-mysql84-gr-g2.yml:16-16,
and ci-mysql84-gr-g3.yml:16-16.
In @.github/workflows/ci-legacy-g7.yml:
- Line 84: Keep the existing job-level 120-minute timeout, and add a shorter
step-level timeout to the long-running step so cleanup/archive steps can run
afterward: update Run legacy-g7 tests in
.github/workflows/ci-legacy-g7.yml:84-84, Run legacy-g8 tests in
.github/workflows/ci-legacy-g8.yml:16-16, Run legacy-g9 tests in
.github/workflows/ci-legacy-g9.yml:16-16, Make-test in
.github/workflows/ci-maketest.yml:18-18, the tests step in
.github/workflows/ci-mariadb10-galera-g1.yml:16-16, and the tests steps in
.github/workflows/ci-mysql84-gr-g4.yml:16-16, ci-mysql84-gr-g5.yml:16-16,
ci-mysql84-gr-g6.yml:16-16, ci-mysql84-gr-g7.yml:16-16,
ci-mysql84-gr-g8.yml:16-16, and ci-mysql84-gr-g9.yml:16-16.
In @.github/workflows/ci-mariadb10-galera-g2.yml:
- Line 16: Update the affected archive/log upload steps that currently use
failure() && !cancelled() so their conditions use always(), preserving uploads
when jobs time out. Apply this in
.github/workflows/ci-mariadb10-galera-g2.yml:16, ci-mariadb10-galera-g3.yml:16,
ci-mariadb10-galera-g4.yml:16, ci-mariadb10-galera-g5.yml:16,
ci-mariadb10-galera-g6.yml:16, ci-mysql90-gr-g1.yml:16, ci-mysql93-gr-g1.yml:16,
ci-mysql95-gr-g1.yml:16, and both affected locations in ci-mysqlx.yml:38 and
ci-mysqlx.yml:512.
In @.github/workflows/ci-trigger.yml:
- Line 10: The ci-trigger workflow’s timeout-minutes must exceed the 120-minute
limit of the watched ci-builds job. Increase the watcher timeout in the workflow
configuration to provide completion-event processing margin while preserving the
existing watch behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 5226e9d7-ab40-49f5-85be-463616cd7a38
📒 Files selected for processing (64)
.github/workflows/ci-3p-aiomysql.yml.github/workflows/ci-3p-django-framework.yml.github/workflows/ci-3p-laravel-framework.yml.github/workflows/ci-3p-mariadb-connector-c.yml.github/workflows/ci-3p-mysql-connector-j.yml.github/workflows/ci-3p-pgjdbc.yml.github/workflows/ci-3p-php-pdo-mysql.yml.github/workflows/ci-3p-php-pdo-pgsql.yml.github/workflows/ci-3p-postgresql.yml.github/workflows/ci-3p-sqlalchemy.yml.github/workflows/ci-basictests.yml.github/workflows/ci-builds.yml.github/workflows/ci-codeql.yml.github/workflows/ci-legacy-clickhouse-g1.yml.github/workflows/ci-legacy-g1.yml.github/workflows/ci-legacy-g2-genai.yml.github/workflows/ci-legacy-g2.yml.github/workflows/ci-legacy-g3.yml.github/workflows/ci-legacy-g4.yml.github/workflows/ci-legacy-g5.yml.github/workflows/ci-legacy-g6.yml.github/workflows/ci-legacy-g7.yml.github/workflows/ci-legacy-g8.yml.github/workflows/ci-legacy-g9.yml.github/workflows/ci-maketest.yml.github/workflows/ci-mariadb10-galera-g1.yml.github/workflows/ci-mariadb10-galera-g2.yml.github/workflows/ci-mariadb10-galera-g3.yml.github/workflows/ci-mariadb10-galera-g4.yml.github/workflows/ci-mariadb10-galera-g5.yml.github/workflows/ci-mariadb10-galera-g6.yml.github/workflows/ci-mariadb10-galera-g7.yml.github/workflows/ci-mariadb10-galera-g8.yml.github/workflows/ci-mariadb10-galera-g9.yml.github/workflows/ci-mysql56-single-g1.yml.github/workflows/ci-mysql84-g1.yml.github/workflows/ci-mysql84-g2.yml.github/workflows/ci-mysql84-g3.yml.github/workflows/ci-mysql84-g4.yml.github/workflows/ci-mysql84-g5.yml.github/workflows/ci-mysql84-g6.yml.github/workflows/ci-mysql84-g7.yml.github/workflows/ci-mysql84-g8.yml.github/workflows/ci-mysql84-g9.yml.github/workflows/ci-mysql84-gr-g1.yml.github/workflows/ci-mysql84-gr-g2.yml.github/workflows/ci-mysql84-gr-g3.yml.github/workflows/ci-mysql84-gr-g4.yml.github/workflows/ci-mysql84-gr-g5.yml.github/workflows/ci-mysql84-gr-g6.yml.github/workflows/ci-mysql84-gr-g7.yml.github/workflows/ci-mysql84-gr-g8.yml.github/workflows/ci-mysql84-gr-g9.yml.github/workflows/ci-mysql90-gr-g1.yml.github/workflows/ci-mysql93-gr-g1.yml.github/workflows/ci-mysql95-gr-g1.yml.github/workflows/ci-mysqlx.yml.github/workflows/ci-package-build.yml.github/workflows/ci-pgsql-socket-g1.yml.github/workflows/ci-selftests.yml.github/workflows/ci-set_parser_algorithm_3-g1.yml.github/workflows/ci-taptests-pgsql-cluster.yml.github/workflows/ci-trigger.yml.github/workflows/ci-unittests.yml
📜 Review details
🔇 Additional comments (18)
.github/workflows/ci-3p-sqlalchemy.yml (1)
209-209: LGTM!.github/workflows/ci-basictests.yml (1)
28-28: LGTM!.github/workflows/ci-codeql.yml (1)
17-17: LGTM!.github/workflows/ci-legacy-g1.yml (1)
28-28: LGTM!.github/workflows/ci-taptests-pgsql-cluster.yml (1)
16-16: LGTM!.github/workflows/ci-legacy-g2.yml (1)
28-28: LGTM!.github/workflows/ci-legacy-g3.yml (1)
28-28: LGTM!.github/workflows/ci-legacy-g4.yml (1)
28-28: LGTM!.github/workflows/ci-legacy-g5.yml (1)
28-28: LGTM!.github/workflows/ci-legacy-g6.yml (1)
28-28: LGTM!.github/workflows/ci-legacy-g7.yml (1)
28-28: LGTM!.github/workflows/ci-mysqlx.yml (1)
238-238: LGTM!.github/workflows/ci-package-build.yml (1)
18-18: LGTM!Also applies to: 57-57, 113-113, 169-169
.github/workflows/ci-mysql84-g1.yml (2)
28-28: LGTM!
84-84: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUpdate post-timeout conditions before merging.
timeout-minutes: 120cancels thetestsjobs on timeout, but the artifact cleanup steps after those jobs still usefailure() && !cancelled()or!cancelled(). Timed-out jobs reachcancelled(), so these post-test permission-fix, log archive, coverage archive, coverage report, and Codecov steps skip archive collection. Usealways()oralways() && (failure() || cancelled())for the intended post-cancellation steps.Proposed condition pattern
- if: ${{ failure() && !cancelled() }} + if: ${{ always() && (failure() || cancelled()) }} - if: ${{ !cancelled() }} + if: ${{ always() }} - if: ${{ !cancelled() && hashFiles(...) != '' }} + if: ${{ always() && hashFiles(...) != '' }}Apply the first pattern to permission fixing and log archival. Apply the second or third pattern to coverage and Codecov steps.
> Likely an incorrect or invalid review comment..github/workflows/ci-mysql84-g2.yml (1)
28-28: LGTM!.github/workflows/ci-mysql84-g3.yml (1)
28-28: LGTM!.github/workflows/ci-mysql84-g4.yml (1)
28-28: LGTM!
| tests: | ||
| needs: pick-runner | ||
| runs-on: ${{ fromJson(needs.pick-runner.outputs.runson) }} | ||
| timeout-minutes: 120 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## workflows found"
git ls-files .github/workflows | sort
echo
echo "## relevant timeout/!cancelled/always/artifact lines"
for f in $(git ls-files .github/workflows/{ci-basictests,ci-builds,ci-legacy-clickhouse-g1,ci-legacy-g1,ci-legacy-g2-genai,ci-pgsql-socket-g1,ci-selftests,ci-set_parser_algorithm_3-g1,ci-taptests-pgsql-cluster}.yml); do
echo "===== $f"
wc -l "$f"
sed -n '1,140p' "$f" | nl -ba | grep -En 'timeout-minutes:|!cancelled\(|always\(|upload-artifact|tar|zip|chmod|archive|logs|bin' || true
done
echo
echo "## diff stat (if available)"
git diff --stat || trueRepository: sysown/proxysql
Length of output: 4021
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
files = [
".github/workflows/ci-basictests.yml",
".github/workflows/ci-builds.yml",
".github/workflows/ci-legacy-clickhouse-g1.yml",
".github/workflows/ci-legacy-g1.yml",
".github/workflows/ci-legacy-g2-genai.yml",
".github/workflows/ci-pgsql-socket-g1.yml",
".github/workflows/ci-selftests.yml",
".github/workflows/ci-set_parser_algorithm_3-g1.yml",
".github/workflows/ci-taptests-pgsql-cluster.yml",
]
for f in files:
p=Path(f)
if not p.exists():
print("MISSING", f)
continue
text=p.read_text()
lines=text.splitlines()
print(f"===== {f}")
for i,line in enumerate(lines,1):
if "timeout-minutes:" in line or "cancelled()" in line or "always()" in line:
print(f"{i}: {line}")
print()
PYRepository: sysown/proxysql
Length of output: 4889
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
files = [
".github/workflows/ci-basictests.yml",
".github/workflows/ci-builds.yml",
".github/workflows/ci-legacy-clickhouse-g1.yml",
".github/workflows/ci-legacy-g1.yml",
".github/workflows/ci-legacy-g2-genai.yml",
".github/workflows/ci-pgsql-socket-g1.yml",
".github/workflows/ci-selftests.yml",
".github/workflows/ci-set_parser_algorithm_3-g1.yml",
".github/workflows/ci-taptests-pgsql-cluster.yml",
]
for f in files:
lines = Path(f).read_text().splitlines()
jobs = {}
job_start, cur = None, None
for n, line in enumerate(lines, 1):
if re.match(r"^ [A-Za-z0-9_-]+:$", line):
if cur and cur is not None and cur.start is not None:
jobs[cur.name] = cur
job_start = n
cur = type("Job", (), {"name": line.strip().rstrip(":").lstrip(" "), "start": n, "lines": []})
if cur and n >= job_start:
cur.lines.append((n, line))
if cur:
jobs[cur.name] = cur
print(f"===== {f}")
for jname, job in jobs.items():
job_lines = "\n".join(line for _, line in job.lines)
if "timeout-minutes:" in job_lines or "cancelled()" in job_lines or "always()" in job_lines or "upload-artifact" in job_lines or "tar " in job_lines:
print("--- job", jname)
for n, line in job.lines:
if any(tok in line for tok in ["timeout-minutes:", "if:", "upload-artifact", "tar ", "chmod", "artifact", "logs", "coverage", "bin"]):
print(f"{n}: {line")
PYRepository: sysown/proxysql
Length of output: 266
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
files = [
".github/workflows/ci-basictests.yml",
".github/workflows/ci-builds.yml",
".github/workflows/ci-legacy-clickhouse-g1.yml",
".github/workflows/ci-legacy-g1.yml",
".github/workflows/ci-legacy-g2-genai.yml",
".github/workflows/ci-pgsql-socket-g1.yml",
".github/workflows/ci-selftests.yml",
".github/workflows/ci-set_parser_algorithm_3-g1.yml",
".github/workflows/ci-taptests-pgsql-cluster.yml",
]
for f in files:
lines = Path(f).read_text().splitlines()
jobs = {}
job_start, cur = None, None
for n, line in enumerate(lines, 1):
if re.match(r"^ [A-Za-z0-9_-]+:$", line):
if cur:
jobs[cur.name] = cur
cur = type("Job", (), {"name": line.strip().rstrip(":").lstrip(" "), "start": n, "lines": []})
if cur and n >= job_start:
cur.lines.append((n, line))
if cur:
jobs[cur.name] = cur
print(f"===== {f}")
for jname, job in jobs.items():
job_lines = "\n".join(line for _, line in job.lines)
if "timeout-minutes:" in job_lines or "cancelled()" in job_lines or "always()" in job_lines or "upload-artifact" in job_lines or "tar " in job_lines:
print("--- job", jname)
for n, line in job.lines:
if any(tok in line for tok in ["timeout-minutes:", "if:", "upload-artifact", "tar ", "chmod", "artifact", "logs", "coverage", "bin"]):
print(f"{n}: {line}")
PYRepository: sysown/proxysql
Length of output: 299
🏁 Script executed:
#!/bin/bash
set -euo pipefail
for f in .github/workflows/ci-basictests.yml .github/workflows/ci-builds.yml .github/workflows/ci-legacy-clickhouse-g1.yml .github/workflows/ci-legacy-g1.yml .github/workflows/ci-legacy-g2-genai.yml .github/workflows/ci-pgsql-socket-g1.yml .github/workflows/ci-selftests.yml .github/workflows/ci-set_parser_algorithm_3-g1.yml .github/workflows/ci-taptests-pgsql-cluster.yml; do
echo "===== $f"
grep -nE 'timeout-minutes:|upload-artifact:|tar |chmod|if:.*(always|!cancelled|failure)|coverage-report|logs' "$f" || true
doneRepository: sysown/proxysql
Length of output: 11188
Run artifact preservation after timeout cancellation.
timeout-minutes: 120 cancels jobs. Several preserved steps now use failure() && !cancelled(), which skips permission fixing, chmod/copy, and uploads when a job is cancelled. Run these steps on failure() || cancelled() under always() handling, then verify one deliberately timed-out job per workflow family.
.github/workflows/ci-basictests.yml#L84: update the permission-fix and log archive conditions..github/workflows/ci-builds.yml#L54: update the build archive upload condition..github/workflows/ci-legacy-clickhouse-g1.yml#L16: update the permission-fix and log archive conditions..github/workflows/ci-legacy-g1.yml#L84: update the permission-fix and log archive conditions..github/workflows/ci-legacy-g2-genai.yml#L16: update the permission-fix and log archive conditions..github/workflows/ci-pgsql-socket-g1.yml#L16: update the permission-fix and log archive conditions..github/workflows/ci-selftests.yml#L16: update the permission-fix and artifact archive conditions..github/workflows/ci-set_parser_algorithm_3-g1.yml#L16: update the permission-fix and log archive conditions..github/workflows/ci-taptests-pgsql-cluster.yml#L63: update the permission-fix and log archive conditions.
📍 Affects 9 files
.github/workflows/ci-basictests.yml#L84-L84(this comment).github/workflows/ci-builds.yml#L54-L54.github/workflows/ci-legacy-clickhouse-g1.yml#L16-L16.github/workflows/ci-legacy-g1.yml#L84-L84.github/workflows/ci-legacy-g2-genai.yml#L16-L16.github/workflows/ci-pgsql-socket-g1.yml#L16-L16.github/workflows/ci-selftests.yml#L16-L16.github/workflows/ci-set_parser_algorithm_3-g1.yml#L16-L16.github/workflows/ci-taptests-pgsql-cluster.yml#L63-L63
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-basictests.yml at line 84, Update the
artifact-preservation conditions to run under always() when a job fails or is
cancelled, replacing failure() && !cancelled() so permission fixes, chmod/copy,
and uploads still execute after timeouts. Apply the permission-fix and
log/archive condition changes in .github/workflows/ci-basictests.yml (84-84),
.github/workflows/ci-legacy-clickhouse-g1.yml (16-16),
.github/workflows/ci-legacy-g1.yml (84-84),
.github/workflows/ci-legacy-g2-genai.yml (16-16),
.github/workflows/ci-pgsql-socket-g1.yml (16-16),
.github/workflows/ci-set_parser_algorithm_3-g1.yml (16-16), and
.github/workflows/ci-taptests-pgsql-cluster.yml (63-63); update the build
archive upload condition in .github/workflows/ci-builds.yml (54-54) and the
permission-fix and artifact archive conditions in
.github/workflows/ci-selftests.yml (16-16). Verify one deliberately timed-out
job per workflow family.
| tests: | ||
| needs: pick-runner | ||
| runs-on: ${{ fromJson(needs.pick-runner.outputs.runson) }} | ||
| timeout-minutes: 120 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow files =="
git ls-files .github/workflows | grep -E '(^|/)(ci-legacy-g[2-6]|ci-mysql84-g[7-9]|ci-mysql84-gr-g[1-3])\.yml$' | sort
echo
echo "== timeout locations with file context =="
for f in $(git ls-files .github/workflows | grep -E '(^|/)(ci-legacy-g[2-6]|ci-mysql84-g[7-9]|ci-mysql84-gr-g[1-3])\.yml$' | sort); do
echo "--- $f ---"
nl -ba "$f" | sed -n '1,130p'
done
echo
echo "== relevant step conditions in these workflows =="
rg -n "timeout-minutes|fix|permission|archive|upload|logs|if:|cancelled|failure|always" .github/workflows/ci-legacy-g[2-6].yml .github/workflows/ci-mysql84-g[7-9].yml .github/workflows/ci-mysql84-gr-g[1-3].yml || trueRepository: sysown/proxysql
Length of output: 701
🏁 Script executed:
#!/bin/bash
set -euo pipefail
for f in .github/workflows/ci-legacy-g2.yml .github/workflows/ci-legacy-g3.yml .github/workflows/ci-legacy-g4.yml .github/workflows/ci-legacy-g5.yml .github/workflows/ci-legacy-g6.yml \
.github/workflows/ci-mysql84-g7.yml .github/workflows/ci-mysql84-g8.yml .github/workflows/ci-mysql84-g9.yml \
.github/workflows/ci-mysql84-gr-g1.yml .github/workflows/ci-mysql84-gr-g2.yml .github/workflows/ci-mysql84-gr-g3.yml; do
echo "--- $f ---"
lines="$(wc -l < "$f")"
limit=140
if [ "$lines" -gt "$limit" ]; then lines=$limit; fi
awk '{printf "%4d\t%s\n", NR, $0}' "$f" | sed -n "1,${lines}p"
done
echo "== relevant strings =="
for f in \
.github/workflows/ci-legacy-g2.yml .github/workflows/ci-legacy-g3.yml .github/workflows/ci-legacy-g4.yml .github/workflows/ci-legacy-g5.yml .github/workflows/ci-legacy-g6.yml \
.github/workflows/ci-mysql84-g7.yml .github/workflows/ci-mysql84-g8.yml .github/workflows/ci-mysql84-g9.yml \
.github/workflows/ci-mysql84-gr-g1.yml .github/workflows/ci-mysql84-gr-g2.yml .github/workflows/ci-mysql84-gr-g3.yml
do
echo "--- $f ---"
grep -nE 'timeout-minutes:|^ tests:|fix|permission|archive|upload|logs|if:|cancelled|failure|always' "$f" || true
doneRepository: sysown/proxysql
Length of output: 50372
🌐 Web query:
GitHub Actions timeout-minutes documentation job cancelled automatically cancellation failure() if condition documentation
💡 Result:
In GitHub Actions, the timeout-minutes property can be applied at both the job level and the step level to prevent workflows from running indefinitely [1]. When a job or step exceeds its defined timeout-minutes, GitHub Actions automatically cancels it [2][1]. Key details regarding this behavior include: Timeout Logic: The timeout-minutes countdown for a job begins as soon as the job starts, which includes time spent in the queue or on setup overhead, not just the duration of individual steps [3]. Cancellation Status: When a job is automatically cancelled due to a timeout, its final status is marked as "cancelled" rather than "failure" [4]. Conditional Execution: Because the job status is "cancelled", the default success() condition (which is implicit if no if condition is provided) will evaluate to false [4][5]. Consequently, if you have a step with an if: failure() condition, it will not execute because the job was cancelled, not failed [5][6]. Handling Cancellation: If you need a step to run specifically when a job is cancelled (e.g., for cleanup or logging), you must use the cancelled() status check function [4][5]. For example, if: ${{ cancelled() }} will allow a step to run even if the job was terminated due to a timeout [5]. To run a step in either a failure or cancellation scenario, you can use if: ${{ failure() || cancelled() }} [6]. Note that for GitHub-hosted runners, there is a hard limit of 6 hours (360 minutes) for job execution time [7][8]. If a configured timeout-minutes exceeds this limit, the job will still be cancelled when the 6-hour limit is reached [2].
Citations:
- 1: https://notes.kodekloud.com/docs/GitHub-Actions/GitHub-Actions-Core-Concepts/Timeout-for-Jobs-and-Steps/page
- 2: https://github.com/github/docs/blob/main/content/actions/reference/workflows-and-actions/workflow-syntax.md
- 3: timeout-minutes is not getting respected actions/runner#3699
- 4: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-cancellation
- 5: https://docs.github.com/actions/reference/evaluate-expressions-in-workflows-and-actions
- 6: https://gist.github.com/kf6kjg/31b80cf0f8c9997ddc7fe7335c580412
- 7: https://docs.github.com/en/actions/reference/limits
- 8: jobs.<job_id>.timeout-minutes description is misleading github/docs#7984
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== all affected workflow contents in compact form =="
find .github/workflows \
\( -name 'ci-legacy-g[2-6].yml' -o \
-name 'ci-mysql84-g[7-9].yml' -o \
-name 'ci-mysql84-gr-g[1-3].yml' \) \
-print -print0 | sort -z | xargs -0 sh -c '
for f; do
echo "--- $f ---"
python3 - <<"PY"
import pathlib, re
p=pathlib.Path("$0")
print("lines:", p.stat().st_size, "bytes")
for i,line in enumerate(p.read_text().splitlines(),1):
if ("timeout-minutes" in line
or "upload" in line
or "archive" in line
or "fix" in line
or "log" in line
or "permission" in line
or "if:" in line
or line.startswith(" tests:")
or "name:" in line):
print(f"{i:04d}\t{line}")
PY
done
' shell
echo
echo "== targeted step-condition scan across affected workflows =="
rg -nH 'timeout-minutes:|upload-artifact|codecov|logs|permission|fix|if:\s*((failure|cancelled|always)\(\)|\${{\s*success|success\(\))' \
.github/workflows/ci-legacy-g[2-6].yml \
.github/workflows/ci-mysql84-g[7-9].yml \
.github/workflows/ci-mysql84-gr-g[1-3].yml || trueRepository: sysown/proxysql
Length of output: 3650
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== targeted step-condition scan across affected workflows =="
rg -nH 'timeout-minutes:|upload-artifact|codecov|logs|permission|fix|if:\s*((failure|cancelled|always)\(\)|\${{\s*success|success\(\))' \
.github/workflows/ci-legacy-g[2-6].yml \
.github/workflows/ci-mysql84-g[7-9].yml \
.github/workflows/ci-mysql84-gr-g[1-3].yml || true
echo
echo "== selected complete workflows: ci-legacy-g2, ci-mysql84-g7, ci-mysql84-gr-g1 =="
for f in \
.github/workflows/ci-legacy-g2.yml \
.github/workflows/ci-mysql84-g7.yml \
.github/workflows/ci-mysql84-gr-g1.yml
do
echo "--- $f ---"
awk '{printf "%4d\t%s\n", NR, $0}' "$f"
doneRepository: sysown/proxysql
Length of output: 45798
Make post-test artifact handling cancellation-safe before adding job timeouts.
Each affected tests job now sets timeout-minutes: 120, but timeout expiry runs with job.status: cancelled. The current post-test conditions do not include cancelled() as a possible status, so permission fixing/uploads can be skipped on a timeout-caused cancellation and the logs may not be preserved. Use a single cancellation-safe policy for these steps across all affected workflows, such as failure() || cancelled().
📍 Affects 11 files
.github/workflows/ci-legacy-g2.yml#L84-L84(this comment).github/workflows/ci-legacy-g3.yml#L84-L84.github/workflows/ci-legacy-g4.yml#L84-L84.github/workflows/ci-legacy-g5.yml#L84-L84.github/workflows/ci-legacy-g6.yml#L84-L84.github/workflows/ci-mysql84-g7.yml#L16-L16.github/workflows/ci-mysql84-g8.yml#L16-L16.github/workflows/ci-mysql84-g9.yml#L16-L16.github/workflows/ci-mysql84-gr-g1.yml#L16-L16.github/workflows/ci-mysql84-gr-g2.yml#L16-L16.github/workflows/ci-mysql84-gr-g3.yml#L16-L16
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-legacy-g2.yml at line 84, Update the post-test artifact
permission-fixing and upload conditions to use a cancellation-safe policy such
as failure() || cancelled(), ensuring logs are preserved when the 120-minute
timeout cancels a job. Apply this consistently in
.github/workflows/ci-legacy-g2.yml:84-84, ci-legacy-g3.yml:84-84,
ci-legacy-g4.yml:84-84, ci-legacy-g5.yml:84-84, ci-legacy-g6.yml:84-84,
ci-mysql84-g7.yml:16-16, ci-mysql84-g8.yml:16-16, ci-mysql84-g9.yml:16-16,
ci-mysql84-gr-g1.yml:16-16, ci-mysql84-gr-g2.yml:16-16, and
ci-mysql84-gr-g3.yml:16-16.
Source: MCP tools
| tests: | ||
| needs: pick-runner | ||
| runs-on: ${{ fromJson(needs.pick-runner.outputs.runson) }} | ||
| timeout-minutes: 120 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Locate workflow files:"
fd -p '.github/workflows/(ci-legacy-g[7-9]|ci-maketest|ci-mariadb10-galera-g1|ci-mysql84-gr-g[4-9])\.yml' . | sort || true
echo
echo "Workflow snippets around timeout and relevant steps:"
for f in \
.github/workflows/ci-legacy-g7.yml \
.github/workflows/ci-legacy-g8.yml \
.github/workflows/ci-legacy-g9.yml \
.github/workflows/ci-maketest.yml \
.github/workflows/ci-mariadb10-galera-g1.yml \
.github/workflows/ci-mysql84-gr-g4.yml \
.github/workflows/ci-mysql84-gr-g5.yml \
.github/workflows/ci-mysql84-gr-g6.yml \
.github/workflows/ci-mysql84-gr-g7.yml \
.github/workflows/ci-mysql84-gr-g8.yml \
.github/workflows/ci-mysql84-gr-g9.yml
do
if [ -f "$f" ]; then
echo "===== $f ====="
nl -ba "$f" | sed -n '1,140p'
fi
done
echo
echo "Search archive steps and cancellation conditions in workflows:"
rg -n "uses: .*github-archive|upload-artifact|!cancelled|cancelled\\(|failure\\(" .github/workflows/ci-legacy-g7.yml .github/workflows/ci-legacy-g8.yml .github/workflows/ci-legacy-g9.yml .github/workflows/ci-maketest.yml .github/workflows/ci-mariadb10-galera-g1.yml .github/workflows/ci-mysql84-gr-g4.yml .github/workflows/ci-mysql84-gr-g5.yml .github/workflows/ci-mysql84-gr-g6.yml .github/workflows/ci-mysql84-gr-g7.yml .github/workflows/ci-mysql84-gr-g8.yml .github/workflows/ci-mysql84-gr-g9.yml || true
echo
echo "GitHub Actions release docs availability via public docs:"
curl -fsSL https://github.com/actions/runner/releases/latest/download/actions-runner-linux-x64.tar.gz -o /tmp/actions-runner-check.tar.gz 2>/dev/null || true
file /tmp/actions-runner-check.tar.gz || trueRepository: sysown/proxysql
Length of output: 318
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Locate workflow files:"
find .github/workflows -maxdepth 1 -type f \( -name 'ci-legacy-g[7-9].yml' -o -name 'ci-maketest.yml' -o -name 'ci-mariadb10-galera-g1.yml' -o -name 'ci-mysql*gr-g[4-9].yml' \) | sort
files=(
.github/workflows/ci-legacy-g7.yml
.github/workflows/ci-legacy-g8.yml
.github/workflows/ci-legacy-g9.yml
.github/workflows/ci-maketest.yml
.github/workflows/ci-mariadb10-galera-g1.yml
.github/workflows/ci-mysql84-gr-g4.yml
.github/workflows/ci-mysql84-fr-g5.yml
.github/workflows/ci-mysql84-gr-g5.yml
.github/workflows/ci-mysql84-gr-g6.yml
.github/workflows/ci-mysql84-gr-g7.yml
.github/workflows/ci-mysql84-gr-g8.yml
.github/workflows/ci-mysql84-gr-g9.yml
)
echo
echo "Workflow snippets around timeout and archive steps:"
for f in "${files[@]}"; do
if [ -f "$f" ]; then
echo "===== $f ====="
awk '{printf "%5d: %s\n", NR, $0}' "$f" | sed -n '1,140p'
else
echo "===== missing: $f ====="
fi
doneRepository: sysown/proxysql
Length of output: 50371
🏁 Script executed:
#!/bin/bash
set -euo pipefail
files=(
.github/workflows/ci-legacy-g7.yml
.github/workflows/ci-legacy-g8.yml
.github/workflows/ci-legacy-g9.yml
.github/workflows/ci-maketest.yml
.github/workflows/ci-mariadb10-galera-g1.yml
.github/workflows/ci-mysql84-gr-g4.yml
.github/workflows/ci-mysql84-gr-g5.yml
.github/workflows/ci-mysql84-gr-g6.yml
.github/workflows/ci-mysql84-gr-g7.yml
.github/workflows/ci-mysql84-gr-g8.yml
.github/workflows/ci-mysql84-gr-g9.yml
)
echo "Jobs and timeout-minutes per workflow:"
python3 - <<'PY'
from pathlib import Path
for path in sorted(Path('.github/workflows').glob('ci-*.yml')):
txt = path.read_text()
if any(x in path.name for x in ['ci-legacy-g7','ci-legacy-g8','ci-legacy-g9','ci-maketest','ci-mariadb10-galera-g1','ci-mysql84-gr-g4','ci-mysql84-gr-g5','ci-mysql84-gr-g6','ci-mysql84-gr-g7','ci-mysql84-gr-g8','ci-mysql84-gr-g9']):
state='job'
indent=None
print(f'===== {path} =====')
for i,line in enumerate(txt.splitlines(),1):
if line.strip().isidentifier() and line.rstrip().endswith(':'):
state='job'; indent=None
print(f"{i}: job {line.strip()}: timeout={line.find('timeout-minutes')}")
if 'timeout-minutes:' in line:
cur_indent = len(line) - len(line.lstrip())
indent = state
if indent != 'job': print(f"{i}: {state} at indent {indent} timeout={idx if False else line.replace(' ','')[:35]}")
PY
echo
echo "Steps / timeouts / upload-artifact conditions in targeted workflows:"
awk '
/^[A-Za-z0-9_-]+:$/ { job=$1; in_job=1; in_step=0; step_name=""; step_indent="" }
/^[[:space:]]+name:[[:space:]]/ {
step_indent=$0; sub(/^[:space:]].*/, "", step_indent); step_name=$0; sub(/^[[:space:]]*name:[[:space:]]+/, "", step_name); in_step=1
}
/^[[:space:]]+timeout-minutes:/ { print FILENAME ":L" NR ": step=" step_name " timeout=" $0 }
/^[[:space:]]+uses:[[:space:]]+actions/upload-artifact@/ { in_step=1; artifact=1 }
/^[[:space:]]+if:/ && artifact { print FILENAME ":L" NR ": artifact condition=" $0; artifact=0 }
/^[[:space:]]+continue-on-error:/ { cont=$0; print FILENAME ":L" NR ": cont=" $0 }
' "${files[@]}"
echo
echo "Longest run block names before first non-run artifact/clean/always step:"
python3 - <<'PY'
from pathlib import Path
for path in [Path(p) for p in files]:
txt=path.read_text().splitlines()
current=None
started=False
longest=[]
lines=[]
for i,line in enumerate(txt,1):
if 'timeout-minutes:' in line:
print(f'{path}:L{i}: timeout-minutes={line.strip()}')
if line.strip().startswith('- name:'):
name=line.strip()[7:].strip()
if current:
s='\n'.join(lines)
if any(s.find(x)>=0 for x in ['Run ', 'Make-test', 'Download build handoff', 'Archive artifacts logs', 'Check build', 'Verify binary', 'GITHUB_OUTPUT', 'curl ', 'make ', 'cd proxysql']):
print(f'{path}: prior_current={current} lines={len(lines)}')
current=name; started=False; lines=[]
if name in ['Download build handoff','Make-test'] or name.endswith(' tests') or name.startswith('Run '):
started=True
elif started and current:
lines.append(line)
if line.strip().startswith('- name:') or line.strip().startswith('- uses:'):
pass
if current:
s='\n'.join(lines)
if any(s.find(x)>=0 for x in ['Run ', 'Make-test', 'Download build handoff', 'Archive artifacts logs', 'Check build', 'Verify binary', 'GITHUB_OUTPUT', 'curl ', 'make ', 'cd proxysql']):
print(f'{path}: last_current={current} lines={len(lines)}')
PYRepository: sysown/proxysql
Length of output: 1203
🏁 Script executed:
#!/bin/bash
set -euo pipefail
files=(
.github/workflows/ci-legacy-g7.yml
.github/workflows/ci-legacy-g8.yml
.github/workflows/ci-legacy-g9.yml
.github/workflows/ci-maketest.yml
.github/workflows/ci-mariadb10-galera-g1.yml
.github/workflows/ci-mysql84-gr-g4.yml
.github/workflows/ci-mysql84-gr-g5.yml
.github/workflows/ci-mysql84-gr-g6.yml
.github/workflows/ci-mysql84-gr-g7.yml
.github/workflows/ci-mysql84-gr-g8.yml
.github/workflows/ci-mysql84-gr-g9.yml
)
echo "Targeted timeout and artifact-step locations:"
python3 - <<'PY'
from pathlib import Path
paths=[Path(p) for p in '''
.github/workflows/ci-legacy-g7.yml
.github/workflows/ci-legacy-g8.yml
.github/workflows/ci-legacy-g9.yml
.github/workflows/ci-maketest.yml
.github/workflows/ci-mariadb10-galera-g1.yml
.github/workflows/ci-mysql84-gr-g4.yml
.github/workflows/ci-mysql84-gr-g5.yml
.github/workflows/ci-mysql84-gr-g6.yml
.github/workflows/ci-mysql84-gr-g7.yml
.github/workflows/ci-mysql84-gr-g8.yml
.github/workflows/ci-mysql84-gr-g9.yml
'''.split()]
for path in paths:
lines=path.read_text().splitlines()
print(f'===== {path} =====')
seen=None
in_archive=None
for i,line in enumerate(lines,1):
if line.rstrip().endswith(':') and line.lstrip()[0].isupper():
seen=line.strip().rstrip(':')
in_archive=None
if 'timeout-minutes:' in line:
print(f'L{i}: seen={seen} timeout={line.strip()}')
if 'uses: actions/upload-artifact@' in line or 'upload-artifact@v4' in line:
in_archive=i
if in_archive and 'if:' in line:
print(f'L{i}: archive if={line.strip()}')
in_archive=None
PY
echo
echo "Relevant step names and conditions:"
python3 - <<'PY'
from pathlib import Path
paths=[Path(p) for p in '''
.github/workflows/ci-legacy-g7.yml
.github/workflows/ci-legacy-g8.yml
.github/workflows/ci-legacy-g9.yml
.github/workflows/ci-maketest.yml
.github/workflows/ci-mariadb10-galera-g1.yml
.github/workflows/ci-mysql84-gr-g4.yml
.github/workflows/ci-mysql84-gr-g5.yml
.github/workflows/ci-mysql84-gr-g6.yml
.github/workflows/ci-mysql84-gr-g7.yml
.github/workflows/ci-mysql84_gr-g8.yml
.github/workflows/ci-mysql84-gr-g8.yml
.github/workflows/ci-mysql84-gr-g9.yml
'''.split()]
for path in paths:
lines=path.read_text().splitlines()
if not path.exists():
print(f'MISSING {path}')
continue
step=None
for i,line in enumerate(lines,1):
if line.strip().startswith('- name:') or line.strip().startswith('- uses:'):
step=line.strip()[3:].strip()
if (step and any(x in step for x in ['Run legacy', 'Make-test', 'Archive artifacts logs'] or False)) \
or 'cancelled()' in line or 'failure()' in line or 'timeout-minutes:' in line:
print(f'{path}:L{i}:{line}')
PY
echo
echo "Read ci-legacy-g7 relevant body around long-running step:"
python3 - <<'PY'
from pathlib import Path
lines=Path('.github/workflows/ci-legacy-g7.yml').read_text().splitlines()
for i,line in enumerate(lines,1):
if i >= 130 and i <= 290:
print(f'{i}: {line}')
PYRepository: sysown/proxysql
Length of output: 16933
🌐 Web query:
GitHub Actions workflow syntax timeout-minutes job cancels workflow run
💡 Result:
In GitHub Actions, the timeout-minutes property at the job level defines the maximum number of minutes a job is allowed to run before GitHub automatically cancels it [1][2]. Key behaviors regarding this timeout include: Timing Calculation: The timeout-minutes countdown begins when the job starts, not when the individual steps begin executing [3]. This includes time spent on setup, such as initializing the runner or performing internal GitHub Actions setup tasks [3]. If a runner takes time to pick up a job from the queue, that duration may also contribute to the elapsed time depending on the environment [3]. Automatic Cancellation and Failure: If a job exceeds the specified timeout-minutes value, GitHub cancels the job [4][1]. A job that is canceled due to a timeout is marked as failed [4]. Execution Limits: While you can set timeout-minutes for a job, it is still subject to the platform's hard execution limits [4][1]. For example, GitHub-hosted runners have a maximum execution time limit (e.g., 6 hours), and even if you set a higher timeout-minutes value, the job will be terminated if it hits the platform's execution limit [4][1]. Cancellation Process: When a job is canceled due to a timeout, GitHub attempts to shut down the processes gracefully by sending signals (SIGINT/SIGTERM) to the running steps, allowing for cleanup before the process tree is forcibly killed [5]. If you require more granular control, you can also set timeout-minutes at the individual step level within a job [3][2].
Citations:
- 1: jobs.<job_id>.timeout-minutes description is misleading github/docs#7984
- 2: https://stackoverflow.com/questions/72900262/general-timeout-for-github-actions-jobs
- 3: timeout-minutes is not getting respected actions/runner#3699
- 4: https://docs.github.com/en/actions/reference/limits
- 5: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-cancellation
Move the 120-minute timeout from the job to the long-running step.
These workflows apply timeout-minutes at job scope, so it counts all steps in the job and later steps can also consume part of the budget. Keep the outer job limit and add a step-level timeout to the build/test steps so cleanup/archive steps can still run when the long-running step hangs.
.github/workflows/ci-legacy-g7.yml#L84-L84: add a shorter timeout toRun legacy-g7 tests..github/workflows/ci-legacy-g8.yml#L16-L16: add a shorter timeout toRun legacy-g8 tests..github/workflows/ci-legacy-g9.yml#L16-L16: add a shorter timeout toRun legacy-g9 tests..github/workflows/ci-maketest.yml#L18-L18: add a shorter timeout toMake-test..github/workflows/ci-mariadb10-galera-g1.yml#L16-L16: add a shorter timeout to the tests step..github/workflows/ci-mysql84-gr-g4.ymlthroughci-mysql84-gr-g9.yml#L16-L16: add a shorter timeout to the tests step in each workflow.
📍 Affects 11 files
.github/workflows/ci-legacy-g7.yml#L84-L84(this comment).github/workflows/ci-legacy-g8.yml#L16-L16.github/workflows/ci-legacy-g9.yml#L16-L16.github/workflows/ci-maketest.yml#L18-L18.github/workflows/ci-mariadb10-galera-g1.yml#L16-L16.github/workflows/ci-mysql84-gr-g4.yml#L16-L16.github/workflows/ci-mysql84-gr-g5.yml#L16-L16.github/workflows/ci-mysql84-gr-g6.yml#L16-L16.github/workflows/ci-mysql84-gr-g7.yml#L16-L16.github/workflows/ci-mysql84-gr-g8.yml#L16-L16.github/workflows/ci-mysql84-gr-g9.yml#L16-L16
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-legacy-g7.yml at line 84, Keep the existing job-level
120-minute timeout, and add a shorter step-level timeout to the long-running
step so cleanup/archive steps can run afterward: update Run legacy-g7 tests in
.github/workflows/ci-legacy-g7.yml:84-84, Run legacy-g8 tests in
.github/workflows/ci-legacy-g8.yml:16-16, Run legacy-g9 tests in
.github/workflows/ci-legacy-g9.yml:16-16, Make-test in
.github/workflows/ci-maketest.yml:18-18, the tests step in
.github/workflows/ci-mariadb10-galera-g1.yml:16-16, and the tests steps in
.github/workflows/ci-mysql84-gr-g4.yml:16-16, ci-mysql84-gr-g5.yml:16-16,
ci-mysql84-gr-g6.yml:16-16, ci-mysql84-gr-g7.yml:16-16,
ci-mysql84-gr-g8.yml:16-16, and ci-mysql84-gr-g9.yml:16-16.
| jobs: | ||
| tests: | ||
| runs-on: ubuntu-22.04 | ||
| timeout-minutes: 120 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Candidate workflow files:\n'
git ls-files '.github/workflows/ci-*.yml' | sed -n '1,80p'
printf '\nRelevant workflow headers and archive step contexts:\n'
for f in \
.github/workflows/ci-mariadb10-galera-g2.yml \
.github/workflows/ci-mariadb10-galera-g3.yml \
.github/workflows/ci-mariadb10-galera-g4.yml \
.github/workflows/ci-mariadb10-galera-g5.yml \
.github/workflows/ci-mariadb10-galera-g6.yml \
.github/workflows/ci-mysql90-gr-g1.yml \
.github/workflows/ci-mysql93-gr-g1.yml \
.github/workflows/ci-mysql95-gr-g1.yml \
.github/workflows/ci-mysqlx.yml
do
echo "===== $f ====="
wc -l "$f"
sed -n '1,30p' "$f"
echo "--- archive/upload candidates ---"
rg -n "upload|archive|if:|failure\\(\\).*cancelled\\(\\)|cancelled\\(\\).*failure\\(\\)|timeout-minutes" "$f" | sed -n '1,140p'
done
printf '\nBehavioral probe of GitHub Actions condition semantics from docs-like textual search:\n'
rg -n "timeout-minutes|always\\(\\)|cancelled\\(\\)" .github/workflows README* . 2>/dev/null | sed -n '1,200p'Repository: sysown/proxysql
Length of output: 38760
🌐 Web query:
GitHub Actions workflow timeout-minutes cancels job always condition cancelled failure documentation
💡 Result:
When a GitHub Actions job reaches its timeout-minutes limit, GitHub automatically terminates and cancels the job [1][2]. This process results in a job conclusion of timed_out, which is distinct from a standard failure or cancelled conclusion [3][4][5]. Key behaviors regarding timeouts and cancellation: 1. Conclusion State: A job that hits its timeout limit is marked with the timed_out conclusion [5]. Note that while the runner treats this as a cancellation, standard conditional functions like failure() or cancelled() may not behave as expected because the specific conclusion is timed_out [6]. 2. Interaction with always() and !cancelled(): - Steps defined with if: always() will execute even if the job times out, because always() evaluates to true regardless of the job's conclusion [7][8]. - The !cancelled() condition is generally recommended over always() for most scenarios where you want to run cleanup tasks [8]. However, because a timed-out job has a timed_out conclusion, there is currently no direct built-in status check function (such as timed_out()) to specifically target this state [6]. 3. Cancellation vs. Timeout: When a job is manually cancelled, the server re-evaluates conditional expressions [7][9]. If a job or step uses if: always(), it may continue to run during a manual cancellation [7][8]. However, timeout-minutes is a hard limit enforced by the runner service [1][2]; once this limit is hit, the job is forcibly terminated [1][7][9]. 4. Practical Workarounds: Because there is no native timed_out() status function, developers often use environment variables or custom scripts to track elapsed time if they need to differentiate a timeout from other failure types [6]. Alternatively, if you need to perform actions after a job reaches its time limit, ensure those steps are marked with if: always() to ensure they trigger despite the timed_out conclusion [7][8].
Citations:
- 1: https://docs.github.com/en/actions/reference/limits
- 2: jobs.<job_id>.timeout-minutes description is misleading github/docs#7984
- 3: https://docs.github.com/en/rest/actions/workflow-jobs?apiVersion=2022-11-28
- 4: https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2026-03-10
- 5: feat: detect agent timeout and report it with frontmatter hint in failure issue/comment github/gh-aw#19307
- 6: https://stackoverflow.com/questions/78233438/github-action-cannot-get-timeout-status-from-previous-step
- 7: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-cancellation
- 8: Provide a way to force-cancel actions and improve cancellation feedback actions/runner#1846
- 9: https://docs.github.com/en/enterprise-server@3.19/actions/reference/workflows-and-actions/workflow-cancellation
Preserve log uploads when a job reaches the new timeout.
A job that reaches timeout-minutes: 120 is terminated as timed_out, so the current failure() && !cancelled() archive steps can skip uploads. Use a condition that still runs on timeout cancellation, such as always(), for the affected upload steps.
📍 Affects 9 files
.github/workflows/ci-mariadb10-galera-g2.yml#L16-L16(this comment).github/workflows/ci-mariadb10-galera-g3.yml#L16-L16.github/workflows/ci-mariadb10-galera-g4.yml#L16-L16.github/workflows/ci-mariadb10-galera-g5.yml#L16-L16.github/workflows/ci-mariadb10-galera-g6.yml#L16-L16.github/workflows/ci-mysql90-gr-g1.yml#L16-L16.github/workflows/ci-mysql93-gr-g1.yml#L16-L16.github/workflows/ci-mysql95-gr-g1.yml#L16-L16.github/workflows/ci-mysqlx.yml#L38-L38.github/workflows/ci-mysqlx.yml#L512-L512
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-mariadb10-galera-g2.yml at line 16, Update the affected
archive/log upload steps that currently use failure() && !cancelled() so their
conditions use always(), preserving uploads when jobs time out. Apply this in
.github/workflows/ci-mariadb10-galera-g2.yml:16, ci-mariadb10-galera-g3.yml:16,
ci-mariadb10-galera-g4.yml:16, ci-mariadb10-galera-g5.yml:16,
ci-mariadb10-galera-g6.yml:16, ci-mysql90-gr-g1.yml:16, ci-mysql93-gr-g1.yml:16,
ci-mysql95-gr-g1.yml:16, and both affected locations in ci-mysqlx.yml:38 and
ci-mysqlx.yml:512.
Review on the previous commit was right: a job-level deadline alone does not solve the motivating problem. When timeout-minutes fires at job scope GitHub terminates the job, so 'Fix artifact permissions' and 'Archive artifacts logs' -- guarded by 'failure() && !cancelled()' -- never run. That is the same artifact gap the 6-hour ceiling produced, just reached sooner. A step-level timeout fails only the step. job.status becomes failure, cancelled() stays false, and the archive/coverage steps run normally, so a hung group now yields logs instead of silence. Adds timeout-minutes: 90 to the 'Run <group> tests' step in 55 workflows, leaving the 120-minute job budget as the outer backstop with 30 minutes of headroom for infra setup, cleanup and artifact upload. 90 is far above any real group: the slowest observed successful group is CI-legacy-g4 at ~58 minutes, and the slowest single TAP test measured across 47 groups is reg_test_3765_ssl_pollout-t at 12.5 minutes. Also raises ci-trigger from 120 to 240. That job blocks on 'gh run watch' until ci-builds completes, and ci-builds is itself capped at 120 -- an equal budget let the watcher expire first, so the workflow_run[completed] fan-out would never fire. The archive step conditions are deliberately left as failure() && !cancelled(): with the step-level timeout they are now reached, and widening them to always()/cancelled() would also upload on genuine user cancellations.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/ci-legacy-g6.yml (1)
84-84: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winPreserve artifacts and coverage after these job-level timeouts expire.
These six jobs use
timeout-minutes: 120, but their post-test preservation steps still skip cancelled jobs. Update the artifact-permission, artifact-log-archive, coverage-artifact-upload, and coverage-upload conditions in each workflow to run on cancellation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci-legacy-g6.yml at line 84, Update the post-test preservation conditions for the artifact-permission, artifact-log-archive, coverage-artifact-upload, and coverage-upload steps so they execute when jobs are cancelled, not only when they complete normally. Apply the same condition changes in .github/workflows/ci-legacy-g6.yml:84-84, .github/workflows/ci-mariadb10-galera-g1.yml:16-16, .github/workflows/ci-mariadb10-galera-g6.yml:16-16, .github/workflows/ci-mysql84-g5.yml:16-16, .github/workflows/ci-mysql84-gr-g3.yml:16-16, and .github/workflows/ci-mysql93-gr-g1.yml:16-16.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/ci-legacy-g6.yml:
- Line 84: Update the post-test preservation conditions for the
artifact-permission, artifact-log-archive, coverage-artifact-upload, and
coverage-upload steps so they execute when jobs are cancelled, not only when
they complete normally. Apply the same condition changes in
.github/workflows/ci-legacy-g6.yml:84-84,
.github/workflows/ci-mariadb10-galera-g1.yml:16-16,
.github/workflows/ci-mariadb10-galera-g6.yml:16-16,
.github/workflows/ci-mysql84-g5.yml:16-16,
.github/workflows/ci-mysql84-gr-g3.yml:16-16, and
.github/workflows/ci-mysql93-gr-g1.yml:16-16.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1d0f0e7b-fdaa-4067-ab05-de9dc8a8e307
📒 Files selected for processing (56)
.github/workflows/ci-3p-aiomysql.yml.github/workflows/ci-3p-django-framework.yml.github/workflows/ci-3p-laravel-framework.yml.github/workflows/ci-3p-mariadb-connector-c.yml.github/workflows/ci-3p-mysql-connector-j.yml.github/workflows/ci-3p-pgjdbc.yml.github/workflows/ci-3p-php-pdo-mysql.yml.github/workflows/ci-3p-php-pdo-pgsql.yml.github/workflows/ci-3p-postgresql.yml.github/workflows/ci-3p-sqlalchemy.yml.github/workflows/ci-basictests.yml.github/workflows/ci-legacy-clickhouse-g1.yml.github/workflows/ci-legacy-g1.yml.github/workflows/ci-legacy-g2.yml.github/workflows/ci-legacy-g3.yml.github/workflows/ci-legacy-g4.yml.github/workflows/ci-legacy-g5.yml.github/workflows/ci-legacy-g6.yml.github/workflows/ci-legacy-g7.yml.github/workflows/ci-legacy-g8.yml.github/workflows/ci-legacy-g9.yml.github/workflows/ci-mariadb10-galera-g1.yml.github/workflows/ci-mariadb10-galera-g2.yml.github/workflows/ci-mariadb10-galera-g3.yml.github/workflows/ci-mariadb10-galera-g4.yml.github/workflows/ci-mariadb10-galera-g5.yml.github/workflows/ci-mariadb10-galera-g6.yml.github/workflows/ci-mariadb10-galera-g7.yml.github/workflows/ci-mariadb10-galera-g8.yml.github/workflows/ci-mariadb10-galera-g9.yml.github/workflows/ci-mysql56-single-g1.yml.github/workflows/ci-mysql84-g1.yml.github/workflows/ci-mysql84-g2.yml.github/workflows/ci-mysql84-g3.yml.github/workflows/ci-mysql84-g4.yml.github/workflows/ci-mysql84-g5.yml.github/workflows/ci-mysql84-g6.yml.github/workflows/ci-mysql84-g7.yml.github/workflows/ci-mysql84-g8.yml.github/workflows/ci-mysql84-g9.yml.github/workflows/ci-mysql84-gr-g1.yml.github/workflows/ci-mysql84-gr-g2.yml.github/workflows/ci-mysql84-gr-g3.yml.github/workflows/ci-mysql84-gr-g4.yml.github/workflows/ci-mysql84-gr-g5.yml.github/workflows/ci-mysql84-gr-g6.yml.github/workflows/ci-mysql84-gr-g7.yml.github/workflows/ci-mysql84-gr-g8.yml.github/workflows/ci-mysql84-gr-g9.yml.github/workflows/ci-mysql90-gr-g1.yml.github/workflows/ci-mysql93-gr-g1.yml.github/workflows/ci-mysql95-gr-g1.yml.github/workflows/ci-mysqlx.yml.github/workflows/ci-pgsql-socket-g1.yml.github/workflows/ci-set_parser_algorithm_3-g1.yml.github/workflows/ci-trigger.yml
🚧 Files skipped from review as they are similar to previous changes (47)
- .github/workflows/ci-mysql84-gr-g2.yml
- .github/workflows/ci-3p-pgjdbc.yml
- .github/workflows/ci-mysql56-single-g1.yml
- .github/workflows/ci-mariadb10-galera-g2.yml
- .github/workflows/ci-3p-sqlalchemy.yml
- .github/workflows/ci-mysql84-gr-g8.yml
- .github/workflows/ci-mariadb10-galera-g7.yml
- .github/workflows/ci-mysql84-g2.yml
- .github/workflows/ci-3p-aiomysql.yml
- .github/workflows/ci-3p-django-framework.yml
- .github/workflows/ci-3p-mariadb-connector-c.yml
- .github/workflows/ci-legacy-g8.yml
- .github/workflows/ci-legacy-g9.yml
- .github/workflows/ci-legacy-g7.yml
- .github/workflows/ci-mysql84-g9.yml
- .github/workflows/ci-mariadb10-galera-g8.yml
- .github/workflows/ci-mysqlx.yml
- .github/workflows/ci-3p-php-pdo-pgsql.yml
- .github/workflows/ci-legacy-g5.yml
- .github/workflows/ci-mysql84-g6.yml
- .github/workflows/ci-pgsql-socket-g1.yml
- .github/workflows/ci-3p-php-pdo-mysql.yml
- .github/workflows/ci-mariadb10-galera-g9.yml
- .github/workflows/ci-legacy-g3.yml
- .github/workflows/ci-mariadb10-galera-g5.yml
- .github/workflows/ci-legacy-g2.yml
- .github/workflows/ci-mysql84-g3.yml
- .github/workflows/ci-mysql90-gr-g1.yml
- .github/workflows/ci-set_parser_algorithm_3-g1.yml
- .github/workflows/ci-mariadb10-galera-g4.yml
- .github/workflows/ci-mysql84-gr-g6.yml
- .github/workflows/ci-mysql84-gr-g7.yml
- .github/workflows/ci-3p-mysql-connector-j.yml
- .github/workflows/ci-mysql95-gr-g1.yml
- .github/workflows/ci-mysql84-g7.yml
- .github/workflows/ci-legacy-clickhouse-g1.yml
- .github/workflows/ci-mysql84-gr-g4.yml
- .github/workflows/ci-mysql84-gr-g9.yml
- .github/workflows/ci-mysql84-gr-g5.yml
- .github/workflows/ci-mysql84-g4.yml
- .github/workflows/ci-legacy-g4.yml
- .github/workflows/ci-mysql84-g8.yml
- .github/workflows/ci-mysql84-g1.yml
- .github/workflows/ci-mysql84-gr-g1.yml
- .github/workflows/ci-mariadb10-galera-g3.yml
- .github/workflows/ci-3p-postgresql.yml
- .github/workflows/ci-3p-laravel-framework.yml
📜 Review details
🔇 Additional comments (9)
.github/workflows/ci-basictests.yml (1)
28-28: LGTM!Also applies to: 239-239
.github/workflows/ci-trigger.yml (1)
10-14: LGTM!.github/workflows/ci-legacy-g1.yml (1)
28-28: LGTM!Also applies to: 245-245
.github/workflows/ci-legacy-g6.yml (1)
28-28: LGTM!Also applies to: 245-245
.github/workflows/ci-mariadb10-galera-g1.yml (1)
177-177: LGTM!.github/workflows/ci-mariadb10-galera-g6.yml (1)
177-177: LGTM!.github/workflows/ci-mysql84-g5.yml (1)
176-176: LGTM!.github/workflows/ci-mysql84-gr-g3.yml (1)
177-177: LGTM!.github/workflows/ci-mysql93-gr-g1.yml (1)
177-177: LGTM!
The previous commit matched step names ending in 'tests', which silently skipped every workflow whose long-running step is named differently: ci-legacy-g2-genai Run legacy-g2 tests (GenAI + Coverage) ci-unittests Run unit-tests-g1 ci-taptests-pgsql-cluster Run test_cluster_sync_pgsql-t ci-maketest Make-test ci-selftests Self-tests ci-repltests Replication-tests ci-shuntest Shun-tests ci-pg-compat Run pg-compat suite (non-gating, discovery phase) ci-builds Build All nine upload artifacts, so all nine had the gap this series exists to close: a job-level timeout terminates the job before the upload steps run. ci-repltests and ci-shuntest get 35 rather than 90 -- their job budget is 45, so a 90-minute step timeout could never fire before the job died.
d3e9416 to
10e7e8c
Compare
|
1 participant
No TAP job had a timeout, and
TEST_TAP_TIMEOUTdefaults to 0, so a single hung test ran to GitHub's 6-hour ceiling — as happened on #5991 (CI-mysql84-g9stalled ontest_ssl_fast_forward-3_libmariadb-t).Worse than slow: GitHub marks a 6h-ceiling kill as
cancelled, notfailure, so archive steps guarded byfailure() && !cancelled()are skipped — the run that most needs its logs uploads none.Adds
timeout-minutes: 120to 83 jobs across 64 workflows. 120 is well above the slowest observed successful group (CI-legacy-g4, ~58 min) and matches whatci-pg-compat.ymlalready uses.Untouched:
ci-pg-compat.yml(already 120),ci-repltests.yml/ci-shuntest.yml(already 45, deliberately tighter).Jobs that only call a reusable workflow are untouched —
timeout-minutesis not a valid key on auses:job, which is why this belongs on the callee side rather than in theCI-*.ymlcallers onv3.0.All 64 files re-parse as valid YAML.
Summary by CodeRabbit