Skip to content

Report slow tests results to Slack based on the job status - #6754

Open
albertvillanova wants to merge 4 commits into
mainfrom
ci-slow-tests-slack-status
Open

Report slow tests results to Slack based on the job status#6754
albertvillanova wants to merge 4 commits into
mainfrom
ci-slow-tests-slack-status

Conversation

@albertvillanova

@albertvillanova albertvillanova commented Aug 14, 2026

Copy link
Copy Markdown
Member

Stacked on top of #6753.

This PR makes the slow tests workflow report to Slack based on the job status, using the shared post-slack action already used by the other test workflows, and reduces scripts/log_reports.py to writing the job summary.

Motivation

The slow tests workflow posts to Slack from scripts/log_reports.py, which only sends a message when it finds failed tests in the pytest report log. Any failure that prevents the log from being written, a failed installation, a container killed by the runner, a job timeout, or a pytest startup error, leaves the report empty, so the script sends nothing and the failure goes unnoticed.

Reporting the job status instead makes the notification independent of what pytest managed to write, and it is what the other test workflows already do, so the workflows now share a single Slack mechanism.

Solution

Both jobs end with the shared post-slack action, passing ${{ job.status }}, as in tests.yml. The report step keeps generating the job summary, and log_reports.py no longer builds Slack payloads, so it no longer needs slack_sdk nor the SLACK_API_TOKEN variable. The script also now reports missing and empty log files in the job summary, which were previously only mentioned in the message it failed to send.

Changes

  • Post the slow tests results to Slack with the shared post-slack action, based on the job status
  • Remove the Slack payload handling from scripts/log_reports.py, keeping the failed tests table and adding the missing and empty log file warnings to the job summary
  • Report in the job summary when all tests passed
  • Replace the SLACK_API_TOKEN environment variable with CI_SLACK_CHANNEL, as in the other test workflows
  • Stop installing slack_sdk in the report steps

Note for reviewers

The Slack channel now comes from the CI_PUSH_MAIN_CHANNEL secret, for consistency with the other test workflows, whereas scripts/log_reports.py had trl-push-ci hardcoded as a default. Please confirm both point to the same channel.


Note

Low Risk
CI and reporting-only changes with no runtime library impact; channel secret alignment with other workflows is the main operational check.

Overview
Slow-test CI now notifies Slack from job status (shared post-slack action on both single- and multi-GPU jobs), matching tests.yml, so failures still alert when pytest never writes a log.

scripts/log_reports.py no longer builds or sends Slack messages: it only prints the GitHub job summary (failed-test tables, pass/no-log warnings). The workflow drops slack_sdk and SLACK_API_TOKEN, uses CI_PUSH_MAIN_CHANNEL via CI_SLACK_CHANNEL, and keeps tabulate for the summary step.

Reviewed by Cursor Bugbot for commit 8897542. Bugbot is set up for automated code reviews on this repo. Configure here.

@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c32dd56. Configure here.

Comment thread scripts/log_reports.py Outdated

@qgallouedec qgallouedec left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, only two nits

Comment thread scripts/log_reports.py Outdated
Comment thread scripts/log_reports.py Outdated
@qgallouedec
qgallouedec force-pushed the ci-slow-tests-slack-status branch from c32dd56 to 176a66f Compare August 14, 2026 18:41
@qgallouedec

Copy link
Copy Markdown
Member

Ah, for the record, because it's about slow tests: I’ve never felt that the slow tests add much value. Most are redundant with the default tests, and some aren’t actually slow, they’re just labeled as such for some reason. Cleaning this up and seeing whether we can simply drop them has always been a low-priority item in the back of my mind.

Base automatically changed from ci-fix-report-table-headers to main August 17, 2026 05:29
@albertvillanova
albertvillanova force-pushed the ci-slow-tests-slack-status branch from 176a66f to ffb19a7 Compare August 17, 2026 05:29
@albertvillanova
albertvillanova force-pushed the ci-slow-tests-slack-status branch from ffb19a7 to 906c70d Compare August 17, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants