Skip to content

feat: integration test fail on error in logs#612

Open
sdas-neuraco wants to merge 2 commits into
mainfrom
feat/make-intigration-more-strict-for-logs
Open

feat: integration test fail on error in logs#612
sdas-neuraco wants to merge 2 commits into
mainfrom
feat/make-intigration-more-strict-for-logs

Conversation

@sdas-neuraco
Copy link
Copy Markdown
Contributor

Features

  • When we check when training is completed if any of the logs from the VM are error messages this will cause the integration tests to fail.

@sdas-neuraco sdas-neuraco self-assigned this May 12, 2026
@sdas-neuraco sdas-neuraco added the version:none tests, docs, CI only; no functional changes label May 12, 2026
Comment thread tests/integration/ml/test_training_flow.py Outdated
Comment on lines +612 to +618
job_id, max_entries=10, severity_filter="ERROR"
)
assert "logs" in filtered
assert not filtered["logs"], (
"Training VM emitted ERROR logs while RUNNING:\n\n"
f"{_format_training_log_entries(filtered['logs'])}"
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel there is a bit of repetition here since we introduced this function _assert_no_training_log_errors and it seemed like it does something similar yet not being used which caused some confusion to me. if it cant be used please leave a comment why, and if can maybe it's better to refactor to use it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, this was mainly to chek only errors in log for max_entries=10 while the other uses critial and error, with better formatting

Copy link
Copy Markdown
Contributor

@mark-neuracore mark-neuracore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM w/c

left my comments earlier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

version:none tests, docs, CI only; no functional changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants