Skip to content

Always forward Gator warnings/errors to user regardless of --all-msg flag#27

Closed
EdNutting wants to merge 1 commit intoIntuity:mainfrom
EdNutting:ednutting/always-forward-gator-warnings
Closed

Always forward Gator warnings/errors to user regardless of --all-msg flag#27
EdNutting wants to merge 1 commit intoIntuity:mainfrom
EdNutting:ednutting/always-forward-gator-warnings

Conversation

@EdNutting
Copy link
Contributor

Problem

By default --all-msg=False, so Gator's own warnings and errors were not being surfaced to the user. This meant that important messages from Gator itself - such as the new command substitution warning, resource limit violations, or dependency errors - would be silently filtered out along with tool output.

This problem was also present with the --quiet flag.

Solution

This PR modifies the logger to always forward WARNING, ERROR, and CRITICAL messages that originate from Gator itself to the parent layer, regardless of the --all-msg/--quiet flag setting. Messages forwarded from child processes (tools that Gator invokes) continue to respect the --all-msg flag as expected.

Key Changes

  • Modified gator/common/logger.py: Updated the log() method to check if a message originates from Gator itself (not forwarded from children) and automatically forward high-severity messages (WARNING+) regardless of the forward setting
  • Added test: Created test_linked_no_forward in tests/common/test_logger.py to verify that:
    • DEBUG and INFO messages are NOT forwarded when forward=False
    • WARNING, ERROR, and CRITICAL messages ARE forwarded even when forward=False
    • Messages from child processes (forwarded=True) are NOT forwarded when forward=False

Behaviour After This Change

Scenario Debug/Info Messages Gator Warnings/Errors Tool Messages
Default or --quiet Not forwarded Always forwarded Not forwarded
--all-msg Forwarded Forwarded Forwarded

Testing

  • All 137 existing tests pass
  • New test specifically validates the forwarding behaviour with forward=False
  • Coverage increased in logger module from 94% to 97%

Impact

This change ensures that users are always aware of important issues detected by Gator itself, improving visibility and debugging while still maintaining the ability to filter out verbose tool output with the --quiet flag.

…flag

When --all-msg is NOT specified, Gator's own warnings and errors
(like the command substitution warning) were not being surfaced to
the user. This change ensures that WARNING, ERROR, and CRITICAL
messages originating from Gator itself are always forwarded to the
parent layer, while still respecting the --all-msg flag for
filtering verbose output from invoked tools.

Key changes:
- Modified Logger.log() to forward high-severity messages (WARNING+)
  from Gator itself regardless of the forward setting
- Messages forwarded from child processes are still filtered by --all-msg
- Added test_linked_no_forward to verify the new behavior

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
timestamp = datetime.now()
# Always forward warnings, errors, and critical messages from Gator itself
# (not forwarded from children), regardless of forward setting
should_forward = forward or (not forwarded and severity >= LogSeverity.WARNING)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I've not missed something, won't this only forward gator messages from the first child layer - anything below that would have the forwarded flag set and be lost?

I suspect there would need to be some special marker on messages raising up the stack indicating they were coming from gator's own channel, so that they're always forwarded.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh yeah...I forgot Gator invokes itself in infinite nesting. I need to ask you about that architecture as I'm not sure I fully get why it needs to do nested invocation rather than just using one Python instance to recurse down the tree of jobs / manage invocations accordingly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I still don't quite get what it's doing - local scheduler invokes itself for nested job groups/arrays (I think). Slurm I'm not quite sure if it's dispatching JobArray/JobGroup to a node of Slurm to be executed or just aggregating locally then dispacting the end Jobs to Slurm??

@EdNutting
Copy link
Contributor Author

Closing until I fix the nested logging case as you've highlighted.

@EdNutting EdNutting closed this Jan 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants