Skip to content

Fix executor validation error referencing non-existent [core] executors - #71336

Open
roshanprabu wants to merge 1 commit into
apache:mainfrom
roshanprabu:fix-executor-config-error-message
Open

Fix executor validation error referencing non-existent [core] executors#71336
roshanprabu wants to merge 1 commit into
apache:mainfrom
roshanprabu:fix-executor-config-error-message

Conversation

@roshanprabu

Copy link
Copy Markdown

Summary

_validate_executor_fields (airflow-core/src/airflow/dag_processing/dagbag.py) raises UnknownExecutorException when a task's executor= isn't available, and tells the user to check their "[core] executors configuration". That config key does not exist:

executor_config = conf.get_mandatory_value("core", "executor")

(airflow-core/src/airflow/executors/executor_loader.py) -- Airflow has a single [core] executor key, documented in config.yml as accepting a comma-separated list to configure multiple executors. The plural "executors" only appears in this error message text; grepping the whole tree, it does not exist as an actual config option anywhere.

This is exactly the kind of message a user hits when trying to run a task with an explicit executor= that isn't in their configured list (e.g. from a Helm chart deployment), and the message sends them looking for a setting ([core] executors) that isn't there instead of telling them to make [core] executor a comma-separated list.

Fix: both branches of the message (with and without a team name) now say [core] executor.

Test plan

  • Confirmed via conf.get_mandatory_value("core", "executor") that this is the only config key involved -- no plural variant exists in config.yml or anywhere else in the codebase.
  • Updated the 4 matching assertions in airflow-core/tests/unit/dag_processing/test_dagbag.py.
  • Ran the full test suite for this: pytest tests/unit/dag_processing/test_dagbag.py -k executor -- 12 passed.
  • Ran the full test_dagbag.py file (72 tests) to confirm no other breakage -- passes aside from 4 pre-existing errors unrelated to this change (missing airflow_shared module in an unrelated fixture, present before this change too).
  • ruff check and ruff format --check pass on both changed files.

_validate_executor_fields raised UnknownExecutorException telling users
to check their "[core] executors configuration" when a task's executor
isn't available. That config key doesn't exist -- Airflow has a single
[core] executor key (conf.get_mandatory_value("core", "executor")),
which accepts a comma-separated list to configure multiple executors.
The plural "executors" only ever appears in this error text, sending
users looking for a setting that isn't there.

Fixes both branches of the message (with and without a team name).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant