[pull] master from saltstack:master#1
Open
pull[bot] wants to merge 10000 commits intojohnvox:masterfrom
Open
Conversation
These man pages are still generated and packaged with every release.
…eeded for the packages.
Change-Id: I17222e50986c3a65eb8919367f49b4ea0fb74623
## Problem The `mod_data` function in `salt/client/ssh/__init__.py` was only discovering modules from the fileserver backend (e.g., `_modules`, `_states` directories accessed via `fsclient.cache_file()`). This caused salt-ssh to ignore: - Custom modules registered via entry-points - Modules from `extension_modules` configuration - Modules from `module_dirs` CLI options - Custom fileserver backends and other loader plugins This resulted in inconsistent behavior between `salt-call` (which loads all extensions) and `salt-ssh` (which ignores them). ## Solution Modified `mod_data()` to collect modules from three sources: ### 1. Global Loader Paths Uses `salt.loader._module_dirs()` to discover all module directories, including: - Entry-point registered plugins - `extension_modules` from configuration - `module_dirs` from CLI - System-wide module paths ### 2. File Roots Directly scans `file_roots` directories for `_modules`, `_states`, etc., avoiding network-dependent fileserver operations that require master connectivity. ### 3. Error Handling Wraps all discovery operations in try-except blocks to ensure salt-ssh continues working even if: - Some module paths are inaccessible - Entry-point plugins fail to load - File roots are misconfigured ## Benefits - **Consistency**: salt-ssh now uses the same module discovery logic as salt-call - **Entry-point support**: Custom plugins registered via setuptools entry-points are now available - **No master dependency**: Avoids `KeyError: 'master_uri'` by not using fileserver client operations - **Resilience**: Graceful error handling ensures partial failures don't break salt-ssh ## Technical Details **Before**: Only scanned fileserver via `fsclient.cache_file()` → required master connection **After**: 1. Calls `salt.loader._module_dirs(opts, ref, tag=...)` for each module type 2. Scans directories directly from filesystem 3. Collects modules from `file_roots` configuration 4. Builds tarball with all discovered modules for deployment to targets **Key Changes**: - Removed dependency on `fsclient.cache_file()` which triggers master authentication - Added direct filesystem scanning of `file_roots` - Added comprehensive exception handling - Preserved backward compatibility with existing module discovery ## Impact All salt-ssh deployments now have access to: - Custom execution modules - Custom states - Custom grains - Custom renderers - Custom returners - Custom fileserver backends (when registered via entry-points) This resolves the inconsistency where `salt-call --local sys.list_fileserver_backends` would show custom backends but `salt-ssh <target> sys.list_fileserver_backends` would not.
refactor with chain constant value assignment to make code more Pythonic, concise and efficient.
…tenv In those cases, the explicit saltenv overrides this master config option.
This implements a new optional argument "prune" in the file.keyvalue state function, which, if set to True, deletes lines not configured using the "key_values" or "key"/"value" arguments as part of the execution. Useful for ensuring a file only contains the desired key/value pairs and no additional data. Signed-off-by: Georg Pfuetzenreuter <mail@georg-pfuetzenreuter.net>
With this option it's able to only fire events that match the running state
rather than force a cache.cache() interface and a uniform expiry across the cache, we add a more common per store() expiry that can also be used instead or in addition to the original cache-wide expiry.
previously 'bank' was minions/$minion, and key was 'data', which was a merge of both pillar and grains. rather than do that, we organize pillar and grains into separate banks for better index optimizations for backends that can take advantage of it. it also just makes more sense this way
took the liberty of making it a proper subclass in the process. this now uses the salt.cache infrastructure such that it can be driven by the base cache driver or a different one if so desired. functionality should be equivalent, including using the base bank=pillar key=minion_id for merged pillar, such that minion_data_cache can take advantage of the same cache. because we are updating the cache at the source, we no longer need to update the cache in master/masterapi.
the loops involved dont use sets which seems to have an oversized impact on lookup performance. using sets for containment checks makes this function go 15x faster on a 50k dataset (from 30+ sec to 2~ sec).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )