Feature Request: Configure SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS Per Field
Summary
Currently, SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS sets a global level for all fields in Spidermon. It would be helpful to enable per-field configuration. For example, while most fields could use the default level (e.g., 1), certain fields (e. g., X) could be set to a different level (e.g., 0).
Motivation
This flexibility improves control over data monitoring and validation. Some fields may not always require the same coverage level as others due to business or scraping requirements.
Proposed Solution
- Allow
SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS to accept both global and field-level settings.
- Example usage:
SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS = {
'default': 1,
'X': 0, # Set coverage level to 0 for field X only
}
If no per-field value exists, default to the global setting.
Additional Context
- This mirrors the flexibility found in other Spidermon settings.
- Backward compatibility should be considered for existing users.
Feature Request: Configure
SPIDERMON_DICT_FIELDS_COVERAGE_LEVELSPer FieldSummary
Currently,
SPIDERMON_DICT_FIELDS_COVERAGE_LEVELSsets a global level for all fields in Spidermon. It would be helpful to enable per-field configuration. For example, while most fields could use the default level (e.g., 1), certain fields (e. g.,X) could be set to a different level (e.g., 0).Motivation
This flexibility improves control over data monitoring and validation. Some fields may not always require the same coverage level as others due to business or scraping requirements.
Proposed Solution
SPIDERMON_DICT_FIELDS_COVERAGE_LEVELSto accept both global and field-level settings.Additional Context