Skip to content

Ability to configure SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS per field #466

@kishan3

Description

@kishan3

Feature Request: Configure SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS Per Field

Summary

Currently, SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS sets a global level for all fields in Spidermon. It would be helpful to enable per-field configuration. For example, while most fields could use the default level (e.g., 1), certain fields (e. g., X) could be set to a different level (e.g., 0).

Motivation

This flexibility improves control over data monitoring and validation. Some fields may not always require the same coverage level as others due to business or scraping requirements.

Proposed Solution

  • Allow SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS to accept both global and field-level settings.
  • Example usage:
    SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS = {
      'default': 1,
      'X': 0,  # Set coverage level to 0 for field X only
    }
    If no per-field value exists, default to the global setting.

Additional Context

  • This mirrors the flexibility found in other Spidermon settings.
  • Backward compatibility should be considered for existing users.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions