feat(data-collection): create DataCollection option in client

[ericapisani]

Support more granular PII controls via the data_collection option as seen in the spec.

These changes:

• introduce the option
• normalizes the user-provided options into a DataCollection dict
• handles populating the data_collection option based on what the user provided, and whether or not send_default_pii (which will eventually be legacy) was set.

They don't currently hook up to anything in order to keep the size of this pull request somewhat sane.

Some high-level design decisions to call out:

1. Introducing 2 types for the different config options - one that's more flexible for user input, the other more strict after we've received the input.

This was done so that, once we've received the config from the user and have set defaults for values that weren't provided, we can avoid needing to constantly check if the value is None within the SDK.

2. Including a provided_by_user property within the DataCollection type

This is mainly to deal with spotlight-related logic within the client, which you can see here.

Fixes PY-2570
Fixes #6700

[github-actions]

Codecov Results 📊

✅ 90435 passed | ⏭️ 6302 skipped | Total: 96737 | Pass Rate: 93.49% | Execution Time: 315m 59s

📊 Comparison with Base Branch

Metric	Change
Total Tests	📈 +390
Passed Tests	📈 +390
Failed Tests	—
Skipped Tests	—

All tests are passing successfully.

✅ Patch coverage is 98.18%. Project has 2439 uncovered lines.
✅ Project coverage is 89.79%. Comparing base (base) to head (head).

Files with missing lines (1)

File	Patch %	Lines
sentry_sdk/data_collection.py	98.04%	⚠️ 1 Missing and 2 partials

Coverage diff

@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    89.74%    89.79%    +0.05%
==========================================
  Files          192       193        +1
  Lines        23823     23878       +55
  Branches      8226      8244       +18
==========================================
+ Hits         21378     21439       +61
- Misses        2445      2439        -6
- Partials      1349      1351        +2

---

Generated by Codecov Action

[ericapisani]

bugbot run

[linear-code]

PY-2570

[alexander-alderman-webb]

Just did a quick pass, not a proper review.
Can we add underscores to everything we don't want users to access? 🙏

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 15c21c0. Configure here.}

[cursor]

Spotlight frame flags disagree

Medium Severity

In DSN-less Spotlight mode, re-derived data_collection uses include_local_variables is not False (and the same for include_source_context), while _resolve_data_collection treats those options with bool(value) when they are not None. Falsy non-boolean values such as 0 resolve to disabled frame settings initially but become enabled after the Spotlight override.

 

^{Reviewed by Cursor Bugbot for commit 15c21c0. Configure here.}

[cursor]

Unpickled client missing data_collection

Low Severity

_Client.__setstate__ restores options from pickled state and calls _init_impl without ensuring a resolved data_collection entry exists. Clients pickled before this option was added have no data_collection key, so DSN-less Spotlight setup or the new data_collection property can raise KeyError.

Additional Locations (2)

• sentry_sdk/client.py#L638-L639
• sentry_sdk/client.py#L762-L763

 

^{Reviewed by Cursor Bugbot for commit 15c21c0. Configure here.}

[ericapisani]

Moving back to draft while I investigate the bot comments

[sentrivana]

LGTM. Left a few comments.

[sentrivana]

For my understanding, we have these type pairs (UserOptions and Behaviour) so that users don't need to specify all options (total=False on the UserOptions types), but also so that the materialized DataCollection that we'll use internally won't nag us about an option potentially being None, because it has to be fully filled out with default values in case any are missing, right?

I see why we have it like this but I'm afraid the two might drift eventually because we forget to update one of them.

It's not like I have better ideas 😀 The complexity we introduce with the config needs to be dealt with someplace, and I guess this is an ok place.

[ericapisani]

For my understanding, we have these type pairs (UserOptions and Behaviour) so that users don't need to specify all options (total=False on the UserOptions types), but also so that the materialized DataCollection that we'll use internally won't nag us about an option potentially being None, because it has to be fully filled out with default values in case any are missing, right?

Yup, you've got it! I wanted to spare us from all the extra code we'd have to write to satisfy the typechecker when performing the checks within the integrations 😅

I agree with you about the risk of drift. I thought it'd be worth it in this case because after the data collection spec is implemented and settled, the amount of updates we'd need to do here would likely be minimal.

If we think of anything better throughout the implementation of the spec though, happy to come back and change the approach! 😄