Rayz/tagfilter cache by rayz · Pull Request #1260 · DataDog/saluki

rayz · 2026-03-25T18:23:47Z

Summary

Change Type

Bug fix
New feature
Non-functional (chore, refactoring, docs)
Performance

How did you test this PR?

References

pr-commenter · 2026-03-25T18:24:19Z

Binary Size Analysis (Agent Data Plane)

Target: 1d50af4 (baseline) vs 642e524 (comparison) diff
Analysis Type: Stripped binaries (debug symbols excluded)
Baseline Size: 26.19 MiB
Comparison Size: 26.29 MiB
Size Change: +102.90 KiB (+0.38%)
Pass/Fail Threshold: +5%
Result: PASSED ✅

Changes by Module

Module	File Size	Symbols
`saluki_components::transforms::tag_filterlist`	+31.65 KiB	18
`core`	+29.57 KiB	72
`serde_core`	+8.78 KiB	8
`agent_data_plane::cli::run`	+8.06 KiB	2
`[sections]`	+6.07 KiB	6
`hashbrown`	+5.59 KiB	9
`saluki_context::resolver::ContextResolver`	+3.99 KiB	1
`[Unmapped]`	+3.20 KiB	1
`saluki_config::dynamic::watcher`	-2.34 KiB	1
`saluki_context::hash::hash_context`	+2.15 KiB	1
`tokio`	+1.81 KiB	2
`saluki_core::topology::blueprint`	+1.19 KiB	1
`saluki_context::resolver::TagsResolver`	+913 B	2
`saluki_context::context::Context`	+887 B	1
`smallvec`	+753 B	1
`saluki_components::encoders::datadog`	+379 B	2
`serde_json`	+281 B	1
`saluki_config::secrets::resolver`	-88 B	1
`figment`	+76 B	17
`saluki_components::destinations::prometheus`	+59 B	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +3.3% +51.2Ki  +3.3% +37.9Ki    [525 Others]
  [NEW] +16.1Ki  [NEW] +15.9Ki    _<saluki_components::transforms::tag_filterlist::TagFilterlist as saluki_core::components::transforms::Transform>::run::_{{closure}}::h2ab4b7b55a05b531
  [NEW] +12.6Ki  [NEW] +12.4Ki    _<core::marker::PhantomData<T> as serde_core::de::DeserializeSeed>::deserialize::h823409a611a5cfb1
  [NEW] +10.7Ki  [NEW] +10.6Ki    <saluki_core::data_model::event::Event as core::clone::Clone>::clone.10049
  +7.0% +8.06Ki  +7.1% +8.06Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h4a1b0b37b7e2d03e
  [NEW] +5.66Ki  [NEW] +5.49Ki    serde_core::de::impls::_<impl serde_core::de::Deserialize for alloc::vec::Vec<T>>::deserialize::h34dcebfa9500250f
  [NEW] +5.52Ki  [NEW] +5.38Ki    <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.8502
  [NEW] +5.13Ki  [NEW] +5.00Ki    <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.11186
  [NEW] +4.76Ki  [NEW] +4.61Ki    _<core::marker::PhantomData<T> as serde_core::de::DeserializeSeed>::deserialize::h4e9adec8ddd65cfa
  [NEW] +4.71Ki  [NEW] +4.59Ki    <rustls::error::Error as core::clone::Clone>::clone.10025
  [NEW] +4.71Ki  [NEW] +4.59Ki    <rustls::error::Error as core::clone::Clone>::clone.11201
  [NEW] +4.26Ki  [NEW] +4.15Ki    saluki_components::transforms::tag_filterlist::compile_filters::hd808d986c0d7a078
  [NEW] +3.99Ki  [NEW] +3.89Ki    saluki_context::resolver::ContextResolver::resolve_inner::h7d8a1e0c844cd7fb
  [NEW] +3.83Ki  [NEW] +3.71Ki    <webpki::error::Error as core::fmt::Debug>::fmt.11379
  [DEL] -3.66Ki  [DEL] -3.54Ki    <rustls::error::Error as core::fmt::Debug>::fmt.9641
  [DEL] -3.83Ki  [DEL] -3.71Ki    <webpki::error::Error as core::fmt::Debug>::fmt.11359
  [DEL] -4.71Ki  [DEL] -4.59Ki    <rustls::error::Error as core::clone::Clone>::clone.11181
  [DEL] -4.71Ki  [DEL] -4.59Ki    <rustls::error::Error as core::clone::Clone>::clone.10017
  [DEL] -5.13Ki  [DEL] -5.00Ki    <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.11166
  [DEL] -5.52Ki  [DEL] -5.38Ki    <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.8496
  [DEL] -10.7Ki  [DEL] -10.6Ki    <saluki_core::data_model::event::Event as core::clone::Clone>::clone.10041
  +0.4%  +102Ki  +0.4% +88.9Ki    TOTAL

pr-commenter · 2026-03-25T18:40:45Z

Regression Detector (Agent Data Plane)

This comment was omitted because it was over 65,536 characters.Please check the Gitlab Job logs to see its output.

Copilot

Pull request overview

Adds an LRU-style cache to the tag_filterlist transform to reuse previously filtered tag sets and reduce repeated allocations when multiple metrics share the same underlying tag storage.

Changes:

Introduce a concurrent cache keyed by tagset identity + rule discriminator to reuse filtered SharedTagSet results.
Add a configuration-driven cache capacity and initialize the cache in the transform builder.
Route sketch metrics through the new cached filtering path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

lib/saluki-components/src/transforms/tag_filterlist/mod.rs

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-26T16:35:19Z

lib/saluki-components/src/transforms/tag_filterlist/mod.rs

 fn apply_tag_filter(
    tags: &SharedTagSet, is_exclude: bool, names: &HashSet<String, FoldHashState>,
 ) -> Option<FilteredTagSet> {


The documentation immediately above apply_tag_filter still says it returns Some(TagSet), but the function returns Option<FilteredTagSet> (including a removed count). Please update the doc comment to reflect the actual return type/behavior.

Copilot · 2026-03-26T16:35:19Z

lib/saluki-components/src/transforms/tag_filterlist/mod.rs

+    /// This should generally match `aggregate_context_limit` so the resolver can retain one
+    /// filtered context per distinct aggregated context. Defaults to 5000.
+    #[serde(rename = "aggregate_context_limit", default = "default_context_resolver_limit")]


context_resolver_limit is exposed in config as aggregate_context_limit via #[serde(rename = ...)], but this setting controls the tag-filter resolver cache (not the aggregate transform’s context limit). This key/name mismatch is likely to confuse users and can cause accidental coupling with configs copied from the aggregate transform. Consider renaming the config field to something tag-filter specific (e.g., tag_filterlist_context_resolver_limit / context_resolver_limit) and updating the comment accordingly; if reusing the key is intentional, please document that explicitly in user-facing config docs.

Suggested change

/// This should generally match `aggregate_context_limit` so the resolver can retain one

/// filtered context per distinct aggregated context. Defaults to 5000.

#[serde(rename = "aggregate_context_limit", default = "default_context_resolver_limit")]

/// When used alongside the aggregate transform, this can be set to match that transform's

/// context limit so the resolver can retain one filtered context per distinct aggregated

/// context. Defaults to 5000.

#[serde(

rename = "tag_filterlist_context_resolver_limit",

alias = "aggregate_context_limit",

default = "default_context_resolver_limit"

)]

cache filtered SharedTagSet

91b1a67

dd-octo-sts bot added area/core Core functionality, event model, etc. area/components Sources, transforms, and destinations. area/ci CI/CD, automated testing, etc. area/test All things testing: unit/integration, correctness, SMP regression, etc. labels Mar 25, 2026

rayz changed the base branch from main to raymond/tag-agg-smp March 25, 2026 18:24

lru

3605a17

Copilot AI review requested due to automatic review settings March 25, 2026 19:49

dd-octo-sts bot removed area/core Core functionality, event model, etc. area/ci CI/CD, automated testing, etc. area/test All things testing: unit/integration, correctness, SMP regression, etc. labels Mar 25, 2026

Copilot started reviewing on behalf of rayz March 25, 2026 19:49 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

rayz added 3 commits March 25, 2026 16:27

feedback

af79a31

Trigger ci

7fd1b39

Use ContextResolver

642e524

Copilot AI review requested due to automatic review settings March 26, 2026 16:29

Copilot started reviewing on behalf of rayz March 26, 2026 16:30 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rayz/tagfilter cache#1260

Rayz/tagfilter cache#1260
rayz wants to merge 5 commits intoraymond/tag-agg-smpfrom
rayz/tagfilter-cache

rayz commented Mar 25, 2026

Uh oh!

pr-commenter bot commented Mar 25, 2026 •

edited

Loading

Changes by Module

Detailed Symbol Changes

Uh oh!

pr-commenter bot commented Mar 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 26, 2026

Uh oh!

Copilot AI Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rayz commented Mar 25, 2026

Summary

Change Type

How did you test this PR?

References

Uh oh!

pr-commenter bot commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary Size Analysis (Agent Data Plane)

Changes by Module

Detailed Symbol Changes

Uh oh!

pr-commenter bot commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector (Agent Data Plane)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pr-commenter bot commented Mar 25, 2026 •

edited

Loading

pr-commenter bot commented Mar 25, 2026 •

edited

Loading