Skip to content

[Python] Deflake anomaly transform tests - #39669

Draft
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/deflake-anomaly-detectors
Draft

[Python] Deflake anomaly transform tests#39669
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/deflake-anomaly-detectors

Conversation

@bvolpato

@bvolpato bvolpato commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Fixes #37779.

RunOneDetector reshuffles input before scoring. Existing transform tests used online Z-score detectors and matched each example against scores calculated in creation order. Runner scheduling can legally change that order independently for each ensemble branch, changing detector state and scores. This caused the keyed failure reported in #37779 and a new unkeyed failure in Python ML precommit.

Replace online Z-score use only in the ensemble tests with a small value detector and deterministic score fields. Original x1 and x2 data, expected predictions, and the single-detector Z-score test remain unchanged. Coverage remains for keyed and unkeyed inputs, feature selection, thresholds, ensemble grouping, and AnyVote aggregation.

Reproduction

Forcing one detector branch to consume reversed input reproduces score mismatches before this change. Deterministic score fields remove branch-order dependence from the four ensemble cases.

Testing

  • ./gradlew :sdks:python:test-suites:tox:py313:testPython313 -PpythonVersion=3.13 -Pposargs='apache_beam.ml.anomaly.transforms_test' (12 passed)
  • Four ensemble cases repeated 20 times with two pytest workers (80 passed)
  • YAPF, Ruff, and Pyrefly checks

No CHANGES.md entry. Test-only change.

@github-actions github-actions Bot added the python label Aug 7, 2026
@bvolpato
bvolpato marked this pull request as draft August 7, 2026 04:16
@bvolpato
bvolpato force-pushed the bvolpato/deflake-anomaly-detectors branch from 9ed678a to 1dbdc51 Compare August 7, 2026 04:33
@bvolpato
bvolpato force-pushed the bvolpato/deflake-anomaly-detectors branch from 1dbdc51 to 5ba530e Compare August 7, 2026 05:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Failing Test]: TestAnomalyDetection.test_multiple_detectors_without_aggregation_0 is possibly flaky

1 participant