Skip to content

Add metrics for how often a dag gets serialised.#68906

Open
Ei-Sandi wants to merge 1 commit into
apache:mainfrom
Ei-Sandi:dag-serialisation-metrics
Open

Add metrics for how often a dag gets serialised.#68906
Ei-Sandi wants to merge 1 commit into
apache:mainfrom
Ei-Sandi:dag-serialisation-metrics

Conversation

@Ei-Sandi

@Ei-Sandi Ei-Sandi commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Adds a new dag.serialization counter metric that is emitted every time a Dag is serialized and written to the metadata DB.
  • The metric is tagged with dag_id and bundle_name, so serialization frequency can be tracked per Dag and per bundle.

(This gives operators visibility into how often DAGs are being re-serialized, to ensure we aren't unnecessarily re-serializing DAGs. )

Tests

  • Covered by unit tests.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Co-authored-by: Claude Sonnet 4.6, Claude Opus 4.8 following the guidelines

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Ei-Sandi Ei-Sandi marked this pull request as ready for review June 23, 2026 16:35

@xBis7 xBis7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good! I have a few suggestions for the tests. Thanks!

BTW, I've tested it manually with breeze and metrics work as expected.

Comment thread airflow-core/tests/unit/models/test_serialized_dag.py Outdated
Comment thread airflow-core/tests/unit/models/test_serialized_dag.py
@Ei-Sandi Ei-Sandi force-pushed the dag-serialisation-metrics branch from 27b6bfa to 88a6071 Compare June 23, 2026 17:56
@Ei-Sandi

Copy link
Copy Markdown
Contributor Author

@xBis7 The comments has been addressed. Can you please review it again ? Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants