Skip to content

Optimize push-to-Dune CI: sync only changed SQL queries#20

Open
VaitaR wants to merge 2 commits intoduneanalytics:mainfrom
VaitaR:main
Open

Optimize push-to-Dune CI: sync only changed SQL queries#20
VaitaR wants to merge 2 commits intoduneanalytics:mainfrom
VaitaR:main

Conversation

@VaitaR
Copy link

@VaitaR VaitaR commented Feb 8, 2026

What this PR changes

Optimize CI/CD for query sync: on push, only changed queries/*.sql files are pushed to Dune instead of all queries from queries.yml.

Also adds:

  • fallback to full sync when diff context is unavailable (FULL_SYNC=true)
  • safer query ID extraction/error handling
  • README update for new behavior

Is this linked to an existing issue
N/A

Fill out the following table describing your edits:

Original Updated Change Reasoning
.github/workflows/push_to_dune.yml .github/workflows/push_to_dune.yml Detect changed SQL files and conditionally run push script Reduce unnecessary updates and speed up CI
scripts/push_to_dune.py scripts/push_to_dune.py Support changed-only sync (CHANGED_QUERY_FILES) with full-sync fallback and ID parsing guards Update only impacted queries while keeping robust behavior
README.md README.md Document changed-only sync and FULL_SYNC=true override Keep contributor workflow clear

Provide any other context or screenshots that explain or justify the changes above:

Validated locally:

  • Python syntax check (py_compile) for scripts/push_to_dune.py
  • Workflow YAML parsing

…y those in Dune; add error handling for query ID extraction
@cursor
Copy link

cursor bot commented Feb 8, 2026

PR Summary

Medium Risk
Changes the CI deploy behavior and query selection logic, so mis-detection or parsing issues could cause intended query updates to be skipped or trigger an unexpected full sync.

Overview
On pushes to main, CI now detects which queries/*.sql files changed and only runs scripts/push_to_dune.py when there are relevant changes; it also fetches full git history and falls back to FULL_SYNC=true when the prior commit SHA isn’t available.

push_to_dune.py is updated to support changed-only sync via CHANGED_QUERY_FILES, adds safer query-id parsing/validation and early exits, and documents the new behavior (including the FULL_SYNC=true override) in README.md.

Written by Cursor Bugbot for commit 6eff0f3. Configure here.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant