Skip to content

fix: report migrations chained behind the deployed head - #47

Closed
RizhongLin wants to merge 1 commit into
Mergifyio:mainfrom
RizhongLin:report-migrations-chained-behind-head
Closed

fix: report migrations chained behind the deployed head#47
RizhongLin wants to merge 1 commit into
Mergifyio:mainfrom
RizhongLin:report-migrations-chained-behind-head

Conversation

@RizhongLin

@RizhongLin RizhongLin commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

A hybrid migration (a static file whose hardcoded down_revision points at a dynamic revision) is placed immediately after its target, and every dynamic migration added in between is re-parented onto it. That is what keeps the chain linear when two branches fork from the same head, and it is exactly right while none of those migrations has run yet.

It stops being right once one of them has been applied. I hit this on a project where a migration was added with a hardcoded down_revision pointing at a revision that was no longer the head:

chain before:  aaaa -> bbbb -> cccc -> dddd          alembic_version = dddd
chain after:   aaaa -> bbbb -> eeee -> cccc -> dddd  alembic_version = dddd

eeee now sits behind the recorded head. alembic upgrade head walks down from dddd, finds every revision already applied, and does nothing. The migration never runs and nothing reports an error. It is only noticed later, when something depends on a column that was never created.

Why this does not change the chain builder

My first attempt was to reject this at build time. That turns out to be wrong, and test_dynamic_inserted_before_hybrid_no_multiple_heads is the reason: it describes branch A adding a dynamic migration and branch B adding a hybrid onto the same head, with A merging first. The resulting chain is aaaa -> bbbb -> dddd -> cccc. If a database deployed after A merged and before B did, it is sitting at cccc, and dddd never runs. Same mechanism, same outcome.

So the benign case and the damaging one are the same shape in git history. What separates them is only whether a displaced revision has already been applied somewhere, which git cannot know. Refusing to build the chain would break the concurrent-branch case this library deliberately supports.

This PR therefore leaves chain building completely untouched and adds reporting instead.

What it adds

  • find_displaced_revisions(versions_dir, applied=None) returning DisplacedRevision(hybrid, target, displaced).

    • Without applied: advisory. Lists every hybrid whose target is not the newest revision preceding it, including the benign fork case.
    • With applied (the revisions a database has actually run): keeps only hybrids that provably cannot execute, because something after them is already applied.
  • The same split on the CLI, which matters more than it sounds. --check alone exits 0 and words its findings conditionally, because a tool with no database access cannot prove anything, and failing a build on the benign fork case would break the workflow this library exists to support:

    $ alembic-git-revisions --check versions/
    eeee: hardcodes down_revision='bbbb'; 2 later revision(s) now chain after it: cccc, dddd
      If any of those is already applied on a database, eeee sits behind that database's
      head and will not run there. Git history alone cannot tell: two branches forking
      from the same head produce this shape legitimately. Pass --applied to decide.
    

    --check --applied <file|-> narrows to what provably cannot run and exits non-zero. That is the form worth gating CI on:

    $ psql -Atc 'select version_num from alembic_version' \
        | alembic-git-revisions --check --applied - versions/
    eeee: hardcodes down_revision='bbbb', but cccc is already applied and now chains after it.
      eeee sits behind this database's head, so 'alembic upgrade head' will never run it.
    
  • A README section stating the rule plainly: only hardcode down_revision onto the current head.

Notes for review

  • No existing behavior changes. build_chain, get_down_revision and _build_chain_from_git return exactly what they did before; all pre-existing tests pass unmodified.
  • Rebased onto feat: expose the parsed migration view #48, which landed first. The reporting needs classification and git add order but not a chain, which is what feat: expose the parsed migration view #48 made public, so the private helper this branch carried for that is gone and find_displaced_revisions calls parse_versions_dir directly. Nothing else about the two changes interacts.
  • _cli moves from manual sys.argv parsing to argparse to take the flag. The positional form is unchanged, but the exit code for a bad invocation moves from 1 to argparse's 2.
  • Failures that were previously tracebacks are now messages. A missing or shallow git clone, and an unreadable --applied file, all exit 1 as alembic-git-revisions: error: .... This covers the pre-existing generate path too, which raised the same RuntimeError uncaught.
  • The README bullet "Existing migrations with hardcoded down_revision continue to work" is now qualified, since that is the sentence I had believed.
  • Tested on 3.11 / 3.12 / 3.13 / 3.14.

Copilot AI lite review requested due to automatic review settings August 8, 2026 20:01
@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 8, 2026 20:01 Failure
@mergify

mergify Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 2 of 6 protections blocking · waiting on 👀 reviews and 🤖 CI

Protection Waiting on
🔴 👀 Review Requirements 👀 reviews
🟠 🤖 Continuous Integration 🤖 CI
🟢 Enforce conventional commit
🟢 🔎 Reviews
🟢 📕 PR description
🟢 🚦 Auto-queue

🔴 👀 Review Requirements

Waiting for

  • #approved-reviews-by>=1
This rule is failing.
  • any of:
    • #approved-reviews-by>=1
    • author = dependabot[bot]
    • author = mergify-ci-bot

🟠 🤖 Continuous Integration

Waiting for

  • check-success=all-greens
Waiting checks: all-greens.
  • all of:
    • check-success=all-greens

Show 4 satisfied protections

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|internal|docs|style|refactor|perf|test|build|ci|chore|revert|ui)(?:\(.+\))?!?:

🟢 🔎 Reviews

  • #changes-requested-reviews-by = 0
  • #review-requested = 0
  • #review-threads-unresolved = 0

🟢 📕 PR description

  • body ~= (?ms:.{48,})

🟢 🚦 Auto-queue

When all merge protections are satisfied, this pull request will be queued automatically.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds advisory + confirmed reporting for “hybrid” Alembic migrations whose hardcoded down_revision targets a dynamic revision that is no longer the newest predecessor, which can silently place a new migration behind an already-deployed database head. This extends the library with detection/reporting while intentionally leaving chain-building behavior unchanged.

Changes:

  • Introduces find_displaced_revisions(versions_dir, applied=None) and DisplacedRevision to identify hybrids that displace later dynamic revisions (optionally filtering to those provably unreachable given an applied set).
  • Updates the CLI to support --check (advisory, exit 0) and --check --applied <file|-> (confirmed, exit non-zero on findings) via argparse.
  • Documents the “only hardcode down_revision onto the current head” rule and how to check for violations.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
tests/test_chain.py Adds unit tests for displaced-revision detection and CLI --check/--applied behavior.
README.md Documents the deployed-database hazard, the rule, and new CLI/Python checking workflows.
alembic_git_revisions/_chain.py Adds DisplacedRevision and find_displaced_revisions() implementation.
alembic_git_revisions/init.py Exposes new API and rewrites CLI parsing to add --check/--applied.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread alembic_git_revisions/__init__.py Outdated
Comment thread alembic_git_revisions/_chain.py
@RizhongLin
RizhongLin force-pushed the report-migrations-chained-behind-head branch from f1505b0 to 0b7a630 Compare August 8, 2026 20:10
@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 8, 2026 20:10 Failure
@RizhongLin

Copy link
Copy Markdown
Contributor Author

Two things I am happy to change rather than defend, keeping them here rather than in the description since the description becomes the commit message on merge:

  • The argparse move. Taking a flag meant moving _cli off manual sys.argv parsing, which changes the exit code for a bad invocation from 1 to argparse's 2. If you would rather not change that for an existing entry point, I can keep the old parser and hand-roll the flag.
  • --check exiting 0. I made the advisory mode non-failing deliberately, because a tool with no database access cannot prove a finding is real, and failing on the fork-from-the-same-head case would break the workflow this library exists to support. If you would prefer --check to fail on any finding, that is a one-line change, but I think it would make the flag unusable in CI.

mergify Bot pushed a commit that referenced this pull request Aug 11, 2026
Tooling built around this library has to re-implement parsing and ordering, or reach into private helpers, to answer questions the library already answers internally: which migrations are dynamic, what order git puts them in, and what the generated artifact is called.

This exports the pieces that answer those questions without building a chain:

- `parse_versions_dir(versions_dir)` returns the migrations as `MigrationFile` objects in git add order. A thin wrapper over the existing git-order and parse steps.
- `MigrationFile` is already a public class, it simply was not exported.
- `CHAIN_FILENAME` so callers locating or cleaning up the generated file do not hardcode `"revision_chain.json"`. The private `_CHAIN_FILENAME` is removed rather than aliased: its only two references were internal to `_chain.py`, both updated here, and a leading-underscore name carries no compatibility guarantee.

Concretely, this is what I needed to write the `--check` reporting in #47: it wants classification and git order, but not a chain. It is useful on its own for anything that inspects a versions directory, such as a CI lint or a migration linter.

## Notes for review

- Purely additive. No existing behavior changes and all pre-existing tests pass unmodified.
- `parse_versions_dir` raises `RuntimeError` when git history is unavailable, rather than returning `None` as the private helper does, because ordering cannot be recovered from the directory alone and a silent wrong order is worse than an error. Note this differs from `build_chain`, which first falls back to a chain file.
- Independent of #47; either can merge first. They touch neighbouring lines in `__init__.py`, the README API section and the end of the test file, so whichever lands second needs a trivial rebase.
- Tested on 3.11 / 3.12 / 3.13 / 3.14.
@mergify

mergify Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

@RizhongLin this pull request is now in conflict 😩

@mergify mergify Bot added the conflict label Aug 11, 2026
A hybrid migration is placed immediately after the revision it hardcodes,
so any dynamic migration added in between is re-parented onto it. That is
what keeps the chain linear when two branches fork from the same head, and
it is safe while none of those migrations has run yet.

Once one of them has been applied it is destructive. The hybrid lands
behind that database's recorded head, so `alembic upgrade head` walks down
from a revision that is already applied, finds nothing to do, and the
migration never runs. Nothing reports an error.

Git history alone cannot separate the two cases, so this does not change
how the chain is built, and neither mode of the report treats git history
as proof on its own:

  find_displaced_revisions(versions_dir) lists candidates, and
  `--check` prints them and exits 0, because a fork from the same head
  produces this shape legitimately.

  find_displaced_revisions(versions_dir, applied=...) keeps only the
  hybrids that a given database has provably walked past, and
  `--check --applied` exits non-zero on those. That is the form worth
  gating CI on.
@RizhongLin
RizhongLin force-pushed the report-migrations-chained-behind-head branch from 0b7a630 to 695dce8 Compare August 11, 2026 13:24
@mergify
mergify Bot had a problem deploying to Mergify Merge Protections August 11, 2026 13:24 Failure
@mergify mergify Bot removed the conflict label Aug 11, 2026
@jd

jd commented Aug 13, 2026

Copy link
Copy Markdown
Member

Thanks for this — the diagnosis is right, and the write-up is what made it
reviewable. The re-parenting is real: adding a hybrid does change the
down_revision of migrations that may already be applied, and I reproduced
both shapes you describe.

I'm going to close it, though, because I don't think either shape should be
solved here.

Hardcoding a down_revision that isn't the head is an authoring error.
The README section you wrote states the rule correctly — only hardcode onto
the current head — and I think that's where it ends. If you point a migration
at a revision your database is already past, you've asked Alembic for
something that can't work; catching that isn't this library's job.

The concurrent-branch case is the more interesting one, and it's the
reason I don't want --check in the package either. That shape is prevented
by the merge queue: branch B is tested against the rebased base, so the stale
hardcode is visible before it can land. But that only holds if the check runs
against the post-merge state and is re-run whenever the base moves — which
is a property of the merge queue, not of this package. A --check invoked on
a branch's own base passes cleanly and merges anyway, which is worse than no
check at all, because it reads as assurance. So the gate belongs in merge
queue configuration, alongside whatever else enforces the repo's invariants.

--applied I like even less: it couples a library that migrations import at
runtime to live database credentials in CI, per environment, to answer a
question git already answers.

None of that means the shape you hit isn't real. In a repo without such a
gate it drops a migration silently, exactly as you describe. I just don't
think this package is where it gets solved, so the chain builder stays as it
is.

One part is worth keeping and is independent of all of the above:
generate_chain_file raises a bare RuntimeError on a shallow clone, so the
CLI prints a traceback. The argparse conversion and that error message are a
real improvement on their own — happy to take them as a separate PR if you're
up for splitting them out.

@jd jd closed this Aug 13, 2026
mergify Bot pushed a commit that referenced this pull request Aug 13, 2026
)

A hybrid whose ``down_revision`` points at a revision that is no longer
the head is spliced in behind it, re-parenting the revisions in between.
That keeps the chain linear and single-headed, which is the property
this library exists to provide, but nothing pinned it -- so a future
change could turn it into a fork without a test noticing.

Two cases, both intended:

* the parent was already stale when the file was written (an authoring
  error: the rule is to hardcode only onto the current head)
* the parent was the head when the file was written, and another branch
  merged first (prevented upstream by the merge queue, which tests the
  branch against the rebased base)

The second case is driven through real branches and real merge commits
rather than a hand-written git order, so it also covers
``_get_git_commit_order`` producing the order that
``test_dynamic_inserted_before_hybrid_no_multiple_heads`` assumes.

Both walk the reconstructed chain, so they fail loudly on a fork rather
than only on a changed dict.

Refs #47

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Claude-Session-Id: 934c0f4e-8a8c-425a-826e-ed4ae67d6d2d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants