fix: report migrations chained behind the deployed head - #47
Conversation
Merge Protections🔴 2 of 6 protections blocking · waiting on 👀 reviews and 🤖 CI
🔴 👀 Review RequirementsWaiting for
This rule is failing.
🟠 🤖 Continuous IntegrationWaiting for
Waiting checks:
|
There was a problem hiding this comment.
Pull request overview
Adds advisory + confirmed reporting for “hybrid” Alembic migrations whose hardcoded down_revision targets a dynamic revision that is no longer the newest predecessor, which can silently place a new migration behind an already-deployed database head. This extends the library with detection/reporting while intentionally leaving chain-building behavior unchanged.
Changes:
- Introduces
find_displaced_revisions(versions_dir, applied=None)andDisplacedRevisionto identify hybrids that displace later dynamic revisions (optionally filtering to those provably unreachable given an applied set). - Updates the CLI to support
--check(advisory, exit 0) and--check --applied <file|->(confirmed, exit non-zero on findings) viaargparse. - Documents the “only hardcode
down_revisiononto the current head” rule and how to check for violations.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/test_chain.py | Adds unit tests for displaced-revision detection and CLI --check/--applied behavior. |
| README.md | Documents the deployed-database hazard, the rule, and new CLI/Python checking workflows. |
| alembic_git_revisions/_chain.py | Adds DisplacedRevision and find_displaced_revisions() implementation. |
| alembic_git_revisions/init.py | Exposes new API and rewrites CLI parsing to add --check/--applied. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
f1505b0 to
0b7a630
Compare
|
Two things I am happy to change rather than defend, keeping them here rather than in the description since the description becomes the commit message on merge:
|
Tooling built around this library has to re-implement parsing and ordering, or reach into private helpers, to answer questions the library already answers internally: which migrations are dynamic, what order git puts them in, and what the generated artifact is called. This exports the pieces that answer those questions without building a chain: - `parse_versions_dir(versions_dir)` returns the migrations as `MigrationFile` objects in git add order. A thin wrapper over the existing git-order and parse steps. - `MigrationFile` is already a public class, it simply was not exported. - `CHAIN_FILENAME` so callers locating or cleaning up the generated file do not hardcode `"revision_chain.json"`. The private `_CHAIN_FILENAME` is removed rather than aliased: its only two references were internal to `_chain.py`, both updated here, and a leading-underscore name carries no compatibility guarantee. Concretely, this is what I needed to write the `--check` reporting in #47: it wants classification and git order, but not a chain. It is useful on its own for anything that inspects a versions directory, such as a CI lint or a migration linter. ## Notes for review - Purely additive. No existing behavior changes and all pre-existing tests pass unmodified. - `parse_versions_dir` raises `RuntimeError` when git history is unavailable, rather than returning `None` as the private helper does, because ordering cannot be recovered from the directory alone and a silent wrong order is worse than an error. Note this differs from `build_chain`, which first falls back to a chain file. - Independent of #47; either can merge first. They touch neighbouring lines in `__init__.py`, the README API section and the end of the test file, so whichever lands second needs a trivial rebase. - Tested on 3.11 / 3.12 / 3.13 / 3.14.
|
@RizhongLin this pull request is now in conflict 😩 |
A hybrid migration is placed immediately after the revision it hardcodes, so any dynamic migration added in between is re-parented onto it. That is what keeps the chain linear when two branches fork from the same head, and it is safe while none of those migrations has run yet. Once one of them has been applied it is destructive. The hybrid lands behind that database's recorded head, so `alembic upgrade head` walks down from a revision that is already applied, finds nothing to do, and the migration never runs. Nothing reports an error. Git history alone cannot separate the two cases, so this does not change how the chain is built, and neither mode of the report treats git history as proof on its own: find_displaced_revisions(versions_dir) lists candidates, and `--check` prints them and exits 0, because a fork from the same head produces this shape legitimately. find_displaced_revisions(versions_dir, applied=...) keeps only the hybrids that a given database has provably walked past, and `--check --applied` exits non-zero on those. That is the form worth gating CI on.
0b7a630 to
695dce8
Compare
|
Thanks for this — the diagnosis is right, and the write-up is what made it I'm going to close it, though, because I don't think either shape should be Hardcoding a The concurrent-branch case is the more interesting one, and it's the
None of that means the shape you hit isn't real. In a repo without such a One part is worth keeping and is independent of all of the above: |
) A hybrid whose ``down_revision`` points at a revision that is no longer the head is spliced in behind it, re-parenting the revisions in between. That keeps the chain linear and single-headed, which is the property this library exists to provide, but nothing pinned it -- so a future change could turn it into a fork without a test noticing. Two cases, both intended: * the parent was already stale when the file was written (an authoring error: the rule is to hardcode only onto the current head) * the parent was the head when the file was written, and another branch merged first (prevented upstream by the merge queue, which tests the branch against the rebased base) The second case is driven through real branches and real merge commits rather than a hand-written git order, so it also covers ``_get_git_commit_order`` producing the order that ``test_dynamic_inserted_before_hybrid_no_multiple_heads`` assumes. Both walk the reconstructed chain, so they fail loudly on a fork rather than only on a changed dict. Refs #47 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session-Id: 934c0f4e-8a8c-425a-826e-ed4ae67d6d2d
A hybrid migration (a static file whose hardcoded
down_revisionpoints at a dynamic revision) is placed immediately after its target, and every dynamic migration added in between is re-parented onto it. That is what keeps the chain linear when two branches fork from the same head, and it is exactly right while none of those migrations has run yet.It stops being right once one of them has been applied. I hit this on a project where a migration was added with a hardcoded
down_revisionpointing at a revision that was no longer the head:eeeenow sits behind the recorded head.alembic upgrade headwalks down fromdddd, finds every revision already applied, and does nothing. The migration never runs and nothing reports an error. It is only noticed later, when something depends on a column that was never created.Why this does not change the chain builder
My first attempt was to reject this at build time. That turns out to be wrong, and
test_dynamic_inserted_before_hybrid_no_multiple_headsis the reason: it describes branch A adding a dynamic migration and branch B adding a hybrid onto the same head, with A merging first. The resulting chain isaaaa -> bbbb -> dddd -> cccc. If a database deployed after A merged and before B did, it is sitting atcccc, andddddnever runs. Same mechanism, same outcome.So the benign case and the damaging one are the same shape in git history. What separates them is only whether a displaced revision has already been applied somewhere, which git cannot know. Refusing to build the chain would break the concurrent-branch case this library deliberately supports.
This PR therefore leaves chain building completely untouched and adds reporting instead.
What it adds
find_displaced_revisions(versions_dir, applied=None)returningDisplacedRevision(hybrid, target, displaced).applied: advisory. Lists every hybrid whose target is not the newest revision preceding it, including the benign fork case.applied(the revisions a database has actually run): keeps only hybrids that provably cannot execute, because something after them is already applied.The same split on the CLI, which matters more than it sounds.
--checkalone exits 0 and words its findings conditionally, because a tool with no database access cannot prove anything, and failing a build on the benign fork case would break the workflow this library exists to support:--check --applied <file|->narrows to what provably cannot run and exits non-zero. That is the form worth gating CI on:A README section stating the rule plainly: only hardcode
down_revisiononto the current head.Notes for review
build_chain,get_down_revisionand_build_chain_from_gitreturn exactly what they did before; all pre-existing tests pass unmodified.find_displaced_revisionscallsparse_versions_dirdirectly. Nothing else about the two changes interacts._climoves from manualsys.argvparsing toargparseto take the flag. The positional form is unchanged, but the exit code for a bad invocation moves from 1 to argparse's 2.--appliedfile, all exit 1 asalembic-git-revisions: error: .... This covers the pre-existing generate path too, which raised the sameRuntimeErroruncaught.down_revisioncontinue to work" is now qualified, since that is the sentence I had believed.