ColumnSpec: compile a column fact once

[0x054]

Problem Statement

A Ferro model field declaration is one statement, but the facts it declares (primary key, autoincrement, nullability, uniqueness, storage type) are packed, unpacked, re-serialized, and re-parsed through four representations before reaching SchemaIR — and two of those facts (autoincrement, nullable) are derived twice, in different modules, with different defaulting rules. The rules can silently disagree: a str primary key auto-increments or not depending on which declaration syntax was used, and #153 was exactly this class of bug. Maintainers adding one column option must edit five modules in lockstep; the round-trip is only covered end-to-end.

Solution

Compile every column fact exactly once into a Column spec (now defined in CONTEXT.md): a frozen value object built from the field declaration at class-body time (provisional) and replaced during relationship resolution (resolved), mirroring the provisional/resolved registration epochs of ADR-0001. The SchemaIR compiler renders specs directly into the locked envelope shape; every runtime consumer of a column fact reads specs; the enriched-JSON-schema-dict interchange format is deleted. The two divergent autoincrement rules are first concentrated into one derivation site, then unified (ADR-0002): explicit declaration wins, else pk AND integer-typed, on both declaration paths.

User Stories

1. As a Ferro maintainer, I want each column fact derived in exactly one place, so that a defaulting-rule fix cannot silently miss a second derivation site (the [IR-P8.6] Unify the CREATE TABLE path onto the Python SchemaIR #153 failure mode).
2. As a Ferro maintainer, I want adding a new column option to be one edit in one module, so that I do not have to update the field packer, metaclass whitelist, schema enricher, and IR compiler in lockstep.
3. As a Ferro user, I want autoincrement to follow one rule regardless of whether I declare fields via Field()/FerroField or raw json_schema_extra, so that my schema does not change meaning with declaration syntax.
4. As a Ferro user with a non-integer primary key declared via raw json_schema_extra, I want autoincrement to default to false, so that my TEXT/UUID PK does not carry a nonsensical autoincrement flag into DDL.
5. As a Ferro user, I want relationship descriptors and shadow-FK typing to find my primary key even when it was declared via raw json_schema_extra, so that M2M join columns type from my real PK type instead of a string fallback.
6. As a Ferro maintainer, I want the spec object to be immutable, so that the current write-back mutation of shared field metadata and the "live reference, treat as read-only" hazard are structurally impossible.
7. As a test author, I want to unit-test column-fact derivation through build_column_specs without defining a model, connecting, or compiling an envelope, so that the defaulting matrix is pinned cheaply and exhaustively.
8. As a test author, I want spec-level assertions (spec.nullable, spec.db_type) instead of reaching into a schema dict's string keys, so that tests cross the module's interface rather than its implementation.
9. As an AI agent navigating the codebase, I want "what are this column's facts" answerable by reading one object from one module, so that I do not bounce across five files to trace one declaration.
10. As a Ferro maintainer, I want Pydantic's model_json_schema() consumed in exactly one module, so that Pydantic-version drift in schema output is quarantined to one seam.
11. As a Ferro maintainer, I want join tables to construct typed specs directly instead of hand-built schema-dict fragments, so that synthetic tables share the same single derivation path as real models.
12. As a Ferro maintainer, I want the recompile paths (envelope eviction, dirty registry, relationship resolution's second pass) to flow through one compile choke point that also refreshes the class-level specs, so that specs and persisted envelopes can never diverge.
13. As a Ferro user upgrading, I want the SchemaIR envelope byte-identical for every model except the one characterized autoincrement case, so that upgrade risk is limited to a documented edge.
14. As a Ferro user with existing databases, I want the changelog to state plainly whether the migrate planner will propose a change for raw-declared non-integer PKs, so that I can plan the upgrade.
15. As a reviewer, I want the pure refactor, the consumer migration, and the behaviour change in three separate commits (rebase-merged), so that a bisect can never land between "refactor broke it" and "the rule change broke it".
16. As a Ferro maintainer, I want the query-column set and enum-field registration derived from specs, so that no runtime attribute re-derives column facts from annotations independently.

Implementation Decisions

Settled in the 2026-07-08 grilling session (full task-level plan: docs/superpowers/plans/2026-07-09-columnspec-refactor.md):

• Output contract: SchemaIR envelope byte-identical through the refactor commits; the golden IR vectors are the conformance oracle. Exactly one characterized behaviour change lands in the final commit.
• Scope: ColumnSpec owns every per-column fact the IR compiler needs (ferro-declared and type-derived) plus the resolved Python type that runtime consumers need. Table-level facts (composite groups) stay in their own modules, validated against spec names.
• Lifecycle: frozen value objects; two epochs mirroring provisional/resolved registration (ADR-0001). Resolution replaces spec mappings for recompile targets — never mutates. Runtime consumers read the class attribute at use time.
• Interface (the module's whole public surface): ColumnSpec, build_column_specs(model_cls), fk_shadow_spec(...), pk_spec(...). Derivation rules (derive_autoincrement, derive_nullable) are module-private, one site each.
• The enriched-dict format is eliminated, not wrapped: the compiler and registration entry points take spec bundles; join tables construct specs; the schema-dict class attribute is deleted.
• Full consumer migration: save's PK assignment, PK-name lookup, both relationship-descriptor scans (one is a dead duplicate — deleted), shadow-FK typing, enum-field registration, and the query-column set all read specs.
• Unification (ADR-0002): autoincrement = explicit, else pk AND integer-typed, both paths; raw-path PKs become visible to runtime consumers. A transitional declared_via marker preserves today's behaviour during the two refactor commits and is deleted in the unification commit.
• Landing: one PR, three conventional commits (refactor:, refactor:, fix:); rebase-merge to preserve bisect boundaries.

Testing Decisions

• Highest seam first: the golden IR-vector contract suite is the primary oracle — if vectors pass unmodified after the refactor commits, every fact compiled correctly. No new seam needed for this; it already exists.
• One new unit seam: build_column_specs — the defaulting matrix (both declaration paths × pk/non-pk × integer/non-integer × explicit/inferred nullability) is pinned as fast unit tests with no DB, no connect, no envelope. Prior art: the metaclass-internals unit suite.
• Runtime consumers are gated by existing e2e suites (CRUD, relationships, identity map, auto-migrate) — no new tests needed for behaviour-preserving swaps; the suites' current green is the spec.
• The behaviour change gets its own pinned artifacts: a unit test asserting the unified rule and one new golden vector for a raw-declared non-integer PK.
• Good tests here assert facts through interfaces (spec fields, envelope payloads), never schema-dict string keys or module-global registries.

Out of Scope

• Any Rust-side change (the envelope contract is locked; both emitters and the migrate planner are untouched).
• Unifying the five annotation-unwrap implementations beyond the copies this refactor already touches (architecture-review candidate 7 remainder).
• The Registry module deepening (architecture-review candidate 2) and every other review candidate.
• Changing what the Alembic bridge consumes (it reads envelopes; unaffected).

Further Notes

• Domain term Column spec added to CONTEXT.md; ADR-0002 (unified column-fact derivation) is authored as part of the final commit, with two investigation findings to fill in: what the DDL emitter currently does with autoincrement=true on non-integer columns, and whether the migrate planner diffs autoincrement (determines the changelog's upgrade note).
• Origin: top-adjacent candidate from the 2026-07-08 architecture review; design stress-tested via grilling with all decisions recorded in the plan document.