Skip to content

Model selection: support per-table models and handle model change on populated embedding tables #27

Description

@dpage

Raised by @danolivo during review of #14.

Two related concerns about pgedge_vectorizer.model:

  1. Global, not per-table. The model is a single GUC applied to every table. Different datasets often want different embedding models, so a per-table (per-vectorizer) model setting would be more flexible.
  2. Changing the model on an existing embedding table. If a table already has embeddings of dimension N and the model is changed to one producing dimension M, subsequent inserts will mismatch the column dimension and error. There is currently no guard or migration path.

Action: design per-table model configuration and define behaviour when the model (and therefore embedding dimension) changes for an already-populated table — at minimum a clear error, ideally a guided re-embed/migration.

Pre-existing architecture concern; not introduced by #14.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions