From a8069b02c888b1518cdc0f29d6a9765be1dace99 Mon Sep 17 00:00:00 2001 From: Dave Page Date: Tue, 11 Aug 2026 15:59:22 +0100 Subject: [PATCH 1/2] Document that changing the model breaks an existing chunk table Nothing warned users that the embedding model fixes the dimension of a chunk table. enable_vectorization() writes the provider's dimension into the embedding vector(N) column when it creates the table, and changing pgedge_vectorizer.model to a model of a different size afterwards leaves the worker unable to write. It handles that correctly, comparing the two dimensions before writing and marking the batch failed rather than storing anything wrong, but the documentation mentioned neither the constraint nor the recovery. There was no mention of dimensions at all in configuration.md, best_practices.md or troubleshooting.md. The correction matters more than the warning does. recreate_chunks() is the function a reader would reach for, and best_practices.md recommends it for "a complete chunk regeneration", but it cannot resolve a dimension change: it deletes the rows of the chunk table and leaves the column type untouched, so every requeued row fails exactly as before. Verified on a live table, where the column stayed vector(3) across recreate_chunks() and only became vector(5) after disable_vectorization() with drop_chunk_table and a fresh enable_vectorization(). Adds a warning admonition to the provider settings, a troubleshooting section giving the symptoms and both recovery routes, and three practice notes covering model choice, the rebuild and its provider cost. The quoted error text and every function and column named were checked against the source rather than written from memory. Also drops an em-dash from a neighbouring paragraph, per the house style. --- docs/best_practices.md | 9 ++++++ docs/configuration.md | 14 ++++++++ docs/troubleshooting.md | 71 ++++++++++++++++++++++++++++++++++++++++- 3 files changed, 93 insertions(+), 1 deletion(-) diff --git a/docs/best_practices.md b/docs/best_practices.md index a0ca4ff..0e1c845 100644 --- a/docs/best_practices.md +++ b/docs/best_practices.md @@ -34,3 +34,12 @@ Effective data management ensures clean operations and provides flexibility when - Use the `reprocess_chunks()` function to queue existing chunks that are missing embeddings. - Use the `recreate_chunks()` function for a complete chunk regeneration, which deletes all existing chunks first. - Each column gets independent chunk tables and triggers, so you can disable them selectively as needed. +- Settle on an embedding model before enabling vectorization, because the + model's dimension is fixed into the chunk table when the table is + created. +- Rebuild the vectorizer with `disable_vectorization()` and + `enable_vectorization()` after changing to a model of a different + dimension, since `recreate_chunks()` deletes chunk rows without + altering the column. +- Budget for the provider cost of re-embedding an entire table before + changing the model on a populated one. diff --git a/docs/configuration.md b/docs/configuration.md index 532e38e..5b595e0 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -14,6 +14,20 @@ These settings configure the connection to your embedding provider, including th | `pgedge_vectorizer.model` | `text-embedding-3-small` | Model name | No | No | No | | `pgedge_vectorizer.extra_headers` | (empty) | Semicolon-separated `key: value` HTTP headers added to all API requests | No | No | No | +!!! warning "The model fixes the vector dimension of a chunk table" + + The model determines how many dimensions the provider returns, and + `enable_vectorization()` fixes that number into the chunk table's + `embedding vector(N)` column when the table is created. Changing + `pgedge_vectorizer.model` to a model with a different dimension does + not migrate an existing chunk table. The background worker compares + the dimensions before writing, so nothing is corrupted, but it marks + the affected queue items `failed` with the message + `Dimension mismatch: model=N, table=M` and no new embeddings are + stored for that table until you act. The + [Troubleshooting](troubleshooting.md) document describes how to + recover. + ## Worker Settings These settings control the background workers that process the embedding queue, including concurrency, batch sizes, and retry behavior. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 3b86a3e..ec2a11b 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -13,7 +13,7 @@ SHOW shared_preload_libraries; ## Workers Not Processing After CREATE EXTENSION -Background workers start with PostgreSQL — before the extension is created. Workers automatically detect the extension using exponential backoff (checking every 5s, 10s, 20s, ... up to 5 minutes). After running `CREATE EXTENSION pgedge_vectorizer`, workers should discover it within seconds. +Background workers start with PostgreSQL, before the extension is created. Workers automatically detect the extension using exponential backoff (checking every 5s, 10s, 20s, ... up to 5 minutes). After running `CREATE EXTENSION pgedge_vectorizer`, workers should discover it within seconds. If workers don't start processing: @@ -64,3 +64,72 @@ SELECT * FROM pgedge_vectorizer.failed_items; ```sql SELECT pgedge_vectorizer.retry_failed(); ``` + +## Dimension Mismatch After Changing the Model + +Each chunk table stores its vectors in an `embedding vector(N)` column, +where N is fixed when `enable_vectorization()` creates the table. If you +change `pgedge_vectorizer.model` to a model returning a different number +of dimensions, the worker cannot write the new vectors into the existing +column, and embeddings stop being produced for that table. + +Nothing is corrupted when this happens. The worker compares the two +dimensions before it writes, so the existing embeddings are left intact +and no vector of the wrong size is ever stored. + +The affected queue items move to `failed` rather than being retried, +because retrying cannot succeed. Run the following query to identify +them: + +```sql +SELECT chunk_table, error_message, count(*) + FROM pgedge_vectorizer.queue + WHERE status = 'failed' + GROUP BY chunk_table, error_message; +``` + +An affected item reports `Dimension mismatch: model=N, table=M`, where N +is the dimension the configured model returned and M is the dimension +the chunk table expects. The server log carries a matching warning that +names the table. + +Restoring the previous model is the quicker of the two remedies, and is +the right one if the change was accidental: + +```sql +ALTER SYSTEM SET pgedge_vectorizer.model = 'text-embedding-3-small'; +SELECT pg_reload_conf(); +SELECT pgedge_vectorizer.retry_failed(); +``` + +Rebuilding the vectorizer keeps the new model and re-embeds the table +under it. Note that `recreate_chunks()` does not resolve a dimension +change, because that function deletes the rows of a chunk table without +altering the type of the column. Follow these steps instead: + +1. Set the new model and reload the configuration so that the dimension + detection uses the model you want. + + ```sql + ALTER SYSTEM SET pgedge_vectorizer.model = 'text-embedding-3-large'; + SELECT pg_reload_conf(); + ``` + +2. Drop the vectorizer together with its chunk table, which discards the + embeddings of the old dimension. + + ```sql + SELECT pgedge_vectorizer.disable_vectorization( + 'docs', 'body', drop_chunk_table => TRUE); + ``` + +3. Enable vectorization again, which detects the new dimension, creates + the chunk table to match, and queues every source row. + + ```sql + SELECT pgedge_vectorizer.enable_vectorization('docs', 'body'); + ``` + +Re-embedding calls the provider for every chunk in the table, so confirm +the cost against your provider's pricing before starting on a large +table. From 7a905cc421916c7ca19fd4b531a713230336b96c Mon Sep 17 00:00:00 2001 From: Dave Page Date: Tue, 11 Aug 2026 16:25:34 +0100 Subject: [PATCH 2/2] Address review of the model-change documentation Four corrections from CodeRabbit's review, all of them fair. The best practices bullet said to rebuild with disable_vectorization() and enable_vectorization() without naming drop_chunk_table. Left out, the chunk table survives, the column keeps its old dimension and the mismatch continues, so the bullet described a rebuild that does not work. The troubleshooting steps had the argument; the summary did not. The restore example hardcoded text-embedding-3-small while the prose said to restore the previous model. Anyone whose table was built by a different model would have followed it and stayed mismatched. It now carries a placeholder and points at the table=M figure in the error as the way to identify which model to go back to. The rebuild was written around a single docs/body example, but the model is one global setting while chunk tables are independent per column, so changing it strands every vectorizer whose dimension no longer matches. The steps now say to repeat them for each, and give a query listing them. The worker discovery paragraph promised detection "within seconds", which the backoff does not support once it has reached its five minute ceiling. It now states the real bound and mentions that a reload brings the check forward. That paragraph is also wrapped to the house width, having been one long line before. --- docs/best_practices.md | 10 ++++++---- docs/troubleshooting.md | 33 +++++++++++++++++++++++++++------ 2 files changed, 33 insertions(+), 10 deletions(-) diff --git a/docs/best_practices.md b/docs/best_practices.md index 0e1c845..f8cde5c 100644 --- a/docs/best_practices.md +++ b/docs/best_practices.md @@ -37,9 +37,11 @@ Effective data management ensures clean operations and provides flexibility when - Settle on an embedding model before enabling vectorization, because the model's dimension is fixed into the chunk table when the table is created. -- Rebuild the vectorizer with `disable_vectorization()` and - `enable_vectorization()` after changing to a model of a different - dimension, since `recreate_chunks()` deletes chunk rows without - altering the column. +- Rebuild the vectorizer with `disable_vectorization(..., + drop_chunk_table => TRUE)` and then `enable_vectorization()` after + changing to a model of a different dimension, repeating this for every + vectorized column. Neither `recreate_chunks()` nor a + `disable_vectorization()` that keeps the chunk table alters the column, + so neither resolves the mismatch. - Budget for the provider cost of re-embedding an entire table before changing the model on a populated one. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index ec2a11b..df509a0 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -13,7 +13,13 @@ SHOW shared_preload_libraries; ## Workers Not Processing After CREATE EXTENSION -Background workers start with PostgreSQL, before the extension is created. Workers automatically detect the extension using exponential backoff (checking every 5s, 10s, 20s, ... up to 5 minutes). After running `CREATE EXTENSION pgedge_vectorizer`, workers should discover it within seconds. +Background workers start with PostgreSQL, before the extension is +created. A worker checks for the extension on an exponential backoff, +waiting 5s, then 10s, then 20s, and so on up to a ceiling of 5 minutes. +After running `CREATE EXTENSION pgedge_vectorizer`, a worker discovers it +on its next check, so a database configured long before the extension was +created can wait up to 5 minutes. Reload the configuration to have the +workers check immediately. If workers don't start processing: @@ -94,14 +100,19 @@ the chunk table expects. The server log carries a matching warning that names the table. Restoring the previous model is the quicker of the two remedies, and is -the right one if the change was accidental: +the right one if the change was accidental. Substitute the model that +built the table rather than the name shown here, because setting any +other model leaves the dimensions mismatched: ```sql -ALTER SYSTEM SET pgedge_vectorizer.model = 'text-embedding-3-small'; +ALTER SYSTEM SET pgedge_vectorizer.model = 'the-previous-model'; SELECT pg_reload_conf(); SELECT pgedge_vectorizer.retry_failed(); ``` +The `table=M` figure in the error gives the dimension the chunk table +expects, so the model you restore must be one that returns M dimensions. + Rebuilding the vectorizer keeps the new model and re-embeds the table under it. Note that `recreate_chunks()` does not resolve a dimension change, because that function deletes the rows of a chunk table without @@ -130,6 +141,16 @@ altering the type of the column. Follow these steps instead: SELECT pgedge_vectorizer.enable_vectorization('docs', 'body'); ``` -Re-embedding calls the provider for every chunk in the table, so confirm -the cost against your provider's pricing before starting on a large -table. +Repeat those steps for every vectorized column, not just the one you +noticed. The model is a single global setting while chunk tables are +independent per column, so changing it affects every vectorizer whose +dimension no longer matches. The following query lists them: + +```sql +SELECT source_table, source_column, chunk_table + FROM pgedge_vectorizer.vectorizers + ORDER BY source_table, source_column; +``` + +Re-embedding calls the provider for every chunk in every table rebuilt, +so confirm the cost against your provider's pricing before starting.