Expose an embedding staleness / coverage metric for vectorized tables

[dpage]

Summary

Embeddings are derived, eventually-consistent data: there is an inherent lag between a source change and the embedding being (re)generated by the async worker. Today there is no way for a user to see how in-sync the embeddings are with the source. Expose a staleness/coverage metric so users can understand and trust the state of their vectorized data.

Analogous to SQL Server's statistics-staleness information, which lets users (and the optimizer) judge whether stats can be trusted.

Motivation

• Users running RAG/search need to know whether results reflect recent source changes.
• Operators need to spot a stalled/backed-up worker (see the worker-coverage bug, Databases beyond num_workers are silently never processed #23) or a misconfigured provider.
• It is a cheap, high-value addition that directly reflects the eventually-consistent reality of the design.

Suggested metric(s)

Per vectorizer (source table + column → chunk table), expose via a view/function:

• Coverage: fraction of source rows that have current embeddings (e.g. rows with at least one chunk whose embedding IS NOT NULL).
• Backlog: count of queue items in pending / processing / failed for this chunk table.
• Lag: age of the oldest pending queue item (NOW() - min(created_at)).
• Last processed: max(processed_at) from the queue, as a freshness timestamp.

Much of this is already derivable from pgedge_vectorizer.queue (status, created_at, processed_at, chunk_table) and the chunk tables, so the building blocks exist.

Possible shape

A view such as pgedge_vectorizer.vectorizer_status keyed off the vectorizers registry (sql/pgedge_vectorizer--1.1.sql:21), joining queue aggregates and chunk-table coverage counts.

Notes

Found during an architecture review. Could also serve as the basis for a periodic reconciliation pass (related to the DELETE/TRUNCATE orphan issue, #24).