diff --git a/AGENTS.md b/AGENTS.md index bbd427b..53683f3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,6 +17,7 @@ - `apps/backend`:NestJS API、Agent 工作流、Prisma 数据层。 - `apps/frontend`:Next.js 前端演示应用。 +- `packages/chat-stream-protocol`:聊天 SSE 协议工具包(event envelope、SSE framing/parsing、terminal 判定、轻量 protocol UI helpers)。 - `packages/shared-types`:前后端共享类型定义。 - `infra`:本地依赖编排(如 `infra/docker-compose.yml`)。 - `data`:本地数据与运行期数据目录。 @@ -175,6 +176,8 @@ CI 参考: - 禁止新增“宽导出中枢”形态依赖。 - 业务域及其兼容根模块(`chat/agent/memory/glossary/rag`)禁止直接 import `modules/data/**` 实现路径。 - 业务域及其兼容根模块禁止依赖 `platform/data/data.module.ts`(`PlatformDataModule` 聚合入口)。 +- 能力边界门禁输出必须包含 RAG 专项 `ragImportReport`(blocked legacy active/direct implementation、allowed shared internal、canonical public entry)。 +- `conversation -> knowledge/**` 计数基线默认 `9`(可通过 `BACKEND_CAPABILITY_BOUNDARY_CONVERSATION_KNOWLEDGE_BASELINE` 覆盖)。 必跑检查: - `pnpm run backend:capability-boundary:check`(落地后) diff --git a/README.ja.md b/README.ja.md new file mode 100644 index 0000000..64f3bbd --- /dev/null +++ b/README.ja.md @@ -0,0 +1,327 @@ +# Text2SQL + +[English](README.md) | [简体中文](README.zh-CN.md) | 日本語 + +Text2SQL は、企業のデータ分析 QA シナリオに向けたフルスタックの学習・デモプロジェクトです。自然言語の質問を、ガバナンス可能で、実行可能で、再生可能な SQL に変換し、データソース接続、セマンティック知識、権限ガバナンス、実行証跡、フロントエンド体験を 1 本のワークフローとしてつなぎます。 + +![Text2SQL プラットフォーム概要](assets/readme-platform-overview.png) + +## プロジェクト背景 + +業務チームは「何を聞きたいか」は分かっていても、そのデータがどのテーブルにあるのか、指標定義が何か、SQL をどう書くべきかまでは把握していないことがよくあります。質問をそのまま LLM に渡すだけでも十分ではありません。モデルはスキーマを誤って推測し、業務用語の文脈を欠き、テーブル権限を自然に守れず、なぜその SQL が生成されたのかを説明しにくいからです。 + +このプロジェクトは、SQL を生成するだけの Prompt demo ではありません。実運用に近い制約を前提にした Text2SQL プラットフォームのプロトタイプです。 + +- ユーザーはデータソースを起点に質問し、システムはセッション、ワークスペース、データ権限を紐付けます。 +- Agent は SQL 生成前に schema、用語、過去サンプル、セマンティック資産を検索します。 +- 生成された SQL は read-only、安全性、権限、方言、実行チェックを通過する必要があります。 +- 各実行には `runId` があり、trace、RAG evidence、delivery artifact、replay を追跡できます。 +- フロントエンドとバックエンドは共有型と SSE プロトコルによって、同期レスポンス、ストリーミング、実行詳細の契約を揃えます。 + +## 解決する課題 + +1. **自然言語と SQL のコンテキスト差分** + ユーザーは「売上」「アクティブ顧客」「直近 30 日のトレンド」と話しますが、データベースにはテーブル、カラム、外部キー、指標、業務用語があります。このプロジェクトは RAG、semantic spine、glossary、modeling workspace によって、SQL 生成前の証跡として文脈を整えます。 + +2. **制御しづらい LLM 生成** + Text2SQL v2 runtime は明示的な LangGraph オーケストレーションを使います。intake、retrieve、assemble-context、semantic-plan、generate-sql、validate、correct、execute、answer を観測可能な段階として分け、すべてを 1 つの Prompt に押し込めません。 + +3. **データ権限と安全な実行** + クエリは「動く」だけでは不十分です。システムは workspace datasource binding、table-permissions、policyVersion をガバナンスの主軸とし、不確実、未承認、安全に解析できない場合は fail-closed に倒します。 + +4. **障害診断の難しさ** + すべての分析実行は `runId` を中心に証跡を保存します。同期レスポンス、stream finish、run view、RAG replay は同じ delivery contract を参照し、RAG ヒット、SQL 生成、修正、実行、フロントエンド表示を振り返りやすくします。 + +5. **複数データソースのデモ閉ループ** + SQLite、MySQL、PostgreSQL、CSV、Excel に対応し、`/data-sources`、`/chat`、`/settings`、`/modeling` などのフロントエンドワークベンチでデモと拡張を続けられます。 + +## 機能プレビュー + +### セマンティックモデリングワークベンチ + +![セマンティックモデリングワークベンチ](assets/screenshots/01-modeling-erd-workbench.png) + +データ関係図は、物理テーブル構造を運用可能なセマンティック資産へ引き上げます。左側は Models / Views の資産ツリー、中央は ERD キャンバス、右側では選択モデルのフィールド、リレーション、プレビュー文脈を確認できます。ユーザーはデータベース同期、自動レイアウト、リレーション管理、Modeling Draft 保存を行い、チェック通過後に active 版として公開できます。 + +### データソース接続ウィザード + +![データソース接続ウィザード](assets/screenshots/02-datasource-create-wizard.png) + +データソースページは、データベースやファイルをワークスペースに接続します。ウィザードは CSV、Excel、SQLite、MySQL、PostgreSQL に対応します。作成または編集時には現在のワークスペースに自動で紐付き、idempotency key により重複送信を防ぎます。作成後はそのまま分析会話を開始することも、テーブル選択やモデリング初期化に進むこともできます。 + +### ChatBI ワークベンチ + +![ChatBI ワークベンチ](assets/screenshots/03-chat-answer-result.png) + +Chat ページは自然言語分析のメイン入口です。各セッションはデータソースとモデル設定に紐付き、サイドバーはデータソースごとに履歴セッションを分離します。メイン領域にはユーザー質問、Agent の実行段階、最終回答、テーブル証跡、Save as View の入口が表示されます。回答は単なるテキストではなく、`validation`、実行サマリ、artifact を含む構造化された delivery result です。 + +### SQL 証跡リプレイ + +![SQL 証跡リプレイ](assets/screenshots/04-chat-sql-evidence.png) + +同じ ChatBI 結果から SQL 証跡タブに切り替え、回答生成に使われた SQL を確認できます。このビューは人による検証、デバッグリプレイ、ガバナンス監査に役立ちます。ユーザーは生成されたクエリ、ソート、集約、フィールド選択が業務期待と合っているかを確認できます。 + +### チャート結果 + +![チャート結果](assets/screenshots/05-chat-chart-result.png) + +結果 artifact はチャートとしても投影できます。この例では支払い方法の比率をグラフ表示しています。Answer、View SQL、Chart は同じ実行証跡を共有するため、テキスト回答、SQL、可視化の結果がずれることを避けられます。 + +## 設計思想 + +- **生成より先に証跡を整える**:SQL 生成前に schema、用語、リレーション、権限、過去例を typed context として構成します。 +- **Prompt 連結よりグラフオーケストレーション**:重要な段階は runtime node として存在し、観測、テスト、ストリーミング投影、局所的な差し替えがしやすくなります。 +- **ガバナンスを主経路に組み込む**:workspace、datasource、table-permissions、安全検証は後付けではなくワークフローの一部です。 +- **黙って劣化しない実用性**:RAG lane は個別に timeout や degrade できますが、その理由は evidence に記録されます。 +- **有界な修正ループ**:SQL 修正は予算付きの閉ループであり、無制限の agent retry を避けます。 +- **安定した契約**:shared-types と chat-stream-protocol がフロントエンドとバックエンドの境界を担い、sync、stream、replay のずれを抑えます。 + +## アーキテクチャ設計 + +### Monorepo 構成 + +```text +apps/backend NestJS API、Text2SQL runtime、ガバナンス、知識、プラットフォーム機能 +apps/frontend Next.js フロントエンドワークベンチ +packages/shared-types フロントエンド/バックエンド共有型 +packages/chat-stream-protocol SSE envelope、parser、terminal guard、UI projection helpers +infra PostgreSQL、Redis、Nginx のローカル編成 +data ローカルアップロード、SQLite、実行時データ +docs ソリューション、標準、トラブルシュート、理解ドキュメント +``` + +### システム概要 + +```mermaid +flowchart LR + User["Browser"] --> Gateway["Nginx gateway
localhost:3000"] + Gateway --> Frontend["Frontend
Next.js :3001"] + Gateway --> Backend["Backend
NestJS :3002"] + + Backend --> Conversation["conversation
chat + text2sql + delivery"] + Backend --> Governance["governance
workspace + datasource + table-permissions"] + Backend --> Knowledge["knowledge
RAG + glossary + semantic spine + graph"] + Backend --> Platform["platform
persistence + query + cache + config + observability"] + + Conversation --> Governance + Conversation --> Knowledge + Conversation --> Platform + Governance --> Platform + Knowledge --> Platform + + Platform --> Postgres["PostgreSQL"] + Platform --> Redis["Redis"] + Platform --> Datasources["SQLite / MySQL / PostgreSQL / CSV / Excel"] +``` + +### バックエンドの 4 つの能力ドメイン + +| ドメイン | ディレクトリ | 役割 | +| --- | --- | --- | +| `conversation` | `apps/backend/src/modules/conversation` | 会話入口、Text2SQL workflow、LangGraph runtime、delivery contract | +| `governance` | `apps/backend/src/modules/governance` | ワークスペース、データソース紐付け、テーブル権限、ユーザー、設定ガバナンス | +| `knowledge` | `apps/backend/src/modules/knowledge` | RAG 検索、セマンティック資産、用語、メモリ、グラフ、モデリング文脈 | +| `platform` | `apps/backend/src/modules/platform` | 永続化、クエリ実行、キャッシュ、設定、観測、read-model guard | + +クロスドメイン依存は次の方向に制限されています。 + +```text +conversation -> governance | knowledge | platform +governance -> platform +knowledge -> platform +platform -> no business-domain dependency +``` + +### Text2SQL v2 ランタイム + +現在の主経路は次の通りです。 + +```text +Text2SQLWorkflowRunner + -> RunV2LangGraphStage + -> Text2SqlV2LangGraphRunnerService +``` + +1 回の分析質問は、おおむね次のライフサイクルを通ります。 + +```mermaid +flowchart TD + A["intake
質問タイプとリスクを識別"] --> B{"route"} + B -- "text_to_sql / metadata" --> C["retrieve
RAG 検索"] + B -- "general / unsafe / unsupported" --> I["answer"] + C --> D["assemble-context
selected context を構成"] + D --> E["semantic-plan
セマンティック計画とルーティング"] + E -- "ready" --> F["generate-sql"] + E -- "clarify / direct answer / fail closed" --> I + F --> G["validate
read-only、安全、権限、方言、dry-run"] + G -- "pass" --> H["execute"] + G -- "correctable" --> J["correct"] + G -- "terminal" --> I + J -- "retry_generation" --> F + J -- "terminal" --> I + H --> I["answer
delivery + evidence + artifact"] +``` + +### RAG とセマンティックコンテキスト + +RAG は単なるテキスト連結ではありません。現在の設計では次を重視しています。 + +- manifest-first semantic asset preparation +- lexical、dense、graph の 3 つの retrieval lane +- fusion と rerank より前に権限フィルタリング +- RRF fusion と 2 段階 rerank +- `selected_context`、`degradeReasons`、`riskTags` を実行証跡へ記録 +- `runId` が trace、delivery、replay を貫通 + +## インストールと起動 + +### 必要環境 + +- Node.js 20 または互換バージョン +- pnpm 10.x。このリポジトリは `pnpm@10.33.0` を宣言しています +- Docker と Docker Compose + +### 1. 依存関係をインストール + +```bash +pnpm install +``` + +### 2. ローカル基盤を起動 + +```bash +docker compose -f infra/docker-compose.yml up -d +``` + +このコマンドは次を起動します。 + +- Nginx gateway: `http://localhost:3000` +- PostgreSQL: `localhost:5432` +- Redis: `localhost:6379` + +### 3. 環境変数を初期化 + +```bash +cp apps/backend/.env.example apps/backend/.env +cp apps/frontend/.env.example apps/frontend/.env +``` + +バックエンドの主要設定は `apps/backend/.env` にあります。 + +- `PORT=3002` +- `POSTGRES_HOST/POSTGRES_PORT/POSTGRES_DB/POSTGRES_USER/POSTGRES_PASSWORD` +- `REDIS_URL=redis://localhost:6379` +- `LLM_PROVIDER`、`LLM_BASE_URL`、`LLM_API_KEY`、`LLM_MODEL` +- `EMBEDDING_PROVIDER`、`EMBEDDING_BASE_URL`、`EMBEDDING_API_KEY`、`EMBEDDING_MODEL` +- `RERANK_*` は任意で、未設定の場合は LLM provider 設定にフォールバックできます + +フロントエンドはデフォルトで同一オリジンの `/api` を使うため、通常 `NEXT_PUBLIC_API_BASE_URL` を変更する必要はありません。 + +### 4. Prisma を準備 + +```bash +pnpm --filter @text2sql/backend run prisma:generate +pnpm --filter @text2sql/backend exec node scripts/prisma-with-database-url.cjs migrate deploy +``` + +空のデータベースから migration を完全に再生できるか検証する場合は、独立したテスト DB で実行します。 + +```bash +DATABASE_URL=postgresql://admin:admin@localhost:5432/text2sql_ci \ +pnpm --filter @text2sql/backend run prisma:verify-empty-db +``` + +### 5. フロントエンドとバックエンドを起動 + +```bash +pnpm dev +``` + +デフォルト URL: + +- 統一入口: `http://localhost:3000` +- データソース入口: `http://localhost:3000/data-sources` +- フロントエンド直接デバッグ: `http://localhost:3001` +- バックエンド health check: `http://localhost:3002/health` + +### 6. クイック smoke + +```bash +node tests/smoke/nginx-dev-gateway-smoke.mjs +``` + +## よく使うワークフロー + +### データソースから質問まで + +1. `http://localhost:3000/data-sources` を開く +2. SQLite、MySQL、PostgreSQL、CSV、Excel のデータソースを作成または選択する +3. 現在の workspace に紐付け、セッションに入る +4. `/chat` で自然言語の質問を入力する +5. 回答、SQL、実行結果、デバッグ証跡を確認する + +### 設定とガバナンス + +- `/settings`: LLM model、RAG configuration、RAG run、system users などの設定入口 +- `/glossary`: 業務用語の管理 +- `/modeling`: データソースモデリング、リレーション、セマンティックビュー +- `/prompts`: プロンプトテンプレート管理 + +## 品質ゲート + +リポジトリ全体: + +```bash +pnpm run format:check +pnpm run test +pnpm run build +``` + +バックエンド: + +```bash +pnpm --filter @text2sql/backend run lint +pnpm --filter @text2sql/backend run test +pnpm --filter @text2sql/backend run build +pnpm --filter @text2sql/backend run prisma:verify-empty-db +``` + +フロントエンド: + +```bash +pnpm --filter @text2sql/frontend run lint +pnpm --filter @text2sql/frontend run test +pnpm --filter @text2sql/frontend run build +``` + +専用ゲート: + +```bash +pnpm run governance:terminology:check +pnpm run backend:capability-boundary:check +pnpm run text2sql:no-legacy-compat:check +pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate +pnpm --filter @text2sql/backend run collect:text2sql-v2-focused-coverage-gate +pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate +``` + +## 重要なルール + +- Prisma の構造変更は必ず `apps/backend/prisma/schema.prisma` から始め、migration は Prisma CLI で生成します。 +- `apps/backend/prisma/migrations/*/migration.sql` を手書きまたは手動編集してはいけません。 +- テーブル構造変更後は `pnpm --filter @text2sql/backend run prisma:generate` を実行します。 +- フロントエンドのインタラクティブコントロールは、shadcn-ui とプロジェクト内の業務ラッパーを優先して再利用します。 +- ガバナンスの active narrative は `workspace datasource binding`、`table-permissions`、`policyVersion` のみを使います。 + +## 詳細ドキュメント + +- `AGENTS.md`: リポジトリ実行入口、ハード境界、品質ゲート +- `docs/text2sql-architecture-and-flow-2026-04-29.md`: 現在のアーキテクチャと Text2SQL メインフロー +- `docs/rag-understanding/text2sql-rag-end-to-end-understanding.md`: Text2SQL + RAG のエンドツーエンド解説 +- `docs/rag-understanding/text2sql-rag-runid-replay-handbook.md`: runId replay と診断 +- `docs/rag-understanding/text2sql-rag-local-learning-lab.md`: ローカル学習ラボ +- `docs/standards/backend-prisma-migration-spec.md`: Prisma migration standard +- `docs/standards/frontend-react-shadcn-spec.md`: frontend React + shadcn standard +- `docs/standards/llm-stream-tool-migration-spec.md`: LLM stream と tool calling migration standard +- `docs/standards/governance-terminology-spec.md`: governance terminology hard-cut standard +- `docs/standards/backend-business-capability-topology-spec.md`: backend capability topology standard diff --git a/README.md b/README.md index 8e57894..edbb1cd 100644 --- a/README.md +++ b/README.md @@ -1,373 +1,327 @@ -# text2sql - -Text2SQL 学习演示版(阶段0-3路线)的单仓项目。 - -## 技术栈 -- 后端:NestJS + TypeScript + Prisma -- Agent:Text2SQL v2 LangGraph runtime(`intake -> retrieve -> assemble-context -> semantic-plan -> generate-sql -> validate -> correct? -> execute -> answer`,Phase A `delegation=0`),由 `conversation/text2sql` 统一入口驱动 -- 前端:Next.js + React + Tailwind CSS v4 + shadcn-ui -- 查询数据:SQLite / MySQL / PostgreSQL / CSV / Excel(会话绑定数据源路由) -- 功能数据:Redis 缓冲 + PostgreSQL 持久化 - -## 项目规范 -- 前端重写规范:`docs/standards/frontend-react-shadcn-spec.md` -- 后端迁移规范:`docs/standards/backend-prisma-migration-spec.md` -- R1 门禁与灰度规范:`docs/standards/r1-gate-and-rollout-spec.md` -- LLM 流式与 Tool Calling 迁移规范:`docs/standards/llm-stream-tool-migration-spec.md` -- 治理术语硬切规范:`docs/standards/governance-terminology-spec.md` -- 后端能力域拓扑规范:`docs/standards/backend-business-capability-topology-spec.md` -- 前端重写需求:`docs/brainstorms/2026-04-10-frontend-react-shadcn-rewrite-requirements.md` - -## Text2SQL + RAG 全流程理解文档(2026-04-21 基线) -- 主白皮书(请求到交付):`docs/rag-understanding/text2sql-rag-end-to-end-understanding.md` -- runId 回放手册(trace/replay/delivery):`docs/rag-understanding/text2sql-rag-runid-replay-handbook.md` -- 本地实验剧本(学习闭环):`docs/rag-understanding/text2sql-rag-local-learning-lab.md` -- 文档合同检查脚本:`node scripts/check-docs-rag-understanding.mjs` -- 文档合同 smoke:`node tests/smoke/docs-rag-understanding-contract-smoke.mjs` - -## 后端能力域拓扑(迁移中) -- 顶层能力域采用:`conversation`、`governance`、`knowledge`、`platform`。 -- 依赖方向固定:`conversation -> governance|knowledge|platform`,`governance|knowledge -> platform`。 -- 迁移阶段允许兼容入口(re-export/wrapper)短期存在,但禁止引入新的跨域实现细节直连。 -- 详细规则见:`docs/standards/backend-business-capability-topology-spec.md`。 - -### 数据库结构改动铁律(必须遵守) -- 禁止手写或手改 `apps/backend/prisma/migrations/*/migration.sql`。 -- 先改 `apps/backend/prisma/schema.prisma`,再执行 `pnpm --filter @text2sql/backend run prisma:migrate --name ` 生成迁移。 -- 每次结构变更必须执行 `pnpm --filter @text2sql/backend run prisma:generate`。 - -## 目录结构 +# Text2SQL + +English | [简体中文](README.zh-CN.md) | [日本語](README.ja.md) + +Text2SQL is a full-stack learning and demo project for enterprise-style data question answering. It turns natural-language questions into governed, executable, replayable SQL, and connects datasource onboarding, semantic knowledge, permission governance, runtime evidence, and frontend interaction into one complete workflow. + +![Text2SQL platform overview](assets/readme-platform-overview.png) + +## Project Background + +Business teams often know what they want to ask, but not which table contains the data, how a metric is defined, or how the SQL should be written. Sending the question directly to an LLM is not enough either: the model can guess the schema incorrectly, miss business terminology, bypass table permissions, and leave little explanation for why a SQL statement was produced. + +This project is not a prompt-only SQL generator. It is a Text2SQL platform prototype shaped around production-like constraints: + +- Users ask questions from a datasource context, while the system binds sessions, workspaces, and data permissions. +- Before SQL generation, the agent retrieves schema, glossary terms, historical examples, and semantic assets. +- Generated SQL must pass read-only, safety, permission, dialect, and execution checks. +- Each run has a `runId` for tracing, RAG evidence, delivery artifacts, and replay. +- Frontend and backend stay aligned through shared types and an SSE protocol package for sync responses, streaming responses, and run details. + +## Problems Solved + +1. **The context gap between natural language and SQL** + Users say "revenue", "active customers", or "trend over the last 30 days"; databases expose tables, columns, foreign keys, metrics, and business terms. The project uses RAG, semantic spine, glossary, and modeling workspace assets to turn that context into evidence before SQL generation. + +2. **Uncontrolled LLM generation** + The Text2SQL v2 runtime uses explicit LangGraph orchestration. Intake, retrieval, context assembly, semantic planning, SQL generation, validation, correction, execution, and answer delivery are observable stages instead of one large prompt. + +3. **Data permissions and safe execution** + A query is not accepted merely because it runs. Governance is centered on workspace datasource binding, table-permissions, and policyVersion. The execution path favors fail-closed behavior when the system is uncertain, unauthorized, or unable to parse safely. + +4. **Hard-to-debug failures** + Every analytical run is saved around a `runId`. Sync responses, streaming finish events, run views, and RAG replay all point toward the same delivery contract, making it easier to inspect retrieval hits, SQL generation, correction, execution, and frontend rendering. + +5. **End-to-end demos across multiple datasource types** + The project supports SQLite, MySQL, PostgreSQL, CSV, and Excel datasources, with frontend workbench pages such as `/data-sources`, `/chat`, `/settings`, and `/modeling` for demos and continued extension. + +## Feature Preview + +### Semantic Modeling Workbench + +![Semantic modeling workbench](assets/screenshots/01-modeling-erd-workbench.png) + +The relationship graph turns physical table structure into manageable semantic assets. The left panel contains the Models / Views asset tree, the center hosts the ERD canvas, and the selected model context can be inspected through fields, relationships, and previews. Users can sync databases, auto-layout the graph, maintain relationships, save a Modeling Draft, and publish it as the active version after checks pass. + +### Datasource Onboarding Wizard + +![Datasource onboarding wizard](assets/screenshots/02-datasource-create-wizard.png) + +The datasource page connects databases and files to a workspace. The wizard supports CSV, Excel, SQLite, MySQL, and PostgreSQL. Creating or editing a datasource automatically binds it to the current workspace, and idempotency keys prevent duplicate submissions. After creation, users can start a data question session directly or continue with table selection and modeling initialization. + +### ChatBI Workbench + +![ChatBI workbench](assets/screenshots/03-chat-answer-result.png) + +The Chat page is the main natural-language analytics entry point. Each session is bound to a datasource and model configuration, while the sidebar separates historical sessions by datasource. The main area shows the user question, agent runtime stages, final answer, table evidence, and the save-as-view entry point. The answer is not plain text only; it is a structured delivery result with `validation`, execution summary, and artifacts. + +### SQL Evidence Replay + +![SQL evidence replay](assets/screenshots/04-chat-sql-evidence.png) + +The same ChatBI result can be switched to the SQL evidence tab to inspect the SQL used for the answer. This view supports human verification, debugging replay, and governance audit: users can confirm whether the generated query, ordering, aggregation, and selected fields match business expectations. + +### Chart Result + +![Chart result](assets/screenshots/05-chat-chart-result.png) + +Result artifacts can also be projected as charts. The example renders payment-method share as a visual result. Answer, View SQL, and Chart all share evidence from the same run, avoiding drift between textual output, SQL, and visualization. + +## Design Philosophy + +- **Evidence before generation**: schema, glossary terms, relationships, permissions, and examples are organized as typed context before SQL is generated. +- **Graph orchestration over prompt chaining**: important stages exist as runtime nodes, making them easier to observe, test, stream, and replace locally. +- **Governance built into the main path**: workspace, datasource, table-permissions, and safety validation are part of the workflow, not after-the-fact patches. +- **Useful without silent degradation**: RAG lanes may time out or degrade independently, but the reasons are written into evidence instead of hidden. +- **Bounded correction**: SQL correction is a budgeted loop, avoiding unbounded agent retries. +- **Stable contracts**: shared-types and chat-stream-protocol packages define the frontend/backend boundary so sync, stream, and replay behavior do not drift apart. + +## Architecture + +### Monorepo Structure + ```text -apps/backend NestJS API + Agent 工作流 -apps/frontend Next.js 演示页面 -packages/shared-types 前后端共享类型 -infra/docker-compose.yml Redis/PostgreSQL 本地依赖 -vibe/plain 需求、计划、评测与运维文档 +apps/backend NestJS API, Text2SQL runtime, governance, knowledge, and platform capabilities +apps/frontend Next.js frontend workbench +packages/shared-types Shared frontend/backend types +packages/chat-stream-protocol SSE envelope, parser, terminal guard, and UI projection helpers +infra Local PostgreSQL, Redis, and Nginx orchestration +data Local uploads, SQLite files, and runtime data +docs Solutions, standards, troubleshooting, and understanding documents ``` -## 本地启动 -1. 安装依赖 +### System Overview + +```mermaid +flowchart LR + User["Browser"] --> Gateway["Nginx gateway
localhost:3000"] + Gateway --> Frontend["Frontend
Next.js :3001"] + Gateway --> Backend["Backend
NestJS :3002"] + + Backend --> Conversation["conversation
chat + text2sql + delivery"] + Backend --> Governance["governance
workspace + datasource + table-permissions"] + Backend --> Knowledge["knowledge
RAG + glossary + semantic spine + graph"] + Backend --> Platform["platform
persistence + query + cache + config + observability"] + + Conversation --> Governance + Conversation --> Knowledge + Conversation --> Platform + Governance --> Platform + Knowledge --> Platform + + Platform --> Postgres["PostgreSQL"] + Platform --> Redis["Redis"] + Platform --> Datasources["SQLite / MySQL / PostgreSQL / CSV / Excel"] +``` + +### Four Backend Capability Domains + +| Domain | Directory | Responsibility | +| --- | --- | --- | +| `conversation` | `apps/backend/src/modules/conversation` | Chat entry points, Text2SQL workflow, LangGraph runtime, delivery contract | +| `governance` | `apps/backend/src/modules/governance` | Workspaces, datasource binding, table permissions, users, and settings governance | +| `knowledge` | `apps/backend/src/modules/knowledge` | RAG retrieval, semantic assets, glossary, memory, graph, and modeling context | +| `platform` | `apps/backend/src/modules/platform` | Persistence, query execution, cache, configuration, observability, and read-model guards | + +Cross-domain dependencies are intentionally constrained: + +```text +conversation -> governance | knowledge | platform +governance -> platform +knowledge -> platform +platform -> no business-domain dependency +``` + +### Text2SQL v2 Runtime + +The current main runtime path is: + +```text +Text2SQLWorkflowRunner + -> RunV2LangGraphStage + -> Text2SqlV2LangGraphRunnerService +``` + +A typical analytical question follows this lifecycle: + +```mermaid +flowchart TD + A["intake
classify question type and risk"] --> B{"route"} + B -- "text_to_sql / metadata" --> C["retrieve
RAG retrieval"] + B -- "general / unsafe / unsupported" --> I["answer"] + C --> D["assemble-context
build selected context"] + D --> E["semantic-plan
semantic plan and routing"] + E -- "ready" --> F["generate-sql"] + E -- "clarify / direct answer / fail closed" --> I + F --> G["validate
read-only, safety, permission, dialect, dry-run"] + G -- "pass" --> H["execute"] + G -- "correctable" --> J["correct"] + G -- "terminal" --> I + J -- "retry_generation" --> F + J -- "terminal" --> I + H --> I["answer
delivery + evidence + artifact"] +``` + +### RAG And Semantic Context + +RAG is not simple text concatenation. The current design emphasizes: + +- manifest-first semantic asset preparation +- lexical, dense, and graph retrieval lanes +- permission filtering before fusion and reranking +- RRF fusion and two-stage rerank +- `selected_context`, `degradeReasons`, and `riskTags` written into runtime evidence +- `runId` threaded through trace, delivery, and replay + +## Installation And Run + +### Requirements + +- Node.js 20 or compatible +- pnpm 10.x; this repository declares `pnpm@10.33.0` +- Docker and Docker Compose + +### 1. Install Dependencies + ```bash pnpm install ``` -2. 启动基础依赖(Redis/PostgreSQL + Nginx 统一入口网关) +### 2. Start Local Infrastructure + ```bash docker compose -f infra/docker-compose.yml up -d ``` -3. 配置环境变量 +This starts: + +- Nginx gateway: `http://localhost:3000` +- PostgreSQL: `localhost:5432` +- Redis: `localhost:6379` + +### 3. Initialize Environment Variables + ```bash cp apps/backend/.env.example apps/backend/.env cp apps/frontend/.env.example apps/frontend/.env ``` -关键配置(`apps/backend/.env`): -- `PORT=3002`(后端内部开发端口) -- `CORS_ALLOWED_ORIGINS=http://localhost:3000`(默认统一入口 origin) -- `POSTGRES_HOST=localhost` -- `POSTGRES_PORT=5432` -- `POSTGRES_DB=text2sql` -- `POSTGRES_USER=admin` -- `POSTGRES_PASSWORD=admin` -- `POSTGRES_SCHEMA=public` -- `DATABASE_URL=`(若配置则优先;未配置时后端会自动使用 `POSTGRES_*` 组装) -- `DATASOURCE_UPLOAD_DIR=`(默认 `data/uploads/datasources`) -- `DATASOURCE_UPLOAD_MAX_BYTES=10485760`(上传上限,默认 10MB) -- `DATASOURCE_CONNECT_TIMEOUT_MS=5000` -- `DATASOURCE_QUERY_TIMEOUT_MS=10000` -- `DATASOURCE_SECRET_KEY=`(生产环境务必配置自定义值) -- `LLM_PROVIDER=volcengine`(或 siliconflow/minimax) -- `LLM_BASE_URL=`(支持 `https://xxx`、`https://xxx/v1` 或完整 `/chat/completions` 路径) -- `LLM_API_KEY=` -- `LLM_MODEL=` -- `LLM_MOCK_MODE=false`(联调真实模型时保持 false) -- `EMBEDDING_PROVIDER=`(Text2SQL v2 dense retrieval 的 embedding provider) -- `EMBEDDING_BASE_URL=` -- `EMBEDDING_API_KEY=` -- `EMBEDDING_MODEL=text-embedding-3-small` -- `EMBEDDING_DIMENSIONS=` -- `EMBEDDING_VECTOR_VERSION=v1` -- `RERANK_PROVIDER=`(可选,默认回退到 `LLM_PROVIDER`) -- `RERANK_BASE_URL=`(可选,默认回退到 `LLM_BASE_URL`) -- `RERANK_API_KEY=`(可选,默认回退到 `LLM_API_KEY`) -- `RERANK_MODEL=`(可选,默认回退到 `LLM_MODEL`) -- `RERANK_TIMEOUT_MS=`(可选,默认回退到 `LLM_TIMEOUT_MS`) -- `/settings` 页面职责: - - `LLM 模型`:provider/model 目录治理 - - `RAG 配置`:Embedding / Rerank 运行配置(settings first, env fallback),支持 `检测草稿(dry-check)` 与 `已保存配置检测(persisted-check)` - - `RAG 运行`:运行态观测与回放证据 -- `LANGSMITH_TRACING=true|false`(是否启用 LangSmith 追踪) -- `LANGSMITH_API_KEY=`(启用追踪时必填) -- `LANGSMITH_PROJECT=text2sql`(可选,默认 `text2sql`) -- `LANGSMITH_ENDPOINT=https://api.smith.langchain.com`(可选) -- `AGENT_PLANNING_SCAFFOLD_ENABLED=false`(R1 规划骨架开关,默认关闭) -- `R1_GATE_WINDOW_MINUTES=60`(线上 Gate 指标窗口) -- 会话 Redis 缓冲 TTL 固定为 12 小时(43200 秒),用于持久化补偿窗口。 - -4. 生成 Prisma Client(可选但推荐) +Important backend configuration lives in `apps/backend/.env`: + +- `PORT=3002` +- `POSTGRES_HOST/POSTGRES_PORT/POSTGRES_DB/POSTGRES_USER/POSTGRES_PASSWORD` +- `REDIS_URL=redis://localhost:6379` +- `LLM_PROVIDER`, `LLM_BASE_URL`, `LLM_API_KEY`, `LLM_MODEL` +- `EMBEDDING_PROVIDER`, `EMBEDDING_BASE_URL`, `EMBEDDING_API_KEY`, `EMBEDDING_MODEL` +- `RERANK_*` is optional and can fall back to the LLM provider configuration when not configured + +The frontend uses same-origin `/api` by default, so `NEXT_PUBLIC_API_BASE_URL` usually does not need to be changed. + +### 4. Prepare Prisma + ```bash -pnpm --filter @text2sql/backend prisma:generate +pnpm --filter @text2sql/backend run prisma:generate +pnpm --filter @text2sql/backend exec node scripts/prisma-with-database-url.cjs migrate deploy ``` -5. 校验空库迁移回放(发布前强烈建议) +To verify that migrations can replay from an empty database, run against a separate test database: + ```bash DATABASE_URL=postgresql://admin:admin@localhost:5432/text2sql_ci \ pnpm --filter @text2sql/backend run prisma:verify-empty-db ``` -6. 启动前后端 +### 5. Start Frontend And Backend + ```bash pnpm dev ``` -默认地址: -- 浏览器默认入口(网关):`http://localhost:3000/data-sources` -- 前端内部开发端口(非默认直连调试):`http://localhost:3001` -- 后端内部开发端口(非默认直连调试):`http://localhost:3002` - -## 多数据源问数主线 -- 入口强制“先选数据源,再进入聊天”:`/data-sources -> 创建会话 -> /chat?datasource=...&sessionId=...` -- `POST /api/v1/sessions` 必须显式传入 `datasource`,缺失时返回 4xx 业务错误。 -- 聊天页会话列表按 `datasource` 过滤,只展示当前数据源历史,避免跨源串扰。 -- 每次发送都严格使用会话绑定数据源执行;数据源失效时会话历史可读、发送阻断并返回 `DATASOURCE_UNAVAILABLE`。 -- CSV / Excel 上传会注册为团队共享可复用数据源,可在后续会话中重复选择。 -- 查询边界默认值:连接超时 `5000ms`、查询超时 `10000ms`、默认 `LIMIT 50`、最大 `LIMIT 200`、上传上限 `10MB`。 - -## 工作空间治理与表清单授权(主叙事) -- 用户侧请求可通过 `x-workspace-id`(或单空间成员自动推断)确定工作空间语境;缺失或非法语境会触发权限错误。 -- `GET /api/v1/datasources` 与 `POST /api/v1/sessions` 已接入工作空间可见性校验,只返回/允许当前空间已绑定的数据源。 -- 治理主模型为“工作空间 -> 数据源 -> 表勾选(replace)”;不再暴露规则组概念。 -- SQL 执行链路按工作空间表授权执行默认拒绝与 fail-closed,未授权或不可安全解析的读表请求会被拒绝。 -- 治理写操作与拒绝决策会写入审计事件(`workspace.datasource.*`),支持追溯变更与拒绝原因。 -- 截至 2026-04-16,`/data-sources` 向导已覆盖连接失败恢复(`retry/previous`)与提交中按钮锁定,且在 URL 带 `workspaceId` 时会透传到创建会话请求。 -- 遗留 `table-acl` 与 `rule-group` 路径已下线,治理能力统一以 `table-permissions` 接口为准。 - -## Chat 前端交互结构 -- 聊天主区已迁移到 `assistant-ui` primitives(Thread / Message / Composer)。 -- 页面采用“聊天主区优先”布局,SQL 详情改为 assistant 消息内展开,不再固定右侧详情栏。 -- 会话侧栏、会话级模型切换、调试开关能力保持不变,仍按会话粒度生效。 -- 移动端保留“会话”与“结果详情”入口,其中“结果详情”用于快速展开最新 SQL 详情块。 - -## 联调检查清单(真实 LLM) -- 网关入口可访问:`http://localhost:3000/data-sources`。 -- 快速网关 smoke 可通过:`node tests/smoke/nginx-dev-gateway-smoke.mjs`。 -- 后端健康检查 `GET http://localhost:3002/health` 中 `llm.configured` 与 `llm.baseUrlConfigured` 为 `true`。 -- `GET http://localhost:3002/health` 中 `dependencies.llm.streamingEnabled` 与 `dependencies.llm.toolCallingEnabled` 为 `true`。 -- 如开启 LangSmith,`GET http://localhost:3002/health` 中 `dependencies.langsmith.ready` 为 `true`。 -- `GET http://localhost:3002/health` 中 `dependencies.sessions.sync` 可查看会话同步状态统计(healthy/pending/degraded)。 -- `GET http://localhost:3002/health` 中 `dependencies.gateMetrics.acceptance` 可查看 R1 门禁指标快照(sampleReady/gatePass)。 -- `GET http://localhost:3002/api/v1/rag/quality/report` 中 `glossarySelectedContext` 可查看术语 selected_context 门禁(sampleVersion/relativeLift/status)。 -- `GET http://localhost:3002/health` 中 `dependencies.ragConfig.embedding/rerank` 可查看当前生效 provider + model + configSource 摘要。 -- 前端能经 `http://localhost:3000` 成功创建会话并发送消息,无跨域报错。 -- 前端从 `/data-sources` 选择任一可用数据源后,可自动创建绑定会话并跳转 `/chat`。 -- `GET /api/v1/sessions?datasource=` 返回的会话均属于指定数据源。 -- `POST /api/v1/sessions/:sessionId/messages` 响应中包含 `run.sql` 与 `run.explanation`。 -- 当 LLM 配置缺失或不可用时,接口返回可读错误(不会回退到规则 SQL)。 - -## LangSmith 覆盖率校验(可选) +Default URLs: + +- Gateway: `http://localhost:3000` +- Datasource entry: `http://localhost:3000/data-sources` +- Frontend direct debugging: `http://localhost:3001` +- Backend health check: `http://localhost:3002/health` + +### 6. Quick Smoke Test + ```bash -ts-node apps/backend/scripts/langsmith-coverage-check.ts \ - --total 120 \ - --threshold 0.95 \ - --min-sample-size 20 \ - --lookback-hours 24 +node tests/smoke/nginx-dev-gateway-smoke.mjs ``` -- `--total`:验收窗口内“可执行请求”总数(分母)。 -- 返回 `pass=true` 表示达到覆盖率门槛。 - -## 核心 API -- `POST /api/v1/sessions`(`datasource` 必填) -- `GET /api/v1/sessions`(支持 `status`、`datasource` 与 `view=current|readonly-history|all`) -- `PATCH /api/v1/sessions/:sessionId`(支持 `title` 与 `debugEnabled` 局部更新) -- `DELETE /api/v1/sessions/:sessionId`(软删除,默认列表隐藏) -- `POST /api/v1/sessions/:sessionId/messages` -- `POST /api/v1/sessions/:sessionId/messages/stream`(SSE 流式) -- `GET /api/v1/sessions/:sessionId/messages`(返回 `session + messages + latestRun`) -- `GET /api/v1/runs/:runId` -- `POST /api/v1/runs/:runId/save-as-view` -- `GET /api/v1/glossary/terms` -- `POST /api/v1/glossary/terms`(管理员) -- `PATCH /api/v1/glossary/terms/:termId`(管理员) -- `POST /api/v1/glossary/terms/:termId/toggle`(管理员) -- `GET /api/v1/glossary/anchors` -- `POST /api/v1/glossary/anchors`(管理员) -- `POST /api/v1/glossary/anchors/rollback`(管理员) -- `GET /api/v1/settings/prompts` -- `POST /api/v1/settings/prompts`(管理员) -- `PATCH /api/v1/settings/prompts/:templateId`(管理员) -- `DELETE /api/v1/settings/prompts/:templateId`(管理员,软删除) -- `GET /api/v1/settings/rag-configs` -- `PUT /api/v1/settings/rag-configs/:taskType`(管理员,`taskType=embedding|rerank`) -- `POST /api/v1/settings/rag-configs/:taskType/health`(管理员,支持可选 `draft` payload;返回 `checkedAgainst=draft|persisted` 与结构化 `reasonCode`) -- `GET /api/v1/datasources` -- `POST /api/v1/datasources` -- `POST /api/v1/datasources/upload` -- `GET /api/v1/system/users` -- `POST /api/v1/system/users` -- `PATCH /api/v1/system/users/:userId` -- `PATCH /api/v1/system/users/:userId/status` -- `POST /api/v1/system/users/:userId/reset-password` -- `DELETE /api/v1/system/users/:userId` -- `POST /api/v1/system/users/batch-delete` -- `GET /api/v1/system/workspaces` -- `POST /api/v1/system/workspaces` -- `PATCH /api/v1/system/workspaces/:workspaceId` -- `DELETE /api/v1/system/workspaces/:workspaceId` -- `GET /api/v1/system/workspaces/:workspaceId/members` -- `POST /api/v1/system/workspaces/:workspaceId/members` -- `PATCH /api/v1/system/workspaces/:workspaceId/members/:memberId/role` -- `DELETE /api/v1/system/workspaces/:workspaceId/members/:memberId` -- `POST /api/v1/system/workspaces/:workspaceId/members/remove-batch` -- `GET /api/v1/system/workspaces/:workspaceId/datasources/bindings` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/bindings/add` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/bindings/remove` -- `GET /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/tables` -- `GET /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/table-permissions` -- `PUT /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/table-permissions` -- `GET /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/graph` -- `PUT /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/graph` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/schema-change/detect` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/schema-change/resolve` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/deploy/precheck` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/deploy` -- `POST /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/deploy/rollback` -- `POST /api/v1/evaluations/run` -- `GET /api/v1/evaluations/:jobId` -- `GET /health` - -## Chat 调试回溯说明 -- 会话级调试开关字段:`Session.debugEnabled`,默认 `false`,按会话持久化保存。 -- 运行记录新增 `SqlRun.llmRaw`:包含 `provider`、`model`、`rawText`、`createdAt`,用于原始输出回溯。 -- `trace.steps` 支持节点级状态 + 可选摘要字段(输入/输出/错误摘要、时长);前端字段缺失时自动降级为节点态。 -- 历史会话不会回填旧 `llmRaw` 数据;开启调试时会显示“该会话无历史原始返回数据”。 -- 当前策略为永久保留调试数据,不做自动清理任务。 - -## Stream & Tool Calling 说明 -- 开发态统一入口为 `http://localhost:3000`(网关转发到内部 `3001/3002`);以下 API 路径与字段契约不变。 -- 流式主路径:`POST /api/v1/sessions/:sessionId/messages/stream`。 -- 同步消息接口 `POST /api/v1/sessions/:sessionId/messages` 返回 `AgentRunResponse`: - - `kind`:固定为 `agent-run` - - `outcome`:`clarification | executionResult | rejected | failed` - - `run`:完整运行结果(含 `trace` 与可选 `llmRaw`) - - 请求体支持可选 `contextEnvelope`(`metricDefinition/timeRange/entityMappings/mustIncludeTables/mustExcludeTables/businessConstraints`) - - `agent`:聚合元信息(provider/model、是否有 SQL、是否有工具调用、是否有错误) -- SSE 事件类型:`start`、`text-delta`、`tool-call`、`tool-result`、`tool-error`、`state`、`finish`、`error`。 -- SSE 事件必填字段:`type`、`runId`、`sessionId`、`at`、`data`;其中 `data` 为结构化对象,不再混用字符串载荷。 -- 当前 Tool Calling 基础能力默认启用,首个工具为 `runReadOnlySql`(只读 SQL 执行,含输入校验与安全守卫)。 -- SQL 运行时提示词模板命中证据通过 `run.trace.promptTemplate` 与 `run.delivery.evidence.promptTemplate` 暴露(字段:`templateId/scene/scope/version/fallbackReason`)。 -- 上下文生效证据通过 `run.trace.effectiveContextSummary/conflictHint` 与 `run.delivery.evidence.effectiveContextSummary/conflictHint` 双层暴露,前端可区分用户显式上下文与系统上下文来源。 -- Text2SQL v2 artifact 通过 `run.trace.v2`、`run.delivery.evidence.v2`、SSE `state` 事件 `data.v2.stageArtifact` 暴露;不会破坏既有 `type/runId/sessionId/at/data` 合同。 -- Full Mermaid strict-completion(2026-04-27)语义补齐: - - metadata 路径走 `retrieve -> assemble-context -> semantic-plan -> answer`,不进入 `generate/validate/execute`。 - - correction 重试携带结构化 `correctionGrounding`(失败 SQL 引用、失败码、重试原因、证据引用、attempt)。 - - delivery 增补 `contextPackSummary / metadataAnswer / correctionGrounding`,用于 sync/stream/run-view/replay 一致诊断。 - - 叙事分层:`007 closeout` 证明 LangGraph 拓扑与 `delegation=0`;`008 strict-completion` 额外要求 metadata grounding / correction grounding / context-pack parity 语义闭环。 -- hard-cut read-model policy:`/api/v1/runs/:runId`、`/api/v1/runs/:runId/save-as-view`、RAG audit replay 仅支持显式 v2 读模型(`run.trace.v2.version/stageOrder/stages`)。授权后若命中历史 shape,会返回 `410 LEGACY_RUN_UNSUPPORTED`(含迁移 runbook 提示)。 - -## Text2SQL v2 评估门禁(新增) -- 评估脚本:`pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate` -- 评估脚本(发布阻断模式):`pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate:strict` -- focused coverage gate:先运行后端 Jest coverage,再执行 `pnpm --filter @text2sql/backend run collect:text2sql-v2-focused-coverage-gate`;发布阻断模式使用 `pnpm --filter @text2sql/backend run collect:text2sql-v2-focused-coverage-gate:strict` -- focused flow matrix:`apps/backend/test/fixtures/text2sql-v2-closeout-flow-matrix.json` -- fixture:`apps/backend/test/fixtures/text2sql-v2-eval-cases.json` -- characterization fixture:`apps/backend/test/fixtures/text2sql-v2-characterization-cases.json` -- 输出指标:`retrievalRelevance`、`rerankLift`、`planCoverageRate`、`validationPassRate`、`correctionSuccessRate`、`clarificationRate`、`executionSuccessRate`、`userVisibleFailureQuality`、`latencyP50Ms/P95Ms`、`denseUnavailableRate`、`rerankUnavailableRate` -- rollout 输出:`summary.rollout`(eval 指标门禁)+ `rollout`(closeout 聚合门禁,含 `recommendedStage/rollbackSuggested/reasons`) -- closeout 聚合门禁内容:`evalMetrics` + `evalTraceability` + `characterization` + `noLegacyCompat` + `focusedCoverage`(含 `delegationZero`;Full Mermaid strict-completion 场景还必须包含 `strictCompletionRows`:metadata grounding / correction grounding / context-pack parity) -- anti-regression 静态检查:`pnpm run text2sql:no-legacy-compat:check` -- 历史 run 迁移手册:`docs/runbooks/text2sql-v2-hardcut-read-model-migration.md` -- 发布姿势:当前为 direct-v2,不提供进程内 `v1/v2/shadow` runtime 切换;回滚依赖 git/deploy rollback。 - -## 测试 +## Common Workflows + +### From Datasource To Data Question + +1. Open `http://localhost:3000/data-sources` +2. Create or select a SQLite, MySQL, PostgreSQL, CSV, or Excel datasource +3. Bind it to the current workspace and enter a session +4. Ask a natural-language question in `/chat` +5. Inspect the answer, SQL, execution result, and debugging evidence + +### Settings And Governance + +- `/settings`: LLM model, RAG configuration, RAG run, system users, and other settings +- `/glossary`: business glossary maintenance +- `/modeling`: datasource modeling, relationships, and semantic views +- `/prompts`: prompt template management + +## Quality Gates + +Repository-wide: + ```bash -pnpm test -pnpm test:backend -pnpm test:frontend +pnpm run format:check +pnpm run test +pnpm run build ``` -## 后端发布完成门禁(Prisma V7) -- 依赖与生成:`pnpm --filter @text2sql/backend run prisma:generate` -- 质量门禁:`pnpm --filter @text2sql/backend run lint && pnpm --filter @text2sql/backend run build && pnpm --filter @text2sql/backend run test` -- R1 离线 Gate:`pnpm --filter @text2sql/backend exec jest test/e2e/stage1-acceptance.spec.ts --runInBand` -- 术语 selected_context 门禁:`pnpm --filter @text2sql/backend test -- glossary-selected-context-gate.spec.ts --runInBand` -- clarification hybrid balance gate: - - 观测模式:`pnpm --filter @text2sql/backend run collect:clarification-balance-gate` - - 强门禁模式(失败返回非 0):`pnpm --filter @text2sql/backend run collect:clarification-balance-gate:strict` - - 样本定义:`apps/backend/test/fixtures/clarification-balance-cases.json` -- modeling parity shadow gate: - - 观测模式:`pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate` - - 强门禁模式(失败返回非 0):`pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate:strict` - - 聚合维度:`relationshipPlatform`、`semanticSpine`、`modelingWorkspace` -- 迁移回放:`pnpm --filter @text2sql/backend run prisma:verify-empty-db` -- Text2SQL v2 评估:`pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate` -- Text2SQL v2 评估(严格):`pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate:strict` -- Text2SQL v2 focused coverage:`pnpm --filter @text2sql/backend run collect:text2sql-v2-focused-coverage-gate` -- Text2SQL hard-cut anti-regression:`pnpm run text2sql:no-legacy-compat:check` -- 启动 smoke:至少验证 `GET http://localhost:3002/health`;关键接口建议覆盖: - - 网关快速检查:`node tests/smoke/nginx-dev-gateway-smoke.mjs` - - 后端健康检查:`GET http://localhost:3002/health` - - `POST /api/v1/sessions` - - `POST /api/v1/sessions/:sessionId/messages` - - `GET /api/v1/settings/models`(管理员上下文) -- CI 可参考:`.github/workflows/backend-prisma-quality.yml` - -## Clarification Hybrid Balanced Rollout Runbook - -1. 混合门控质量信号采集(观测): +Backend: + ```bash -pnpm --filter @text2sql/backend run collect:clarification-balance-gate +pnpm --filter @text2sql/backend run lint +pnpm --filter @text2sql/backend run test +pnpm --filter @text2sql/backend run build +pnpm --filter @text2sql/backend run prisma:verify-empty-db ``` -2. 发布门禁(严格): + +Frontend: + ```bash -pnpm --filter @text2sql/backend run collect:clarification-balance-gate:strict +pnpm --filter @text2sql/frontend run lint +pnpm --filter @text2sql/frontend run test +pnpm --filter @text2sql/frontend run build ``` -3. 一键回退(rules-only): - - 设置 `CLARIFICATION_HYBRID_KILL_SWITCH_RULES_ONLY=true` - - 重启后端 -4. 回退后验证: - - `node tests/smoke/nginx-dev-gateway-smoke.mjs` - - `pnpm --filter @text2sql/backend run collect:clarification-balance-gate:strict` -5. 详细步骤与阈值说明:`docs/runbooks/clarification-hybrid-balanced-rollout.md` -## Modeling Parity Shadow Gate Rollout Runbook +Specialized gates: -1. 采集并生成报告(观测模式): ```bash +pnpm run governance:terminology:check +pnpm run backend:capability-boundary:check +pnpm run text2sql:no-legacy-compat:check +pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate +pnpm --filter @text2sql/backend run collect:text2sql-v2-focused-coverage-gate pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate ``` -2. 发布门禁(CI 或人工 go/no-go)使用严格模式: -```bash -pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate:strict -``` -3. 解读核心字段(`data/reports/modeling-parity-shadow/gate-summary.json`): - - `gatePass`:三维聚合总门禁(relationshipPlatform + semanticSpine + modelingWorkspace)。 - - `modelingWorkspace.metrics.deployBlockRate`、`rollbackRate`、`schemaBacklogAvg`:核心风险指标。 - - `modelingWorkspace.signalCoverage.*`:指标信号覆盖率,避免“样本缺字段导致误判”。 - - `rollout.recommendedStage`: - - `shadow_only`:样本不足,仅允许 shadow 观测。 - - `canary_ready`:可进入灰度。 - - `hold`:维持当前发布面,先修复指标。 - - `rollback_or_hold`:建议优先回滚或冻结发布。 -4. 触发回滚条件(任一命中即执行): - - `rollout.rollbackSuggested=true` - - `rollout.recommendedStage=rollback_or_hold` -5. 回滚入口(管理员): - - `POST /api/v1/system/workspaces/:workspaceId/datasources/:datasourceId/modeling/deploy/rollback` - - 回滚后需重跑 shadow gate,确认 `rollout.recommendedStage` 不再为 `rollback_or_hold`。 - -## 前端术语联动(Wave A/B)验收边界 -- Wave A:术语写入后可影响 `selected_context` 命中,链路异常时前端可见降级但不阻断主回答。 -- Wave B:在保持“改动即生效”前提下补齐锚点创建/回滚治理能力,并在 `/settings` 提供最小可见入口。 -- 发布证据最小集: - - `pnpm -C apps/backend test -- glossary-release-rollback.spec.ts --runInBand` - - `pnpm -C apps/backend test -- glossary-selected-context-gate.spec.ts --runInBand` - - `pnpm -C apps/frontend exec vitest run tests/unit/settings-page-governance-visibility.spec.tsx` - -## 前端质量门禁 -```bash -pnpm --filter @text2sql/frontend lint -pnpm --filter @text2sql/frontend test -pnpm --filter @text2sql/frontend build -``` + +## Important Rules + +- Prisma schema changes must start in `apps/backend/prisma/schema.prisma`, then migrations must be generated by the Prisma CLI. +- Do not handwrite or manually edit `apps/backend/prisma/migrations/*/migration.sql`. +- After table structure changes, run `pnpm --filter @text2sql/backend run prisma:generate`. +- Frontend interactive controls should reuse shadcn-ui and project business wrappers first. +- The active governance narrative uses only `workspace datasource binding`, `table-permissions`, and `policyVersion`. + +## Further Reading + +- `AGENTS.md`: repository execution entry point, hard boundaries, and quality gates +- `docs/text2sql-architecture-and-flow-2026-04-29.md`: current architecture and Text2SQL main flow +- `docs/rag-understanding/text2sql-rag-end-to-end-understanding.md`: end-to-end Text2SQL + RAG explanation +- `docs/rag-understanding/text2sql-rag-runid-replay-handbook.md`: runId replay and diagnosis +- `docs/rag-understanding/text2sql-rag-local-learning-lab.md`: local learning lab +- `docs/standards/backend-prisma-migration-spec.md`: Prisma migration standard +- `docs/standards/frontend-react-shadcn-spec.md`: frontend React + shadcn standard +- `docs/standards/llm-stream-tool-migration-spec.md`: LLM stream and tool calling migration standard +- `docs/standards/governance-terminology-spec.md`: governance terminology hard-cut standard +- `docs/standards/backend-business-capability-topology-spec.md`: backend capability topology standard diff --git a/README.zh-CN.md b/README.zh-CN.md new file mode 100644 index 0000000..1bfefae --- /dev/null +++ b/README.zh-CN.md @@ -0,0 +1,327 @@ +# Text2SQL + +[English](README.md) | 简体中文 | [日本語](README.ja.md) + +Text2SQL 是一个面向企业数据问答场景的全栈学习与演示项目。它把自然语言问题转换为可治理、可执行、可回放的 SQL,并围绕数据源接入、语义知识、权限治理、运行证据和前端交互做成一条完整链路。 + +![Text2SQL 平台总览](assets/readme-platform-overview.png) + +## 项目背景 + +业务团队常常知道自己想问什么,却不一定知道数据在哪张表、指标口径是什么、SQL 应该怎么写。直接把问题交给大模型也不够可靠:模型容易猜错表结构,缺少业务术语上下文,无法自然遵守表权限,也很难在出错后解释“为什么生成了这条 SQL”。 + +这个项目的目标不是做一个只会生成 SQL 的 Prompt demo,而是搭建一个接近真实生产约束的 Text2SQL 平台原型: + +- 用户从数据源出发提问,系统绑定会话、工作空间和数据权限。 +- Agent 在生成 SQL 前先检索 schema、术语、历史样例和语义资产。 +- SQL 生成后必须经过只读、安全、权限和执行校验。 +- 每次运行都有 `runId`,可以追溯 trace、RAG evidence、delivery artifact 和回放结果。 +- 前后端通过共享类型和 SSE 协议保持同步接口、流式接口、运行详情的一致性。 + +## 解决的问题 + +1. **自然语言到 SQL 的上下文缺口** + 用户说的是“销售额”“活跃客户”“近 30 天趋势”,数据库里却是表、列、外键、指标和业务术语。项目通过 RAG、semantic spine、glossary、modeling workspace 把这些上下文变成 SQL 生成前的证据。 + +2. **大模型生成不可控** + Text2SQL v2 runtime 使用显式 LangGraph 节点编排,把 intake、retrieve、assemble-context、semantic-plan、generate-sql、validate、correct、execute、answer 拆成可观察阶段,而不是把所有逻辑塞进一个 Prompt。 + +3. **数据权限与安全执行** + 查询不是“能跑就行”。系统以 workspace datasource binding、table-permissions、policyVersion 为治理主线,执行链路倾向 fail-closed:不确定、未授权、不可安全解析时拒绝执行。 + +4. **故障难诊断** + 每次问数都围绕 `runId` 保存运行证据。同步响应、流式 finish、run view 和 RAG replay 都尽量指向同一份 delivery contract,方便复盘 RAG 命中、SQL 生成、修正、执行和前端展示。 + +5. **多数据源演示闭环** + 项目支持 SQLite、MySQL、PostgreSQL、CSV、Excel 数据源,并提供 `/data-sources -> /chat -> /settings -> /modeling` 等前端工作台页面,适合演示和继续扩展。 + +## 功能预览 + +### 语义建模工作台 + +![语义建模工作台](assets/screenshots/01-modeling-erd-workbench.png) + +数据关系图用于把物理表结构提升为可运营的语义资产。左侧是 Models / Views 资产树,中间是 ERD 画布,右侧是选中模型的字段、关系和预览上下文。用户可以同步数据库、自动布局、维护关系、保存 Modeling Draft,并在检查通过后发布到 active 版本。 + +### 数据源接入向导 + +![数据源接入向导](assets/screenshots/02-datasource-create-wizard.png) + +数据源页面负责把数据库或文件接入工作空间。向导支持 CSV、Excel、SQLite、MySQL、PostgreSQL,创建或编辑时会自动绑定当前工作空间,并通过幂等键避免重复提交。数据源创建后可以直接进入问数会话,也可以继续做表选择和建模初始化。 + +### ChatBI 问数工作台 + +![ChatBI 问数工作台](assets/screenshots/03-chat-answer-result.png) + +Chat 页面是自然语言问数主入口。每个会话绑定一个数据源和模型配置,侧栏按数据源隔离历史会话;主区域展示用户问题、Agent 运行阶段、最终回答、表格证据和可保存视图入口。回答不是单纯文本,而是带 `validation`、执行摘要和 artifact 的结构化交付结果。 + +### SQL 证据回放 + +![SQL 证据回放](assets/screenshots/04-chat-sql-evidence.png) + +同一次 ChatBI 结果可以切换到 SQL 证据分区,查看用于生成当前回答的 SQL。这个视图服务于人工核验、调试回放和治理审计:用户能确认模型生成的查询、排序、聚合和字段选择是否符合业务预期。 + +### 图表结果 + +![图表结果](assets/screenshots/05-chat-chart-result.png) + +结果 artifact 也可以投影成图表。当前示例把支付方式占比渲染为图形视图,和 Answer、View SQL 共用同一次运行证据,避免“文本回答、SQL、图表”三套结果互相漂移。 + +## 设计哲学 + +- **证据先于生成**:SQL 生成前先组织 schema、术语、关系、权限、历史样例等 typed context,减少模型猜测。 +- **图编排优于 Prompt 串联**:关键阶段都作为 runtime 节点存在,便于观测、测试、流式投影和局部替换。 +- **治理内建,而非事后补丁**:workspace、datasource、table-permissions 和安全校验是主链路的一部分。 +- **可用但不静默降级**:RAG lane 可独立超时和降级,但降级原因会进入 evidence,不把风险藏起来。 +- **有界纠错**:SQL 修正是受预算约束的闭环,避免无限 agent loop。 +- **合同稳定**:共享类型包和 chat stream protocol 包承担前后端合同边界,避免 sync、stream、replay 各自漂移。 + +## 架构设计 + +### Monorepo 结构 + +```text +apps/backend NestJS API、Text2SQL runtime、治理、知识与平台能力 +apps/frontend Next.js 前端工作台 +packages/shared-types 前后端共享类型 +packages/chat-stream-protocol SSE envelope、parser、terminal guard、UI projection helpers +infra PostgreSQL、Redis、Nginx 本地编排 +data 本地上传文件、SQLite 与运行期数据 +docs 方案、规范、排障与理解文档 +``` + +### 系统总览 + +```mermaid +flowchart LR + User["Browser"] --> Gateway["Nginx gateway
localhost:3000"] + Gateway --> Frontend["Frontend
Next.js :3001"] + Gateway --> Backend["Backend
NestJS :3002"] + + Backend --> Conversation["conversation
chat + text2sql + delivery"] + Backend --> Governance["governance
workspace + datasource + table-permissions"] + Backend --> Knowledge["knowledge
RAG + glossary + semantic spine + graph"] + Backend --> Platform["platform
persistence + query + cache + config + observability"] + + Conversation --> Governance + Conversation --> Knowledge + Conversation --> Platform + Governance --> Platform + Knowledge --> Platform + + Platform --> Postgres["PostgreSQL"] + Platform --> Redis["Redis"] + Platform --> Datasources["SQLite / MySQL / PostgreSQL / CSV / Excel"] +``` + +### 后端四个能力域 + +| 能力域 | 目录 | 职责 | +| --- | --- | --- | +| `conversation` | `apps/backend/src/modules/conversation` | 会话入口、Text2SQL workflow、LangGraph runtime、delivery contract | +| `governance` | `apps/backend/src/modules/governance` | 工作空间、数据源绑定、表权限、用户与设置治理 | +| `knowledge` | `apps/backend/src/modules/knowledge` | RAG 检索、语义资产、术语、记忆、图谱与建模上下文 | +| `platform` | `apps/backend/src/modules/platform` | 数据持久化、查询执行、缓存、配置、观测和读模型守卫 | + +跨域依赖方向固定为: + +```text +conversation -> governance | knowledge | platform +governance -> platform +knowledge -> platform +platform -> no business-domain dependency +``` + +### Text2SQL v2 运行时 + +当前主链路固定为: + +```text +Text2SQLWorkflowRunner + -> RunV2LangGraphStage + -> Text2SqlV2LangGraphRunnerService +``` + +一次问数的大致生命周期: + +```mermaid +flowchart TD + A["intake
识别问题类型与风险"] --> B{"route"} + B -- "text_to_sql / metadata" --> C["retrieve
RAG 检索"] + B -- "general / unsafe / unsupported" --> I["answer"] + C --> D["assemble-context
组装 selected context"] + D --> E["semantic-plan
语义计划与路由"] + E -- "ready" --> F["generate-sql"] + E -- "clarify / direct answer / fail closed" --> I + F --> G["validate
只读、安全、权限、方言、dry-run"] + G -- "pass" --> H["execute"] + G -- "correctable" --> J["correct"] + G -- "terminal" --> I + J -- "retry_generation" --> F + J -- "terminal" --> I + H --> I["answer
delivery + evidence + artifact"] +``` + +### RAG 与语义上下文 + +RAG 不是简单拼接文本。当前设计强调: + +- manifest-first semantic asset preparation +- lexical、dense、graph 三路召回 +- 权限过滤先于融合和重排 +- RRF 融合与双阶段 rerank +- `selected_context`、`degradeReasons`、`riskTags` 进入运行证据 +- `runId` 贯穿 trace、delivery、replay + +## 安装与运行 + +### 依赖要求 + +- Node.js 20 或兼容版本 +- pnpm 10.x,仓库声明为 `pnpm@10.33.0` +- Docker 与 Docker Compose + +### 1. 安装依赖 + +```bash +pnpm install +``` + +### 2. 启动本地基础设施 + +```bash +docker compose -f infra/docker-compose.yml up -d +``` + +该命令会启动: + +- Nginx gateway:`http://localhost:3000` +- PostgreSQL:`localhost:5432` +- Redis:`localhost:6379` + +### 3. 初始化环境变量 + +```bash +cp apps/backend/.env.example apps/backend/.env +cp apps/frontend/.env.example apps/frontend/.env +``` + +后端关键配置位于 `apps/backend/.env`: + +- `PORT=3002` +- `POSTGRES_HOST/POSTGRES_PORT/POSTGRES_DB/POSTGRES_USER/POSTGRES_PASSWORD` +- `REDIS_URL=redis://localhost:6379` +- `LLM_PROVIDER`、`LLM_BASE_URL`、`LLM_API_KEY`、`LLM_MODEL` +- `EMBEDDING_PROVIDER`、`EMBEDDING_BASE_URL`、`EMBEDDING_API_KEY`、`EMBEDDING_MODEL` +- `RERANK_*` 可选,未配置时可回退到 LLM provider 配置 + +前端默认使用同源 `/api`,通常不需要改 `NEXT_PUBLIC_API_BASE_URL`。 + +### 4. 准备 Prisma + +```bash +pnpm --filter @text2sql/backend run prisma:generate +pnpm --filter @text2sql/backend exec node scripts/prisma-with-database-url.cjs migrate deploy +``` + +如果你是在验证迁移能否从空库完整回放,可以使用独立测试库执行: + +```bash +DATABASE_URL=postgresql://admin:admin@localhost:5432/text2sql_ci \ +pnpm --filter @text2sql/backend run prisma:verify-empty-db +``` + +### 5. 启动前后端 + +```bash +pnpm dev +``` + +默认访问: + +- 统一入口:`http://localhost:3000` +- 数据源入口:`http://localhost:3000/data-sources` +- 前端直连调试:`http://localhost:3001` +- 后端健康检查:`http://localhost:3002/health` + +### 6. 快速 smoke + +```bash +node tests/smoke/nginx-dev-gateway-smoke.mjs +``` + +## 常用工作流 + +### 数据源到问数 + +1. 打开 `http://localhost:3000/data-sources` +2. 创建或选择 SQLite、MySQL、PostgreSQL、CSV、Excel 数据源 +3. 绑定当前 workspace 并进入会话 +4. 在 `/chat` 输入自然语言问题 +5. 查看回答、SQL、执行结果和调试证据 + +### 设置与治理 + +- `/settings`:LLM 模型、RAG 配置、RAG 运行、系统用户等设置入口 +- `/glossary`:业务术语维护 +- `/modeling`:数据源建模、关系与语义视图 +- `/prompts`:提示词模板管理 + +## 质量门禁 + +跨项目: + +```bash +pnpm run format:check +pnpm run test +pnpm run build +``` + +后端: + +```bash +pnpm --filter @text2sql/backend run lint +pnpm --filter @text2sql/backend run test +pnpm --filter @text2sql/backend run build +pnpm --filter @text2sql/backend run prisma:verify-empty-db +``` + +前端: + +```bash +pnpm --filter @text2sql/frontend run lint +pnpm --filter @text2sql/frontend run test +pnpm --filter @text2sql/frontend run build +``` + +专项门禁: + +```bash +pnpm run governance:terminology:check +pnpm run backend:capability-boundary:check +pnpm run text2sql:no-legacy-compat:check +pnpm --filter @text2sql/backend run collect:text2sql-v2-eval-gate +pnpm --filter @text2sql/backend run collect:text2sql-v2-focused-coverage-gate +pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate +``` + +## 重要规范 + +- Prisma 结构变更必须先改 `apps/backend/prisma/schema.prisma`,再由 Prisma CLI 生成迁移。 +- 禁止手写或手改 `apps/backend/prisma/migrations/*/migration.sql`。 +- 表结构变更后必须执行 `pnpm --filter @text2sql/backend run prisma:generate`。 +- 前端交互控件优先复用 shadcn-ui 与项目内业务封装。 +- 治理主链路只使用 `workspace datasource binding`、`table-permissions`、`policyVersion` 作为 active narrative。 + +## 深入阅读 + +- `AGENTS.md`:仓库级执行入口、硬边界和质量门禁 +- `docs/text2sql-architecture-and-flow-2026-04-29.md`:当前架构与 Text2SQL 主流程 +- `docs/rag-understanding/text2sql-rag-end-to-end-understanding.md`:Text2SQL + RAG 端到端解析 +- `docs/rag-understanding/text2sql-rag-runid-replay-handbook.md`:runId 回放与诊断 +- `docs/rag-understanding/text2sql-rag-local-learning-lab.md`:本地实验剧本 +- `docs/standards/backend-prisma-migration-spec.md`:Prisma 迁移规范 +- `docs/standards/frontend-react-shadcn-spec.md`:前端 React + shadcn 规范 +- `docs/standards/llm-stream-tool-migration-spec.md`:LLM stream 与 tool calling 迁移规范 +- `docs/standards/governance-terminology-spec.md`:治理术语硬切规范 +- `docs/standards/backend-business-capability-topology-spec.md`:后端能力域拓扑规范 diff --git a/apps/backend/jest.config.ts b/apps/backend/jest.config.ts index 8e4b51e..916ba10 100644 --- a/apps/backend/jest.config.ts +++ b/apps/backend/jest.config.ts @@ -12,6 +12,10 @@ const config: Config = { coverageDirectory: "coverage", testEnvironment: "node", moduleNameMapper: { + "^@text2sql/chat-stream-protocol$": + "/../../packages/chat-stream-protocol/src", + "^@text2sql/chat-stream-protocol/(.*)$": + "/../../packages/chat-stream-protocol/src/$1", "^@text2sql/shared-types$": "/../../packages/shared-types/src" } }; diff --git a/apps/backend/package.json b/apps/backend/package.json index 0b568e5..68a8b36 100644 --- a/apps/backend/package.json +++ b/apps/backend/package.json @@ -3,16 +3,17 @@ "version": "0.1.0", "private": true, "scripts": { - "predev": "pnpm --filter @text2sql/shared-types build", + "predev": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "dev": "nest start --watch", "backend:restart": "node scripts/backend-restart.cjs", - "prebuild": "pnpm --filter @text2sql/shared-types build", + "prebuild": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "build": "nest build", "start": "node dist/main.js", - "pretest": "pnpm --filter @text2sql/shared-types build", + "pretest": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "test": "jest --passWithNoTests", + "pretest:e2e": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "test:e2e": "jest --config ./test/jest-e2e.json", - "prelint": "pnpm --filter @text2sql/shared-types build", + "prelint": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "lint": "tsc -p tsconfig.json --noEmit", "prisma:generate": "node scripts/prisma-with-database-url.cjs generate", "prisma:migrate": "node scripts/prisma-with-database-url.cjs migrate dev", @@ -39,6 +40,7 @@ "@nestjs/platform-express": "^10.4.2", "@prisma/adapter-pg": "^7.7.0", "@prisma/client": "^7.7.0", + "@text2sql/chat-stream-protocol": "workspace:*", "@text2sql/shared-types": "workspace:*", "ai": "^6.0.157", "class-transformer": "^0.5.1", diff --git a/apps/backend/prisma/dev.db b/apps/backend/prisma/dev.db new file mode 100644 index 0000000..e69de29 diff --git a/apps/backend/scripts/collect-text2sql-v2-eval-gate.ts b/apps/backend/scripts/collect-text2sql-v2-eval-gate.ts index 99d7a90..6961345 100644 --- a/apps/backend/scripts/collect-text2sql-v2-eval-gate.ts +++ b/apps/backend/scripts/collect-text2sql-v2-eval-gate.ts @@ -64,6 +64,7 @@ interface Text2SqlV2CloseoutGateReport { characterization: GateSnapshot; noLegacyCompat: NoLegacyGateSnapshot; focusedCoverage: FocusedCoverageGateSnapshot; + preparationPlane: GateSnapshot; }; rollout: { gatePass: boolean; @@ -208,13 +209,27 @@ export function buildCloseoutRollout(params: { gatePass: params.characterization.gatePass, reasons: params.characterization.reasons }; + const preparationPlane: GateSnapshot = params.focusedCoverage.report + ? { + gatePass: + params.focusedCoverage.report.flowMatrix.incompletePreparationPlaneRows.length === 0, + reasons: + params.focusedCoverage.report.flowMatrix.incompletePreparationPlaneRows.map( + (item) => `${item.id}:${item.reasons.join("|")}` + ) + } + : { + gatePass: params.focusedCoverage.gatePass, + reasons: params.focusedCoverage.gatePass ? [] : ["focused_coverage_report_missing"] + }; const reasons = [ ...evalMetrics.reasons.map((item) => `eval:${item}`), ...evalTraceability.reasons.map((item) => `traceability:${item}`), ...characterizationGate.reasons.map((item) => `characterization:${item}`), ...params.noLegacy.reasons.map((item) => `no_legacy:${item}`), - ...params.focusedCoverage.reasons.map((item) => `focused_coverage:${item}`) + ...params.focusedCoverage.reasons.map((item) => `focused_coverage:${item}`), + ...preparationPlane.reasons.map((item) => `preparation_plane:${item}`) ]; const gatePass = @@ -222,7 +237,8 @@ export function buildCloseoutRollout(params: { evalTraceability.gatePass && characterizationGate.gatePass && params.noLegacy.gatePass && - params.focusedCoverage.gatePass; + params.focusedCoverage.gatePass && + preparationPlane.gatePass; const rollbackSuggested = params.summary.rollout.rollbackSuggested || @@ -242,7 +258,8 @@ export function buildCloseoutRollout(params: { evalTraceability, characterization: characterizationGate, noLegacyCompat: params.noLegacy, - focusedCoverage: params.focusedCoverage + focusedCoverage: params.focusedCoverage, + preparationPlane } }; } diff --git a/apps/backend/scripts/collect-text2sql-v2-focused-coverage-gate.ts b/apps/backend/scripts/collect-text2sql-v2-focused-coverage-gate.ts index b7e7c72..d2b053f 100644 --- a/apps/backend/scripts/collect-text2sql-v2-focused-coverage-gate.ts +++ b/apps/backend/scripts/collect-text2sql-v2-focused-coverage-gate.ts @@ -96,6 +96,16 @@ interface StreamLifecycleCoverageRow { coverageOwnerStatus: "covered" | "partial" | "missing" | "planned"; } +interface PreparationPlaneCoverageRow { + id: string; + implementationOwners: string[]; + evidenceOwners: string[]; + expectedTestFiles: string[]; + behaviorTestStatus: "covered" | "partial" | "missing" | "planned"; + coverageOwnerStatus: "covered" | "partial" | "missing" | "planned"; + requiredEvidenceSignals: string[]; +} + interface RuntimePathPlan { currentActivePath: string[]; targetActivePath: string[]; @@ -115,6 +125,7 @@ export interface CloseoutFlowMatrix { runtimeCoverageRows?: RuntimeCoverageRow[]; runtimeArtifactProducerRows?: RuntimeArtifactProducerRow[]; streamLifecycleRows?: StreamLifecycleCoverageRow[]; + preparationPlaneRows?: PreparationPlaneCoverageRow[]; runtimePaths?: RuntimePathPlan; } @@ -150,6 +161,8 @@ export interface FocusedCoverageGateReport { incompleteRuntimeArtifactProducerRows: Array<{ category: string; reasons: string[] }>; streamLifecycleRowCount: number; incompleteStreamLifecycleRows: Array<{ stage: string; reasons: string[] }>; + preparationPlaneRowCount: number; + incompletePreparationPlaneRows: Array<{ id: string; reasons: string[] }>; runtimePathReasons: string[]; gatePass: boolean; }; @@ -208,6 +221,15 @@ const REQUIRED_STREAM_LIFECYCLE_STAGES = [ "answer" ] as const; +const REQUIRED_PREPARATION_PLANE_ROWS = [ + "manifest-completeness", + "typed-family-generation", + "activation-stale-behavior", + "permission-filtering-before-ranking", + "two-pass-schema-recall", + "prior-sql-diagnostics" +] as const; + const LAYERED_RUNTIME_STAGE_OWNER = "apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts"; const LAYERED_LANGGRAPH_GRAPH_OWNER = @@ -711,6 +733,35 @@ function evaluateFlowMatrix(matrix: CloseoutFlowMatrix): FocusedCoverageGateRepo } ); + const preparationPlaneRows = matrix.preparationPlaneRows ?? []; + const incompletePreparationPlaneRows = REQUIRED_PREPARATION_PLANE_ROWS.flatMap((id) => { + const row = preparationPlaneRows.find((item) => item.id === id); + const reasons: string[] = []; + if (!row) { + reasons.push("missing_preparation_plane_row"); + return [{ id, reasons }]; + } + if (row.implementationOwners.length === 0) { + reasons.push("missing_preparation_implementation_owners"); + } + if (row.evidenceOwners.length === 0) { + reasons.push("missing_preparation_evidence_owners"); + } + if (row.expectedTestFiles.length === 0) { + reasons.push("missing_preparation_behavior_tests"); + } + if (row.requiredEvidenceSignals.length === 0) { + reasons.push("missing_preparation_evidence_signals"); + } + if (row.behaviorTestStatus !== "covered") { + reasons.push(`behavior_test_status_${row.behaviorTestStatus}`); + } + if (row.coverageOwnerStatus !== "covered") { + reasons.push(`coverage_owner_status_${row.coverageOwnerStatus}`); + } + return reasons.length > 0 ? [{ id, reasons }] : []; + }); + const runtimePathReasons: string[] = []; if (!matrix.runtimePaths) { runtimePathReasons.push("missing_runtime_paths"); @@ -759,6 +810,8 @@ function evaluateFlowMatrix(matrix: CloseoutFlowMatrix): FocusedCoverageGateRepo incompleteRuntimeArtifactProducerRows, streamLifecycleRowCount: streamLifecycleRows.length, incompleteStreamLifecycleRows, + preparationPlaneRowCount: preparationPlaneRows.length, + incompletePreparationPlaneRows, runtimePathReasons, gatePass: incompleteNodes.length === 0 && @@ -767,6 +820,7 @@ function evaluateFlowMatrix(matrix: CloseoutFlowMatrix): FocusedCoverageGateRepo incompleteRuntimeCoverageRows.length === 0 && incompleteRuntimeArtifactProducerRows.length === 0 && incompleteStreamLifecycleRows.length === 0 && + incompletePreparationPlaneRows.length === 0 && runtimePathReasons.length === 0 }; } @@ -788,6 +842,9 @@ function scopedFilesFromMatrix(matrix: CloseoutFlowMatrix): string[] { ...(matrix.streamLifecycleRows ?? []).flatMap((row) => row.owners.map(effectiveOwner) ), + ...(matrix.preparationPlaneRows ?? []).flatMap((row) => + row.implementationOwners.map(effectiveOwner) + ), ...(matrix.criticalFileOwnerMigrations ?? []).map((item) => item.to) ]); } @@ -865,6 +922,9 @@ export function evaluateFocusedCoverageGate(params: { ...flowMatrix.incompleteStreamLifecycleRows.map( (item) => `stream_lifecycle:${item.stage}:${item.reasons.join("|")}` ), + ...flowMatrix.incompletePreparationPlaneRows.map( + (item) => `preparation_plane:${item.id}:${item.reasons.join("|")}` + ), ...flowMatrix.runtimePathReasons.map((reason) => `runtime_path:${reason}`) ]; diff --git a/apps/backend/src/common/abort-utils.ts b/apps/backend/src/common/abort-utils.ts new file mode 100644 index 0000000..ac80738 --- /dev/null +++ b/apps/backend/src/common/abort-utils.ts @@ -0,0 +1,56 @@ +import { DomainError } from "./domain-error"; + +export const USER_CANCELLED_CODE = "USER_CANCELLED"; +export const USER_CANCELLED_MESSAGE = "用户已停止本轮生成。"; + +export function createUserCancelledError(details?: Record): DomainError { + return new DomainError(USER_CANCELLED_CODE, USER_CANCELLED_MESSAGE, 499, details); +} + +export function isUserCancelledError(error: unknown): error is DomainError { + return error instanceof DomainError && error.code === USER_CANCELLED_CODE; +} + +export function throwIfAborted( + signal: AbortSignal | undefined, + details?: Record +): void { + if (!signal?.aborted) { + return; + } + throw createUserCancelledError(details); +} + +export function composeAbortSignals( + signals: Array +): AbortSignal | undefined { + const activeSignals = signals.filter((signal): signal is AbortSignal => Boolean(signal)); + if (activeSignals.length === 0) { + return undefined; + } + if (activeSignals.length === 1) { + return activeSignals[0]; + } + + const controller = new AbortController(); + const cleanupCallbacks: Array<() => void> = []; + + const abortFrom = (signal: AbortSignal) => { + cleanupCallbacks.splice(0).forEach((cleanup) => cleanup()); + if (!controller.signal.aborted) { + controller.abort(signal.reason); + } + }; + + for (const signal of activeSignals) { + if (signal.aborted) { + abortFrom(signal); + return controller.signal; + } + const handleAbort = () => abortFrom(signal); + signal.addEventListener("abort", handleAbort, { once: true }); + cleanupCallbacks.push(() => signal.removeEventListener("abort", handleAbort)); + } + + return controller.signal; +} diff --git a/apps/backend/src/modules/conversation/adapters/sql-validation.service.ts b/apps/backend/src/modules/conversation/adapters/sql-validation.service.ts index 073872b..1c05673 100644 --- a/apps/backend/src/modules/conversation/adapters/sql-validation.service.ts +++ b/apps/backend/src/modules/conversation/adapters/sql-validation.service.ts @@ -8,9 +8,12 @@ import type { } from "@text2sql/shared-types"; import { QueryExecutorRouterService, + SqliteQueryService, type SqlTableAccessContext } from "../../platform/data/query"; import { RelationshipDryRunService } from "../../platform/data/query/relationship-dry-run.service"; +import { DomainError } from "../../../common/domain-error"; +import { DatasourceService } from "../../governance/datasource/datasource.service"; interface ValidateSqlInput { sql: string; @@ -31,7 +34,11 @@ export class SqlValidationService { @Optional() private readonly queryExecutorRouter?: QueryExecutorRouterService, @Optional() - private readonly relationshipDryRunService?: RelationshipDryRunService + private readonly relationshipDryRunService?: RelationshipDryRunService, + @Optional() + private readonly datasourceService?: DatasourceService, + @Optional() + private readonly sqliteQuery?: SqliteQueryService ) {} async validate(input: ValidateSqlInput): Promise { @@ -69,8 +76,9 @@ export class SqlValidationService { checks.push(ledgerFulfillment.check); checks.push(this.validateDialect(sql, input.datasourceType)); checks.push( - this.validateDryRun({ + await this.validateDryRun({ sql, + datasourceId: input.datasourceId, datasourceType: input.datasourceType, parseStatus, readOnlyStatus @@ -524,42 +532,48 @@ export class SqlValidationService { private validateDryRun(input: { sql: string; + datasourceId?: string; datasourceType?: DatasourceType; parseStatus: SqlValidationCheckV1; readOnlyStatus: SqlValidationCheckV1; - }): SqlValidationCheckV1 { + }): Promise { if (input.parseStatus.status === "failed" || input.readOnlyStatus.status === "failed") { - return { + return Promise.resolve({ check: "dry-run", status: "skipped", message: "dry-run skipped because parse/read-only check already failed" - }; + }); } const capability = this.resolveDryRunCapability(input.datasourceType); if (!capability.supported) { - return { + return Promise.resolve({ check: "dry-run", status: "skipped", code: "SQL_DRY_RUN_UNSUPPORTED", message: capability.reason - }; + }); } const dryPlan = this.queryExecutorRouter?.buildDryPlan(input.sql); if (dryPlan && !dryPlan.complete) { - return { + return Promise.resolve({ check: "dry-run", status: "failed", code: "SQL_DRY_RUN_PARSE_REJECTED", message: dryPlan.reason ?? "dry-run parse rejected" - }; + }); } - return { - check: "dry-run", - status: "passed" - }; + return this.executeSqliteDryRun(input).then((failure) => { + if (failure) { + return failure; + } + return { + check: "dry-run", + status: "passed" + }; + }); } private validateDryPlan(input: { @@ -665,6 +679,67 @@ export class SqlValidationService { }; } + private async executeSqliteDryRun(input: { + sql: string; + datasourceId?: string; + datasourceType?: DatasourceType; + }): Promise { + if ( + input.datasourceType !== "sqlite" || + !input.datasourceId || + !this.datasourceService || + !this.sqliteQuery + ) { + return undefined; + } + + const datasource = await this.datasourceService.getDatasourceById(input.datasourceId); + if (!datasource || datasource.type !== "sqlite") { + return undefined; + } + + const filePath = this.resolveSqliteFilePath(datasource); + if (!filePath) { + return undefined; + } + + try { + await this.sqliteQuery.dryRun(input.sql, { filePath }); + return undefined; + } catch (error) { + const failure = this.asDomainError(error); + return { + check: "dry-run", + status: "failed", + code: failure?.code ?? "SQL_DRY_RUN_FAILED", + message: + failure?.message ?? + (error instanceof Error ? error.message : "SQLite dry-run 校验失败") + }; + } + } + + private resolveSqliteFilePath(datasource: { + id: string; + config?: Record | string | null; + }): string | undefined { + if (datasource.id === "sqlite_main") { + return this.sqliteQuery?.dbPath; + } + + const config = + datasource.config && typeof datasource.config === "object" + ? datasource.config + : undefined; + const filePath = + config && typeof config.path === "string" ? config.path.trim() : ""; + return filePath || undefined; + } + + private asDomainError(error: unknown): DomainError | undefined { + return error instanceof DomainError ? error : undefined; + } + private selectPrimaryFailure(failures: SqlValidationCheckV1[]): SqlValidationCheckV1 { const severity = (failure: SqlValidationCheckV1): number => { if (failure.check === "read-only") { diff --git a/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts b/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts index 304eeab..a52f743 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts @@ -20,7 +20,7 @@ import type { RagRetrievalBundle, RagContextPack, RagRetrievalChunkPayload -} from "../../../knowledge/rag/retrieval/rag-retrieval.types"; +} from "../../../knowledge"; import { SemanticContextPackService } from "../../adapters/semantic-context-pack.service"; import { SemanticPlanService } from "../../adapters/semantic-plan.service"; import { SemanticPlanValidator } from "../../adapters/semantic-plan.validator"; diff --git a/apps/backend/src/modules/conversation/agent/nodes/resolve-saved-prior-sql.node.ts b/apps/backend/src/modules/conversation/agent/nodes/resolve-saved-prior-sql.node.ts index 2e89bca..3d33f51 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/resolve-saved-prior-sql.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/resolve-saved-prior-sql.node.ts @@ -4,7 +4,7 @@ import type { RagPriorSqlShortcutDecision, RagRetrievalBundle, RagRetrievalChunkPayload -} from "../../../knowledge/rag/retrieval/rag-retrieval.types"; +} from "../../../knowledge"; export interface SavedPriorSqlResolution { status: "hit" | "miss" | "filtered" | "stale" | "ambiguous"; diff --git a/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts b/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts index 70ad2b1..18c63ad 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts @@ -8,13 +8,11 @@ import { AppConfigService } from "../../../config/app-config.service"; import { QueryExecutorRouterService } from "../../../platform/data/query/index"; import { KNOWLEDGE_RAG_CONTRACT, - type KnowledgeRagContract -} from "../../../knowledge/contracts/knowledge-rag.contract"; -import type { + type KnowledgeRagContract, RagRetrievalChunkPayload, RagContextPack, RagRetrievalBundle -} from "../../../knowledge/rag/retrieval/rag-retrieval.types"; +} from "../../../knowledge"; export interface RetrievedKnowledgeTypedSummary { status: "ready" | "degraded"; @@ -59,7 +57,20 @@ interface LaneStateSummary { const MAX_SCHEMA_SUPPLEMENT_TABLES = 4; const SCHEMA_RELEVANCE_HINTS: Record = { - orders: ["订单", "下单", "单量", "gmv", "order"], + orders: [ + "订单", + "下单", + "单量", + "gmv", + "销售额", + "成交额", + "交易额", + "金额", + "营收", + "sales", + "revenue", + "order" + ], order_items: ["订单明细", "商品明细", "明细", "item"], payments: ["支付", "付款", "支付方式", "支付渠道", "payment", "method"], refunds: ["退款", "退货", "refund"], diff --git a/apps/backend/src/modules/conversation/agent/planner/planner-version-lock.service.ts b/apps/backend/src/modules/conversation/agent/planner/planner-version-lock.service.ts index 797d9a5..b96497a 100644 --- a/apps/backend/src/modules/conversation/agent/planner/planner-version-lock.service.ts +++ b/apps/backend/src/modules/conversation/agent/planner/planner-version-lock.service.ts @@ -1,5 +1,5 @@ import { Injectable } from "@nestjs/common"; -import type { RagRetrievalBundle } from "../../../knowledge/rag/retrieval/rag-retrieval.types"; +import type { RagRetrievalBundle } from "../../../knowledge"; import { SemanticRegistryService } from "../../../semantic-registry/semantic-registry.service"; import { SEMANTIC_REGISTRY_DEGRADED_RISK_TAG, diff --git a/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts b/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts index 6ea87f8..78ae422 100644 --- a/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts +++ b/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts @@ -24,12 +24,7 @@ import { } from "./sql-prompt.builder"; import { PromptTemplateService } from "../../../governance/settings/prompt-template.service"; import { Text2SqlSmartDefaultsService } from "../../runtime/smart-defaults/text2sql-smart-defaults.service"; -import type { RetrievedKnowledge } from "../nodes/retrieve-knowledge.node"; -import type { RagContextPack } from "../../../rag/retrieval/rag-retrieval.types"; - -type RagRetrievalChunkPayload = NonNullable< - NonNullable["selected_context"] ->[number]; +import type { RagContextPack, RagRetrievalChunkPayload } from "../../../knowledge"; interface SqlGenerationSelection { datasourceId?: string; @@ -70,6 +65,18 @@ export interface StructuredSqlGenerationArtifact extends SqlGenerationArtifactV1 const MAX_SEMANTIC_REPAIR_RETRY = 1; const COUNT_INTENT_REGEX = /(多少|几条|几笔|总数|数量|计数|count|人数|单量|订单量|客户数|用户数)/i; +const GROUPED_COUNT_INTENT_REGEX = + /(比例|占比|分布|各|每|多少种|几种|方式|类型|类别|渠道|状态)/i; +const AMOUNT_METRIC_INTENT_REGEX = + /(销售额|净销售额|成交额|交易额|营收|收入|金额|gmv|sales|revenue|amount)/i; +const COMPLEX_AMOUNT_METRIC_INTENT_REGEX = + /(平均|均值|最大|最高|最小|最低|按|每|各|分组|分布|趋势|同比|环比|排名|月份|月|周|日|年|地区|渠道|类别|类型|状态|avg|average|max|min|top)/i; +const RECENT_TIME_INTENT_REGEX = + /(最近|近期|近\s*\d*|latest|recent|last\s+\d*)/i; +const AMOUNT_COLUMN_REGEX = + /(^|_)(total_amount|net_sales|sales_amount|gmv|revenue|amount|subtotal_amount|price_total|order_total)$/i; +const LOW_PRIORITY_AMOUNT_COLUMN_REGEX = + /(^|_)(discount_amount|shipping_amount|tax_amount|refund_amount)$/i; const METADATA_INTENT_REGEX = /(有哪些表|哪些表|多少张表|多少个表|表结构|schema|字段|列名|describe|desc\s+\w+|show\s+tables|sqlite_master|sqlite_schema|information_schema|pg_catalog|pragma|元数据|数据库结构)/i; const COUNT_SQL_REGEX = /\bcount\s*\(/i; @@ -161,6 +168,7 @@ export class SqlGenerationService { question: string, selection?: SqlGenerationSelection, options?: { + abortSignal?: AbortSignal; tools?: Record; onEvent?: (event: LlmGatewayStreamEvent) => Promise | void; } @@ -184,6 +192,7 @@ export class SqlGenerationService { ); const immediateShortcut = this.buildGroupedCountProportionShortcut(question, selection) ?? + this.buildAmountMetricShortcut(question, selection) ?? this.buildSimpleCountShortcut(question, selection); let completion: LlmDraft; if (immediateShortcut) { @@ -421,7 +430,8 @@ export class SqlGenerationService { const shortcut = this.buildGroupedCountProportionShortcut( input.question, input.selection - ) ?? this.buildSimpleCountShortcut(input.question, input.selection); + ) ?? this.buildAmountMetricShortcut(input.question, input.selection) ?? + this.buildSimpleCountShortcut(input.question, input.selection); if (!shortcut) { return undefined; } @@ -469,7 +479,7 @@ export class SqlGenerationService { if (!selectedTables || selectedTables.length !== 1) { return undefined; } - if (!/(比例|占比|分布|各|每|多少种|几种|方式|类型|类别|渠道|状态)/i.test(question)) { + if (!GROUPED_COUNT_INTENT_REGEX.test(question)) { return undefined; } @@ -504,7 +514,13 @@ export class SqlGenerationService { if (!selectedTables || selectedTables.length !== 1) { return undefined; } - if (/(比例|占比|分布|各|每|多少种|几种|方式|类型|类别|渠道|状态)/i.test(question)) { + if (GROUPED_COUNT_INTENT_REGEX.test(question)) { + return undefined; + } + if ( + !this.hasExplicitCountShortcutIntent(question, selection?.semanticPlan) || + this.hasExplicitAmountMetricIntent(question, selection?.semanticPlan) + ) { return undefined; } @@ -514,6 +530,167 @@ export class SqlGenerationService { }; } + private buildAmountMetricShortcut( + question: string, + selection: SqlGenerationSelection | undefined + ): { sql: string; model: string } | undefined { + const semanticPlan = selection?.semanticPlan; + const selectedTables = this.resolveShortcutSelectedTables(semanticPlan); + if ( + !selectedTables || + selectedTables.length !== 1 || + !this.hasExplicitAmountMetricIntent(question, semanticPlan) || + COMPLEX_AMOUNT_METRIC_INTENT_REGEX.test(question) + ) { + return undefined; + } + + const table = selectedTables[0]; + const amountColumn = this.resolveAmountMetricColumn({ + table, + columns: semanticPlan?.selectedColumns ?? [] + }); + if (!amountColumn) { + return undefined; + } + const amountExpression = `${table}.${amountColumn}`; + + const timeColumns = this.resolveRecentTimeColumns({ + table, + columns: semanticPlan?.selectedColumns ?? [] + }); + if (RECENT_TIME_INTENT_REGEX.test(question) && timeColumns.length > 0) { + const qualifiedTimeColumns = timeColumns.map((column) => `${table}.${column}`); + const timeExpression = + qualifiedTimeColumns.length === 1 + ? qualifiedTimeColumns[0] + : `COALESCE(${qualifiedTimeColumns.join(", ")})`; + return { + model: "amount-metric-sum-v1", + sql: [ + `SELECT ROUND(SUM(${amountExpression}), 2) AS recent_sales`, + `FROM ${table}`, + `WHERE date(${timeExpression}) >= (`, + ` SELECT date(MAX(${timeExpression}), '-30 days') FROM ${table}`, + ");" + ].join("\n") + }; + } + + return { + model: "amount-metric-sum-v1", + sql: [ + `SELECT ROUND(SUM(${amountExpression}), 2) AS total_sales`, + `FROM ${table};` + ].join("\n") + }; + } + + private hasExplicitCountShortcutIntent( + question: string, + semanticPlan: SemanticPlanV1 | undefined + ): boolean { + return ( + COUNT_INTENT_REGEX.test(question) || + (semanticPlan?.metrics ?? []).some((metric) => + /(^|[._:-])count($|[._:-])|^count$/i.test(metric) + ) + ); + } + + private hasExplicitAmountMetricIntent( + question: string, + semanticPlan: SemanticPlanV1 | undefined + ): boolean { + if (AMOUNT_METRIC_INTENT_REGEX.test(question)) { + return true; + } + if (COUNT_INTENT_REGEX.test(question)) { + return false; + } + const metricHints = semanticPlan?.metrics ?? []; + return metricHints.some((metric) => { + const normalizedMetric = metric.trim().toLowerCase(); + if (!normalizedMetric || normalizedMetric === "count") { + return false; + } + if (/^count($|[._:-])|(^|[._:-])count$/i.test(normalizedMetric)) { + return false; + } + return AMOUNT_METRIC_INTENT_REGEX.test(normalizedMetric); + }); + } + + private resolveAmountMetricColumn(input: { + table: string; + columns: string[]; + }): string | undefined { + const candidates = this.resolveShortcutColumnsForTable(input); + const scored = candidates + .map((column, index) => ({ + column, + index, + score: this.scoreAmountMetricColumn(column) + })) + .filter((item) => item.score > 0) + .sort((left, right) => right.score - left.score || left.index - right.index); + + return scored[0]?.column; + } + + private scoreAmountMetricColumn(column: string): number { + const normalizedColumn = column.toLowerCase(); + const exactScores: Record = { + total_amount: 20, + net_sales: 18, + sales_amount: 17, + gmv: 16, + revenue: 15, + amount: 14, + subtotal_amount: 12, + price_total: 10, + order_total: 10 + }; + if (exactScores[normalizedColumn] !== undefined) { + return exactScores[normalizedColumn]; + } + if (LOW_PRIORITY_AMOUNT_COLUMN_REGEX.test(normalizedColumn)) { + return 1; + } + return AMOUNT_COLUMN_REGEX.test(normalizedColumn) ? 8 : 0; + } + + private resolveRecentTimeColumns(input: { + table: string; + columns: string[]; + }): string[] { + const candidates = this.resolveShortcutColumnsForTable(input); + const preferredColumns = ["paid_at", "created_at", "order_date", "date"]; + return preferredColumns.filter((column) => candidates.includes(column)); + } + + private resolveShortcutColumnsForTable(input: { + table: string; + columns: string[]; + }): string[] { + return this.unique( + input.columns + .map((column) => this.normalizeQualifiedIdentifier(column)) + .filter((column): column is string => Boolean(column)) + .map((column) => { + const [tableName, columnName] = column.includes(".") + ? column.split(".") + : [input.table, column]; + if (tableName !== input.table || !columnName) { + return undefined; + } + return columnName; + }) + .filter((column): column is string => Boolean(column)) + .filter((column) => this.isSafeSqlIdentifier(column)) + ); + } + private resolveShortcutSelectedTables( semanticPlan: SemanticPlanV1 | undefined ): string[] | undefined { diff --git a/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts b/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts index 1d9510b..813316d 100644 --- a/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts +++ b/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts @@ -5,12 +5,7 @@ import type { SqlCorrectionGroundingV1 } from "@text2sql/shared-types"; import type { LlmGatewayPrompt } from "../../../llm/llm-gateway.interface"; -import type { RetrievedKnowledge } from "../nodes/retrieve-knowledge.node"; -import type { RagContextPack } from "../../../rag/retrieval/rag-retrieval.types"; - -type RagRetrievalChunkPayload = NonNullable< - NonNullable["selected_context"] ->[number]; +import type { RagContextPack, RagRetrievalChunkPayload } from "../../../knowledge"; interface SqlPruningColumnHint { name: string; @@ -196,12 +191,32 @@ export class SqlPromptBuilder { return selectedContext.slice(0, 5).map((item, index) => { const domain = item.metadata.domain; const chunkId = item.chunk_id; + const provenance = this.formatContextProvenance(item); const compact = item.content.replace(/\s+/g, " ").trim(); const excerpt = compact.length > 260 ? `${compact.slice(0, 260)}...` : compact; - return `${index + 1}. [${domain}] ${chunkId}: ${excerpt}`; + return `${index + 1}. [${domain}${provenance}] ${chunkId}: ${excerpt}`; }); } + private formatContextProvenance(item: RagRetrievalChunkPayload): string { + const sourceMetadata = this.asRecord(item.metadata.sourceMetadata); + const assetFamily = + this.readString((item.metadata as { assetFamily?: unknown }).assetFamily) ?? + this.readString(sourceMetadata?.assetFamily); + const manifestFingerprint = + this.readString((item.metadata as { manifestFingerprint?: unknown }).manifestFingerprint) ?? + this.readString(sourceMetadata?.manifestFingerprint); + const sourceVersion = + this.readString((item.metadata as { sourceVersion?: unknown }).sourceVersion) ?? + this.readString(sourceMetadata?.sourceVersion); + const parts = [ + assetFamily ? `family=${assetFamily}` : undefined, + manifestFingerprint ? `manifest=${manifestFingerprint}` : undefined, + sourceVersion ? `sourceVersion=${sourceVersion}` : undefined + ].filter((part): part is string => Boolean(part)); + return parts.length > 0 ? ` | ${parts.join(" | ")}` : ""; + } + private buildPruningContextLines(selectedContext: RagRetrievalChunkPayload[]): string[] { return selectedContext .slice(0, 5) diff --git a/apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts b/apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts index f152706..0a07a60 100644 --- a/apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts +++ b/apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts @@ -1,9 +1,16 @@ import { Injectable } from "@nestjs/common"; +import { createChatStreamEventEnvelope } from "@text2sql/chat-stream-protocol"; import type { ChatStreamEvent, ContextEnvelope, SqlRun } from "@text2sql/shared-types"; +import { + USER_CANCELLED_CODE, + createUserCancelledError, + isUserCancelledError, + throwIfAborted +} from "../../../../common/abort-utils"; import { DomainError } from "../../../../common/domain-error"; import { DatasourceRegistryService } from "../../../governance/datasource/datasource-registry.service"; import type { ChatPolicyActorInput } from "../../chat/application/shared/chat-policy-guard.service"; @@ -24,6 +31,7 @@ export interface Text2SqlWorkflowInput { export interface Text2SqlStreamWorkflowInput extends Text2SqlWorkflowInput { onEvent: (event: ChatStreamEvent) => Promise | void; + abortSignal?: AbortSignal; } @Injectable() @@ -49,10 +57,14 @@ export class Text2SQLWorkflowRunner { async runStream(input: Text2SqlStreamWorkflowInput): Promise { const prepared = await this.prepareRunStage.run(input); + const runAbortSignal = input.abortSignal; const emit = async (type: ChatStreamEvent["type"], data: ChatStreamEvent["data"]) => { + if (runAbortSignal?.aborted && type !== "error") { + return; + } await input.onEvent( - this.streamEventMapper.createEnvelope({ + createChatStreamEventEnvelope({ type, data, runId: prepared.runId, @@ -61,6 +73,12 @@ export class Text2SQLWorkflowRunner { ); }; + throwIfAborted(runAbortSignal, { + runId: prepared.runId, + sessionId: prepared.session.id, + phase: "before_start" + }); + await emit("start", { requestId: prepared.requestId ?? null }); @@ -70,6 +88,7 @@ export class Text2SQLWorkflowRunner { try { const run = await this.runV2LangGraphStage.runStream(prepared, this.streamRoute, { + abortSignal: runAbortSignal, onLlmEvent: async (event) => { const mappedEvent = this.streamEventMapper.mapLlmEvent(event); if (mappedEvent.traceToolCall) { @@ -89,6 +108,11 @@ export class Text2SQLWorkflowRunner { }); const finalizedRun = this.applyRejectedFallback(run, prepared.session.datasource); + throwIfAborted(runAbortSignal, { + runId: prepared.runId, + sessionId: prepared.session.id, + phase: "before_finish" + }); finalizedRun.trace.streamStatus = finalizedRun.error ? "failed" : "completed"; finalizedRun.trace.toolCalls = toolCalls; @@ -122,7 +146,17 @@ export class Text2SQLWorkflowRunner { }); return runWithDelivery; } catch (error) { - const messageText = error instanceof Error ? error.message : String(error); + const cancellationError = + isUserCancelledError(error) || + runAbortSignal?.aborted + ? createUserCancelledError({ + runId: prepared.runId, + sessionId: prepared.session.id + }) + : undefined; + const resolvedError = cancellationError ?? error; + const messageText = + resolvedError instanceof Error ? resolvedError.message : String(resolvedError); const run: SqlRun = { runId: prepared.runId, sessionId: prepared.session.id, @@ -143,12 +177,20 @@ export class Text2SQLWorkflowRunner { createdAt: new Date().toISOString() }; - const domainError = error instanceof DomainError ? error : undefined; - await emit("error", { - code: domainError?.code, - message: messageText, - details: (domainError?.details as Record | undefined) ?? null - }); + const domainError = resolvedError instanceof DomainError ? resolvedError : undefined; + if (!runAbortSignal?.aborted) { + await emit("error", { + code: domainError?.code, + message: messageText, + details: (domainError?.details as Record | undefined) ?? null + }); + } else if (domainError?.code === USER_CANCELLED_CODE) { + await emit("error", { + code: domainError.code, + message: messageText, + details: (domainError.details as Record | undefined) ?? null + }); + } const runWithDelivery = await this.enrichDeliveryStage.run(run); await this.persistRunStage.run({ diff --git a/apps/backend/src/modules/conversation/chat/application/stream-message.usecase.ts b/apps/backend/src/modules/conversation/chat/application/stream-message.usecase.ts index bbda059..9add4de 100644 --- a/apps/backend/src/modules/conversation/chat/application/stream-message.usecase.ts +++ b/apps/backend/src/modules/conversation/chat/application/stream-message.usecase.ts @@ -15,6 +15,7 @@ export interface StreamMessageInput { requestId?: string; contextEnvelope?: ContextEnvelope; actor?: ChatPolicyActorInput; + abortSignal?: AbortSignal; onEvent: (event: ChatStreamEvent) => Promise | void; } @@ -28,6 +29,7 @@ export class StreamMessageUsecase { message: input.message, requestId: input.requestId, onEvent: input.onEvent, + abortSignal: input.abortSignal, contextEnvelope: input.contextEnvelope, actor: input.actor }); diff --git a/apps/backend/src/modules/conversation/chat/chat.controller.ts b/apps/backend/src/modules/conversation/chat/chat.controller.ts index a52a423..e790469 100644 --- a/apps/backend/src/modules/conversation/chat/chat.controller.ts +++ b/apps/backend/src/modules/conversation/chat/chat.controller.ts @@ -13,6 +13,10 @@ import { ValidationPipe } from "@nestjs/common"; import type { Request, Response } from "express"; +import { + writeErrorEvent, + writeSseEvent +} from "@text2sql/chat-stream-protocol"; import type { AgentRunResponse, ApiResponse, @@ -191,18 +195,24 @@ export class ChatController { @Req() req: Request, @Res() res: Response ): Promise { + const abortController = new AbortController(); + const abortStream = () => { + if (!abortController.signal.aborted) { + abortController.abort("client_disconnected"); + } + }; + req.on("aborted", abortStream); + req.on("close", abortStream); + res.on("close", abortStream); + res.status(200); res.setHeader("Content-Type", "text/event-stream; charset=utf-8"); res.setHeader("Cache-Control", "no-cache, no-transform"); res.setHeader("Connection", "keep-alive"); res.flushHeaders?.(); - const sendEvent = (type: string, data: unknown) => { - if (res.writableEnded) { - return; - } - res.write(`event: ${type}\n`); - res.write(`data: ${JSON.stringify(data)}\n\n`); + const sendEvent = (event: ChatStreamEvent) => { + writeSseEvent(res, event); }; try { @@ -211,32 +221,26 @@ export class ChatController { body.message, req.requestId, async (event) => { - sendEvent(event.type, event); + sendEvent(event); }, + abortController.signal, body.contextEnvelope, req.actor ); } catch (error) { - const fallbackEvent: ChatStreamEvent = { - type: "error", - runId: "unavailable", - sessionId, - at: new Date().toISOString(), - data: - error instanceof DomainError - ? { - code: error.code, - message: error.message, - details: error.details ?? null - } - : { - code: "INTERNAL_ERROR", - message: error instanceof Error ? error.message : "未知错误", - details: null - } - }; - sendEvent("error", fallbackEvent); + if (!res.writableEnded) { + writeErrorEvent(res, { + runId: "unavailable", + sessionId, + code: error instanceof DomainError ? error.code : "INTERNAL_ERROR", + message: error instanceof Error ? error.message : "未知错误", + details: error instanceof DomainError ? error.details ?? null : null + }); + } } finally { + req.off("aborted", abortStream); + req.off("close", abortStream); + res.off("close", abortStream); if (!res.writableEnded) { res.end(); } diff --git a/apps/backend/src/modules/conversation/chat/chat.service.ts b/apps/backend/src/modules/conversation/chat/chat.service.ts index 9dc1352..645052d 100644 --- a/apps/backend/src/modules/conversation/chat/chat.service.ts +++ b/apps/backend/src/modules/conversation/chat/chat.service.ts @@ -117,6 +117,7 @@ export class ChatService { message: string, requestId: string | undefined, onEvent: (event: ChatStreamEvent) => Promise | void, + abortSignal?: AbortSignal, contextEnvelope?: ContextEnvelope, actor?: ChatPolicyActorInput ): Promise { @@ -125,6 +126,7 @@ export class ChatService { message, requestId, onEvent, + abortSignal, contextEnvelope, actor }); diff --git a/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts b/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts index 860c32b..8291ffc 100644 --- a/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts +++ b/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts @@ -59,7 +59,16 @@ interface RetrievalFusedSnapshot { chunkId: string; sourceLane?: string; domain?: string; + assetFamily?: string; + manifestFingerprint?: string; + lifecycleState?: string; }>; + preparationPlane?: { + manifestFingerprints: string[]; + assetFamilyCounts: Record; + permissionFilteredAssetCount: number; + twoPassSchemaRecallApplied: boolean; + }; skillContextSummary?: { skillCount: number; contextCount: number; @@ -139,6 +148,7 @@ type DeliveryEvidenceWithContext = NonNullable & { conflictHint?: ContextConflictHint; sqlCoverage?: SqlCoverageEvidenceSnapshot; savedPriorSql?: SavedPriorSqlEvidenceSnapshot; + preparationPlane?: RetrievalFusedSnapshot["preparationPlane"]; }; interface SandboxPostProcessOutcome { @@ -245,6 +255,7 @@ export class DeliveryContractMapper { semanticSnapshot.semanticDegradeReason ?? finalSnapshot.contextPackDegradeReasons?.at(0), skillContextSummary: fusedSnapshot.skillContextSummary, + preparationPlane: fusedSnapshot.preparationPlane, evidenceStale: evidenceStale || undefined, effectiveContextSummary: traceContextEvidence.effectiveContextSummary, conflictHint: traceContextEvidence.conflictHint, @@ -579,17 +590,56 @@ export class DeliveryContractMapper { return { chunkId, sourceLane: this.readString(item.sourceLane ?? item.source_lane), - domain: this.readString(item.domain) + domain: this.readString(item.domain), + assetFamily: this.readString(item.assetFamily ?? item.asset_family), + manifestFingerprint: this.readString( + item.manifestFingerprint ?? item.manifest_fingerprint + ), + lifecycleState: this.readString(item.lifecycleState ?? item.lifecycle_state) }; }) .filter((item): item is NonNullable => Boolean(item)); const skillContextSummary = this.readSkillContextSummary( payload.skillContext ?? payload.skill_context ); + const permissionFiltering = this.readRecord( + payload.permissionFiltering ?? payload.permission_filtering + ); + const twoPassSchemaRecall = this.readRecord( + payload.twoPassSchemaRecall ?? payload.two_pass_schema_recall + ); + const assetFamilyCounts = candidates.reduce>( + (accumulator, candidate) => { + if (candidate.assetFamily) { + accumulator[candidate.assetFamily] = + (accumulator[candidate.assetFamily] ?? 0) + 1; + } + return accumulator; + }, + {} + ); + const manifestFingerprints = this.unique( + candidates + .map((candidate) => candidate.manifestFingerprint) + .filter((item): item is string => Boolean(item)) + ); + const preparationPlane = + manifestFingerprints.length > 0 || Object.keys(assetFamilyCounts).length > 0 + ? { + manifestFingerprints, + assetFamilyCounts, + permissionFilteredAssetCount: this.readNonNegativeInt( + permissionFiltering?.filteredCount ?? permissionFiltering?.filtered_count + ), + twoPassSchemaRecallApplied: + this.readString(twoPassSchemaRecall?.status) === "applied" + } + : undefined; return { status, candidates, + preparationPlane, skillContextSummary }; } diff --git a/apps/backend/src/modules/conversation/nodes/generate-sql.node.ts b/apps/backend/src/modules/conversation/nodes/generate-sql.node.ts index 8bbce44..c3d35dd 100644 --- a/apps/backend/src/modules/conversation/nodes/generate-sql.node.ts +++ b/apps/backend/src/modules/conversation/nodes/generate-sql.node.ts @@ -23,7 +23,7 @@ import type { SqlSemanticIntent } from "../agent/sql/sql-prompt.builder"; import type { RagContextPack, RagRetrievalChunkPayload -} from "../../rag/retrieval/rag-retrieval.types"; +} from "../../knowledge"; export interface GenerateSqlNodeResult { draft: { @@ -67,6 +67,7 @@ export class GenerateSqlNode { retryReason?: string; correctionGrounding?: SqlCorrectionGroundingV1; stream?: boolean; + abortSignal?: AbortSignal; tools?: Record; onEvent?: (event: LlmGatewayStreamEvent) => Promise | void; }): Promise { @@ -112,6 +113,7 @@ export class GenerateSqlNode { correctionGrounding: input.correctionGrounding }, { + abortSignal: input.abortSignal, tools: input.tools, onEvent: input.onEvent } diff --git a/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts b/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts index 61fd09f..bf229b9 100644 --- a/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts +++ b/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts @@ -7,6 +7,7 @@ import type { Text2SqlV2StageArtifact, Text2SqlV2StageName } from "@text2sql/shared-types"; +import { throwIfAborted } from "../../../../common/abort-utils"; import { DomainError } from "../../../../common/domain-error"; import type { LlmGatewayToolDefinition } from "../../../llm/llm-gateway.interface"; import { @@ -312,6 +313,7 @@ const createNodeUpdate = async (input: { outputSummary?: Record; patch?: Partial; }): Promise => { + assertStateNotAborted(input.state, `after_${input.node}`); const step = createStep({ state: input.state, node: input.node, @@ -356,6 +358,7 @@ const emitRunningStep = async (input: { evidenceIds?: string[]; metadata?: Record; }): Promise => { + assertStateNotAborted(input.state, `before_${input.node}`); const startedAt = nowIso(); if (!input.state.streamMode || !input.state.streamOptions?.onStep) { return startedAt; @@ -511,6 +514,17 @@ const resolveCorrectRoute = (state: Text2SqlV2LangGraphState): NodeRouteKey => { return "generate-sql"; }; +const assertStateNotAborted = ( + state: Text2SqlV2LangGraphState, + phase: string +): void => { + throwIfAborted(state.streamOptions?.abortSignal, { + runId: state.runId, + sessionId: state.sessionId, + phase + }); +}; + const semanticClarificationPrompt = ( reasons: string[] ): ClarificationPrompt => { @@ -924,6 +938,7 @@ export const createText2SqlV2LangGraph = ( retryReason: state.correctionResult?.artifact.retryReason, correctionGrounding: state.correctionResult?.artifact.grounding, stream: state.streamMode, + abortSignal: state.streamOptions?.abortSignal, tools: deps.resolveSqlTools(state), onEvent: state.streamOptions?.onLlmEvent }); diff --git a/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts b/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts index e07534a..285e047 100644 --- a/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts +++ b/apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts @@ -43,6 +43,7 @@ export type Text2SqlV2LangGraphNodeName = (typeof TEXT2SQL_V2_LANGGRAPH_NODE_ORDER)[number]; export interface Text2SqlV2LangGraphStreamOptions { + abortSignal?: AbortSignal; onLlmEvent?: (event: LlmGatewayStreamEvent) => Promise | void; onStep?: (event: { step: ExecutionTraceStep }) => Promise | void; } diff --git a/apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts b/apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts index 11402c0..70f89c4 100644 --- a/apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts +++ b/apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts @@ -5,6 +5,7 @@ import { Text2SqlV2LangGraphRunnerService } from "../langgraph/text2sql-v2-langg import type { Text2SqlPreparedRunContext } from "../../text2sql/stages/prepare-run.stage"; export interface Text2SqlStreamV2LangGraphOptions { + abortSignal?: AbortSignal; onLlmEvent?: (event: LlmGatewayStreamEvent) => Promise | void; onStep?: (event: { step: ExecutionTraceStep }) => Promise | void; } diff --git a/apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts b/apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts index f18d082..66069e1 100644 --- a/apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts +++ b/apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts @@ -1,6 +1,5 @@ import { Injectable } from "@nestjs/common"; import type { - ChatStreamEvent, ChatStreamEventData, ChatStreamEventType, ExecutionTraceStep, @@ -35,22 +34,6 @@ interface LlmEventOutput { @Injectable() export class Text2SqlStreamEventMapper { - createEnvelope(input: { - type: ChatStreamEventType; - data: ChatStreamEventData; - runId: string; - sessionId: string; - at?: string; - }): ChatStreamEvent { - return { - type: input.type, - runId: input.runId, - sessionId: input.sessionId, - at: input.at ?? new Date().toISOString(), - data: input.data - }; - } - mapLlmEvent(event: LlmGatewayStreamEvent): LlmEventOutput { if (event.type === "text-delta") { return { @@ -62,12 +45,16 @@ export class Text2SqlStreamEventMapper { } if (event.type === "tool-call") { + const summary = this.summarizeToolPayload(event.input); return { type: "tool-call", data: { toolName: event.toolName, toolCallId: event.toolCallId, - input: event.input + input: event.input, + title: `调用 ${event.toolName}`, + stage: "generation", + ...(summary ? { summary } : {}) }, traceToolCall: { toolName: event.toolName, @@ -80,12 +67,16 @@ export class Text2SqlStreamEventMapper { } if (event.type === "tool-result") { + const summary = this.summarizeToolPayload(event.output); return { type: "tool-result", data: { toolName: event.toolName, toolCallId: event.toolCallId, - output: event.output + output: event.output, + title: `读取 ${event.toolName} 的结果`, + stage: "generation", + ...(summary ? { summary } : {}) }, traceToolCall: { toolName: event.toolName, @@ -102,7 +93,10 @@ export class Text2SqlStreamEventMapper { data: { toolName: event.toolName, toolCallId: event.toolCallId, - message: event.message + message: event.message, + title: `${event.toolName} 调用失败`, + stage: "generation", + summary: event.message }, traceToolCall: { toolName: event.toolName, @@ -287,6 +281,41 @@ export class Text2SqlStreamEventMapper { } } + private summarizeToolPayload(payload: unknown): string | undefined { + if (payload === undefined || payload === null) { + return undefined; + } + if (typeof payload === "string") { + return this.truncate(payload); + } + if (typeof payload !== "object") { + return this.truncate(String(payload)); + } + const record = payload as Record; + if (typeof record.sql === "string" && record.sql.trim()) { + return this.truncate(record.sql); + } + if (typeof record.rowCount === "number") { + return `返回 ${record.rowCount} 行`; + } + if (Array.isArray(record.rows)) { + return `返回 ${record.rows.length} 行`; + } + try { + return this.truncate(JSON.stringify(payload)); + } catch { + return undefined; + } + } + + private truncate(value: string, maxLength = 180): string { + const normalized = value.trim(); + if (normalized.length <= maxLength) { + return normalized; + } + return `${normalized.slice(0, maxLength - 1)}…`; + } + private resolveLifecycle( status: ExecutionTraceStep["status"] ): "completed" | "failed" | "skipped" { diff --git a/apps/backend/src/modules/data/sqlite/sqlite-query.service.ts b/apps/backend/src/modules/data/sqlite/sqlite-query.service.ts index d0f8ee1..a9fd75a 100644 --- a/apps/backend/src/modules/data/sqlite/sqlite-query.service.ts +++ b/apps/backend/src/modules/data/sqlite/sqlite-query.service.ts @@ -66,18 +66,37 @@ export class SqliteQueryService { const columns = rows[0] ? Object.keys(rows[0]) : []; return { columns, rows }; } catch (error) { - if (error instanceof DomainError) { - throw error; + throw this.normalizeSqliteError(error, { + fallbackCode: "SQL_EXECUTION_ERROR", + fallbackMessage: "SQLite 查询执行失败", + dbPath + }); + } + } + + async dryRun( + sql: string, + options?: { + filePath?: string; + } + ): Promise { + const safeSql = this.ensureSelectQuery(sql); + const finalSql = this.withLimit(safeSql); + const dbPath = options?.filePath?.trim() || this.dbPath; + try { + const { stderr } = await execFileAsync("sqlite3", [ + dbPath, + `EXPLAIN QUERY PLAN ${finalSql}` + ]); + if (stderr?.trim()) { + throw new DomainError("SQL_DRY_RUN_FAILED", stderr.trim(), 400); } - throw new DomainError( - "SQL_EXECUTION_ERROR", - "SQLite 查询执行失败", - 400, - { - dbPath, - originalMessage: error instanceof Error ? error.message : String(error) - } - ); + } catch (error) { + throw this.normalizeSqliteError(error, { + fallbackCode: "SQL_DRY_RUN_FAILED", + fallbackMessage: "SQLite dry-run 校验失败", + dbPath + }); } } @@ -119,4 +138,42 @@ export class SqliteQueryService { } return normalized; } + + private normalizeSqliteError( + error: unknown, + input: { + fallbackCode: string; + fallbackMessage: string; + dbPath: string; + } + ): DomainError { + if (error instanceof DomainError) { + return error; + } + + const originalMessage = + error instanceof Error ? error.message : String(error); + const normalizedMessage = originalMessage.trim(); + const missingColumnMatch = normalizedMessage.match( + /no such column:\s*("?)([\w.]+)\1/i + ); + if (missingColumnMatch) { + const missingColumn = missingColumnMatch[2]; + return new DomainError( + "SQL_MISSING_COLUMN", + `missing column ${missingColumn}`, + 400, + { + dbPath: input.dbPath, + originalMessage: normalizedMessage, + missingColumn + } + ); + } + + return new DomainError(input.fallbackCode, input.fallbackMessage, 400, { + dbPath: input.dbPath, + originalMessage: normalizedMessage + }); + } } diff --git a/apps/backend/src/modules/knowledge.ts b/apps/backend/src/modules/knowledge.ts index f0383d5..67af3c5 100644 --- a/apps/backend/src/modules/knowledge.ts +++ b/apps/backend/src/modules/knowledge.ts @@ -2,3 +2,22 @@ export { KNOWLEDGE_FACADE_CONTRACT, type KnowledgeFacadeContract } from "./knowledge/contracts/knowledge-facade.contract"; +export { + KNOWLEDGE_RAG_CONTRACT, + type KnowledgeRagContract +} from "./knowledge/contracts/knowledge-rag.contract"; +export type { + RagBudgetSignal, + RagContextPack, + RagPriorSqlLaneEvidence, + RagPriorSqlShortcutDecision, + RagReplayRecord, + RagRerankRequest, + RagRerankResponse, + RagRetrievalBundle, + RagRetrievalCandidate, + RagRetrievalChunkPayload, + RagRetrievalRequest, + RagRetrievalResponse, + WriteRagReplayInput +} from "./knowledge/contracts/knowledge-rag-payload.contract"; diff --git a/apps/backend/src/modules/knowledge/contracts/knowledge-rag-payload.contract.ts b/apps/backend/src/modules/knowledge/contracts/knowledge-rag-payload.contract.ts new file mode 100644 index 0000000..903adf4 --- /dev/null +++ b/apps/backend/src/modules/knowledge/contracts/knowledge-rag-payload.contract.ts @@ -0,0 +1,35 @@ +import type { + RagReplayRecord, + WriteRagReplayInput +} from "../rag/observability/rag-replay.repository"; +import type { + RagRerankRequest, + RagRerankResponse +} from "../rag/rerank/rag-rerank.service"; +import type { + RagBudgetSignal, + RagContextPack, + RagPriorSqlLaneEvidence, + RagPriorSqlShortcutDecision, + RagRetrievalBundle, + RagRetrievalCandidate, + RagRetrievalChunkPayload, + RagRetrievalRequest, + RagRetrievalResponse +} from "../rag/retrieval/rag-retrieval.types"; + +export type { + RagBudgetSignal, + RagContextPack, + RagPriorSqlLaneEvidence, + RagPriorSqlShortcutDecision, + RagReplayRecord, + RagRerankRequest, + RagRerankResponse, + RagRetrievalBundle, + RagRetrievalCandidate, + RagRetrievalChunkPayload, + RagRetrievalRequest, + RagRetrievalResponse, + WriteRagReplayInput +}; diff --git a/apps/backend/src/modules/knowledge/contracts/knowledge-rag.contract.ts b/apps/backend/src/modules/knowledge/contracts/knowledge-rag.contract.ts index f93b643..a09f891 100644 --- a/apps/backend/src/modules/knowledge/contracts/knowledge-rag.contract.ts +++ b/apps/backend/src/modules/knowledge/contracts/knowledge-rag.contract.ts @@ -1,11 +1,30 @@ -import type { RagReplayRepository } from "../rag/observability/rag-replay.repository"; -import type { RagRetrievalService } from "../rag/retrieval/rag-retrieval.service"; -import type { RagRerankService } from "../rag/rerank/rag-rerank.service"; +import type { + RagReplayRecord, + RagRerankRequest, + RagRerankResponse, + RagRetrievalRequest, + RagRetrievalResponse, + WriteRagReplayInput +} from "./knowledge-rag-payload.contract"; export const KNOWLEDGE_RAG_CONTRACT = Symbol("KNOWLEDGE_RAG_CONTRACT"); +export interface KnowledgeRagRetrievalContract { + retrieve(input: RagRetrievalRequest): Promise; +} + +export interface KnowledgeRagRerankContract { + rerank(input: RagRerankRequest): Promise; +} + +export interface KnowledgeRagReplayContract { + writeReplay(input: WriteRagReplayInput): Promise; + getReplay(runId: string, replayKey: string): Promise; + listByRunId(runId: string): Promise; +} + export interface KnowledgeRagContract { - retrieval: Pick; - rerank: Pick; - replay: Pick; + retrieval: KnowledgeRagRetrievalContract; + rerank: KnowledgeRagRerankContract; + replay: KnowledgeRagReplayContract; } diff --git a/apps/backend/src/modules/knowledge/memory/saved-prior-sql.service.ts b/apps/backend/src/modules/knowledge/memory/saved-prior-sql.service.ts index b4fd6ae..8838478 100644 --- a/apps/backend/src/modules/knowledge/memory/saved-prior-sql.service.ts +++ b/apps/backend/src/modules/knowledge/memory/saved-prior-sql.service.ts @@ -11,6 +11,7 @@ import { type SavedPriorSqlRecord } from "../contracts/knowledge-memory.contract"; import { RagReplayRepository } from "../rag/observability/rag-replay.repository"; +import { SEMANTIC_ASSET_REASON_CODES } from "../rag/preparation/semantic-asset-reason-codes"; interface NormalizedCaptureInput { workspaceId: string; @@ -133,6 +134,16 @@ export class SavedPriorSqlService { trusted: true, verified: true, priorSql: true, + assetFamily: "prior_question_sql", + visibilityScope: "workspace", + preparationStatus: "prepared", + reasonCodes: [SEMANTIC_ASSET_REASON_CODES.prepared], + sourceRef: { + type: "saved_prior_sql", + ref: record.priorId + }, + sourceVersion, + sourceHash: contentChecksum, workspaceId: record.workspaceId, datasourceId: record.datasourceId, sourceRunId: record.sourceRunId, @@ -143,7 +154,13 @@ export class SavedPriorSqlService { viewSql: normalized.viewSql, savedAt: record.savedAt, viewExists: true, - viewStatus: "active" + viewStatus: "active", + compatibilitySignals: { + viewExists: true, + viewStatus: "active", + trusted: true, + verified: true + } } }); diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-family-chunk.mapper.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-family-chunk.mapper.ts new file mode 100644 index 0000000..6e3287a --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-family-chunk.mapper.ts @@ -0,0 +1,255 @@ +import { createHash } from "node:crypto"; +import { Injectable } from "@nestjs/common"; +import type { IngestionSourceInput } from "../../../rag/ingestion/ingestion-source.adapter"; +import type { RagChunkProfile } from "../../../rag/ingestion/chunk-profiles"; +import type { + SemanticAssetFamily, + SemanticAssetManifestEntry +} from "./semantic-asset-manifest.types"; +import type { SemanticAssetSourceSnapshot } from "./semantic-asset-source-snapshot.types"; + +export interface SemanticAssetChunkMappingInput { + manifestFingerprint: string; + entry: SemanticAssetManifestEntry; + snapshot: SemanticAssetSourceSnapshot; +} + +@Injectable() +export class SemanticAssetFamilyChunkMapper { + toIngestionSource(input: SemanticAssetChunkMappingInput): IngestionSourceInput { + const metadata = this.metadataFor(input); + const content = this.contentFor(input.entry.family, input.snapshot); + const tableNames = this.tableNamesFor(input.snapshot); + const columnNames = this.columnNamesFor(input.snapshot); + const sourceRef = `${input.entry.sourceRef.type}:${input.entry.sourceRef.ref}`; + + if (input.entry.family === "prior_question_sql") { + return { + sourceType: "sql_example", + datasourceId: input.entry.datasourceId, + sourceVersion: input.entry.sourceVersion, + contentChecksum: input.entry.sourceHash ?? this.digest(content), + sourceRef, + exampleId: input.snapshot.viewId ?? input.entry.sourceRef.ref, + question: input.snapshot.question ?? input.snapshot.title ?? "Saved prior SQL", + sql: input.snapshot.sql ?? content, + rationale: input.snapshot.rationale, + tableNames, + columnNames, + metadata + }; + } + + if (input.entry.family === "business_term") { + return { + sourceType: "semantic_term", + datasourceId: input.entry.datasourceId, + sourceVersion: input.entry.sourceVersion, + contentChecksum: input.entry.sourceHash ?? this.digest(content), + sourceRef, + termId: input.entry.sourceRef.ref, + term: input.snapshot.term ?? input.snapshot.title ?? input.entry.sourceRef.ref, + definition: input.snapshot.definition ?? content, + synonyms: input.snapshot.synonyms, + tableNames, + columnNames, + metadata + }; + } + + return { + sourceType: "semantic_asset", + datasourceId: input.entry.datasourceId, + sourceVersion: input.entry.sourceVersion, + contentChecksum: input.entry.sourceHash ?? this.digest(content), + sourceRef, + assetFamily: input.entry.family, + title: input.snapshot.title ?? this.titleFor(input.entry.family, input.snapshot), + content, + tableNames, + columnNames, + metadata, + chunkProfile: this.chunkProfileFor(input.entry.family) + }; + } + + private metadataFor(input: SemanticAssetChunkMappingInput): Record { + return { + ...(input.snapshot.metadata ?? {}), + assetFamily: input.entry.family, + manifestFingerprint: input.manifestFingerprint, + manifestEntryId: input.entry.id, + semanticAssetSourceRef: input.entry.sourceRef, + sourceHash: input.entry.sourceHash, + sourceVersion: input.entry.sourceVersion, + workspaceId: input.entry.workspaceId, + datasourceId: input.entry.datasourceId, + policyVersion: input.entry.policyVersion, + modelingRevision: input.entry.modelingRevision, + visibilityScope: input.entry.visibilityScope, + preparationStatus: input.entry.status, + reasonCodes: input.entry.reasonCodes, + familySummary: input.entry.summary, + compatibilitySignals: input.snapshot.compatibilitySignals, + trusted: input.snapshot.trusted, + verified: input.snapshot.verified, + viewStatus: input.snapshot.viewStatus + }; + } + + private contentFor( + family: SemanticAssetFamily, + snapshot: SemanticAssetSourceSnapshot + ): string { + if (snapshot.content?.trim()) { + return snapshot.content.trim(); + } + + switch (family) { + case "table_description": + return this.joinLines([ + `Table: ${snapshot.tableName ?? snapshot.sourceRef?.ref ?? "unknown"}`, + snapshot.summary?.description + ? `Description: ${String(snapshot.summary.description)}` + : snapshot.title + ]); + case "full_schema": + return this.joinLines([ + `Schema: ${snapshot.tableName ?? snapshot.sourceRef?.ref ?? "unknown"}`, + this.formatColumns(snapshot.columns), + snapshot.summary?.ddl ? `DDL:\n${String(snapshot.summary.ddl)}` : undefined + ]); + case "column_batch": + return this.joinLines([ + `Columns for ${snapshot.tableName ?? snapshot.sourceRef?.ref ?? "unknown"}`, + this.formatColumns(snapshot.columns) + ]); + case "relationship_binding": + return this.joinLines([ + "Relationships", + ...(snapshot.relationships ?? []).map((relationship) => + this.joinLines([ + `${relationship.fromTable}.${relationship.fromColumn ?? "*"} -> ${ + relationship.toTable + }.${relationship.toColumn ?? "*"}`, + relationship.joinType ? `Join: ${relationship.joinType}` : undefined, + relationship.description + ]) + ) + ]); + case "metric": + return this.joinLines([ + `Metric: ${snapshot.title ?? snapshot.sourceRef?.ref ?? "unnamed"}`, + snapshot.summary?.expression ? `Expression: ${String(snapshot.summary.expression)}` : undefined, + snapshot.summary?.description + ? `Description: ${String(snapshot.summary.description)}` + : undefined + ]); + case "calculated_field": + return this.joinLines([ + `Calculated Field: ${snapshot.title ?? snapshot.sourceRef?.ref ?? "unnamed"}`, + snapshot.summary?.expression ? `Expression: ${String(snapshot.summary.expression)}` : undefined, + snapshot.summary?.description + ? `Description: ${String(snapshot.summary.description)}` + : undefined + ]); + case "prompt_instruction": + return this.joinLines([ + `Instruction: ${snapshot.scene ?? snapshot.sourceRef?.ref ?? "runtime"}`, + snapshot.scope ? `Scope: ${snapshot.scope}` : undefined, + snapshot.instructionSummary ?? snapshot.title + ]); + case "dialect_rule": + return this.joinLines([ + `Dialect: ${snapshot.dialect ?? "unknown"}`, + ...(snapshot.rules ?? []) + ]); + case "project_metadata": + return this.joinLines([ + `Project: ${snapshot.projectName ?? snapshot.workspaceId ?? "workspace"}`, + snapshot.summary ? JSON.stringify(snapshot.summary) : undefined + ]); + case "business_term": + case "prior_question_sql": + return this.joinLines([snapshot.title, snapshot.definition, snapshot.question, snapshot.sql]); + default: { + const exhaustive: never = family; + return exhaustive; + } + } + } + + private titleFor(family: SemanticAssetFamily, snapshot: SemanticAssetSourceSnapshot): string { + const ref = snapshot.sourceRef?.ref ?? snapshot.tableName ?? snapshot.term ?? "asset"; + return `${family.replace(/_/g, " ")} ${ref}`; + } + + private chunkProfileFor(family: SemanticAssetFamily): RagChunkProfile { + const profiles: Record = { + table_description: "semantic_asset_table_description", + full_schema: "semantic_asset_full_schema", + column_batch: "semantic_asset_column_batch", + relationship_binding: "semantic_asset_relationship", + metric: "semantic_asset_metric", + calculated_field: "semantic_asset_calculated_field", + business_term: "semantic_asset_business_term", + prompt_instruction: "semantic_asset_prompt_instruction", + prior_question_sql: "semantic_asset_prior_question_sql", + dialect_rule: "semantic_asset_dialect_rule", + project_metadata: "semantic_asset_project_metadata" + }; + return profiles[family]; + } + + private tableNamesFor(snapshot: SemanticAssetSourceSnapshot): string[] { + return this.unique([ + ...(snapshot.tableNames ?? []), + snapshot.tableName, + ...(snapshot.relationships ?? []).flatMap((item) => [item.fromTable, item.toTable]) + ]); + } + + private columnNamesFor(snapshot: SemanticAssetSourceSnapshot): string[] { + return this.unique([ + ...(snapshot.columnNames ?? []), + snapshot.columnName, + ...(snapshot.columns ?? []).map((item) => item.name), + ...(snapshot.relationships ?? []).flatMap((item) => [item.fromColumn, item.toColumn]) + ]); + } + + private formatColumns(columns?: SemanticAssetSourceSnapshot["columns"]): string | undefined { + if (!columns || columns.length === 0) { + return undefined; + } + return columns + .map((column) => + this.joinLines([ + `- ${column.name}${column.type ? ` ${column.type}` : ""}`, + column.description ? ` ${column.description}` : undefined + ]) + ) + .join("\n"); + } + + private joinLines(values: Array): string { + return values + .map((value) => (typeof value === "string" ? value.trim() : "")) + .filter((value) => value.length > 0) + .join("\n"); + } + + private unique(values: Array): string[] { + return Array.from( + new Set( + values + .map((value) => value?.trim()) + .filter((value): value is string => Boolean(value)) + ) + ); + } + + private digest(value: string): string { + return createHash("sha256").update(value).digest("hex"); + } +} diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-manifest.builder.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-manifest.builder.ts new file mode 100644 index 0000000..c5d47e8 --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-manifest.builder.ts @@ -0,0 +1,278 @@ +import { createHash } from "node:crypto"; +import { Injectable } from "@nestjs/common"; +import { + SEMANTIC_ASSET_MANIFEST_VERSION, + type SemanticAssetEmbeddingProfile, + type SemanticAssetFamily, + type SemanticAssetManifest, + type SemanticAssetManifestEntry, + type SemanticAssetManifestSummary, + type SemanticAssetPreparationStatus, + type SemanticAssetPreparationTrigger, + type SemanticAssetSourceRef +} from "./semantic-asset-manifest.types"; +import type { SemanticAssetSourceSnapshot } from "./semantic-asset-source-snapshot.types"; +import { + SEMANTIC_ASSET_REASON_CODES, + type SemanticAssetReasonCode +} from "./semantic-asset-reason-codes"; + +export interface BuildSemanticAssetManifestInput { + datasourceId: string; + workspaceId?: string; + triggers: readonly SemanticAssetPreparationTrigger[]; + embeddingProfile: SemanticAssetEmbeddingProfile; + sourceVersion?: string; + sourceHash?: string; + modelingRevision?: number; + reasonCodes?: readonly SemanticAssetReasonCode[]; + sourceSnapshots?: Partial< + Record< + SemanticAssetPreparationTrigger, + SemanticAssetSourceSnapshot | readonly SemanticAssetSourceSnapshot[] | unknown + > + >; +} + +@Injectable() +export class SemanticAssetManifestBuilder { + build(input: BuildSemanticAssetManifestInput): SemanticAssetManifest { + const triggers = this.unique(input.triggers).sort(); + const entries = triggers + .flatMap((trigger) => this.buildEntriesForTrigger(input, trigger)) + .sort((left, right) => this.entrySortKey(left).localeCompare(this.entrySortKey(right))); + const reasonCodes = this.unique([ + ...(input.reasonCodes ?? []), + ...(triggers.length === 0 ? [SEMANTIC_ASSET_REASON_CODES.skippedNoTriggers] : []), + ...entries.flatMap((entry) => entry.reasonCodes) + ]); + + const manifestWithoutFingerprint = { + manifestVersion: SEMANTIC_ASSET_MANIFEST_VERSION, + datasourceId: input.datasourceId, + workspaceId: input.workspaceId, + embeddingProfile: input.embeddingProfile, + sourceSnapshotSummary: { + triggers, + modelingRevision: input.modelingRevision, + sourceVersion: input.sourceVersion, + sourceHash: input.sourceHash + }, + entries, + familyCounts: this.countFamilies(entries), + reasonCodes + }; + + return { + ...manifestWithoutFingerprint, + fingerprint: `semantic-assets-${this.digest(manifestWithoutFingerprint).slice(0, 16)}` + }; + } + + summarize(manifest: SemanticAssetManifest): SemanticAssetManifestSummary { + return { + manifestVersion: manifest.manifestVersion, + fingerprint: manifest.fingerprint, + datasourceId: manifest.datasourceId, + workspaceId: manifest.workspaceId, + embeddingProfile: manifest.embeddingProfile, + sourceSnapshotSummary: manifest.sourceSnapshotSummary, + familyCounts: manifest.familyCounts, + reasonCodes: manifest.reasonCodes, + entryCount: manifest.entries.length, + preparedEntryCount: manifest.entries.filter((entry) => entry.status === "prepared").length, + skippedEntryCount: manifest.entries.filter((entry) => entry.status === "skipped").length, + degradedEntryCount: manifest.entries.filter((entry) => entry.status === "degraded").length + }; + } + + private buildEntry( + input: BuildSemanticAssetManifestInput, + trigger: SemanticAssetPreparationTrigger, + rawSnapshot: unknown + ): SemanticAssetManifestEntry { + const snapshot = this.isSnapshot(rawSnapshot) ? rawSnapshot : undefined; + const invalidSnapshot = rawSnapshot !== undefined && !snapshot; + const family = snapshot?.family ?? this.familyForTrigger(trigger); + const sourceRef = snapshot?.sourceRef ?? this.sourceRefForTrigger(trigger); + const sourceVersion = snapshot?.sourceVersion ?? input.sourceVersion ?? `${trigger}:latest`; + const enabled = snapshot?.enabled ?? true; + const defaultSummary = { + trigger, + family, + embeddingModel: input.embeddingProfile.model, + embeddingProvider: input.embeddingProfile.provider, + modelingRevision: snapshot?.modelingRevision ?? input.modelingRevision + }; + const summary = snapshot?.summary ?? defaultSummary; + const snapshotWasProvided = rawSnapshot !== undefined; + const sourceHash = + snapshot?.sourceHash ?? + (snapshotWasProvided ? undefined : this.digest({ sourceRef, sourceVersion, summary })); + const status = this.resolveStatus({ + enabled, + invalidSnapshot, + sourceHash, + snapshotWasProvided, + snapshotStatus: snapshot?.status + }); + const reasonCodes = this.reasonCodesForEntry({ + trigger, + status, + enabled, + invalidSnapshot, + sourceHash, + snapshotWasProvided, + snapshotReasonCodes: snapshot?.reasonCodes ?? [] + }); + + return { + id: `semantic-asset-entry-${this.digest({ + datasourceId: input.datasourceId, + workspaceId: input.workspaceId, + family, + sourceRef, + sourceVersion, + sourceHash + }).slice(0, 16)}`, + family, + sourceRef, + sourceVersion, + sourceHash, + workspaceId: input.workspaceId, + datasourceId: input.datasourceId, + policyVersion: snapshot?.policyVersion, + modelingRevision: snapshot?.modelingRevision ?? input.modelingRevision, + visibilityScope: snapshot?.visibilityScope ?? "datasource", + enabled, + status, + reasonCodes, + summary, + generatedChunkRefs: snapshot?.generatedChunkRefs ?? [] + }; + } + + private buildEntriesForTrigger( + input: BuildSemanticAssetManifestInput, + trigger: SemanticAssetPreparationTrigger + ): SemanticAssetManifestEntry[] { + const rawSnapshot = input.sourceSnapshots?.[trigger]; + if (Array.isArray(rawSnapshot)) { + return rawSnapshot.map((snapshot) => this.buildEntry(input, trigger, snapshot)); + } + return [this.buildEntry(input, trigger, rawSnapshot)]; + } + + private resolveStatus(input: { + enabled: boolean; + invalidSnapshot: boolean; + sourceHash?: string; + snapshotWasProvided: boolean; + snapshotStatus?: SemanticAssetPreparationStatus; + }): SemanticAssetPreparationStatus { + if (!input.enabled) { + return "skipped"; + } + if (input.snapshotStatus) { + return input.snapshotStatus; + } + if (input.invalidSnapshot || (input.snapshotWasProvided && !input.sourceHash)) { + return "degraded"; + } + return "prepared"; + } + + private reasonCodesForEntry(input: { + trigger: SemanticAssetPreparationTrigger; + status: SemanticAssetPreparationStatus; + enabled: boolean; + invalidSnapshot: boolean; + sourceHash?: string; + snapshotWasProvided: boolean; + snapshotReasonCodes: readonly SemanticAssetReasonCode[]; + }): SemanticAssetReasonCode[] { + const reasonCodes: SemanticAssetReasonCode[] = [ + `trigger:${input.trigger}`, + ...input.snapshotReasonCodes + ]; + if (!input.enabled) { + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.skippedDisabled); + } else if (input.invalidSnapshot) { + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.degradedInvalidSourceSnapshot); + } else if (input.snapshotWasProvided && !input.sourceHash) { + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.degradedMissingSourceHash); + } else if (input.status === "prepared") { + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.prepared); + } + return this.unique(reasonCodes); + } + + private familyForTrigger(trigger: SemanticAssetPreparationTrigger): SemanticAssetFamily { + const familyByTrigger: Record = { + schema: "full_schema", + modeling_revision: "relationship_binding", + examples: "prior_question_sql", + instructions: "prompt_instruction", + prior_sql: "prior_question_sql", + embedding_model: "project_metadata" + }; + return familyByTrigger[trigger]; + } + + private sourceRefForTrigger(trigger: SemanticAssetPreparationTrigger): SemanticAssetSourceRef { + const refByTrigger: Record = { + schema: { type: "datasource_schema", ref: "active-schema" }, + modeling_revision: { type: "modeling_graph", ref: "active-revision" }, + examples: { type: "sql_examples", ref: "eligible-examples" }, + instructions: { type: "prompt_instruction", ref: "runtime-eligible" }, + prior_sql: { type: "saved_prior_sql", ref: "eligible-prior-sql" }, + embedding_model: { type: "embedding_profile", ref: "active-runtime" } + }; + return refByTrigger[trigger]; + } + + private countFamilies( + entries: SemanticAssetManifestEntry[] + ): Partial> { + return entries.reduce>>((acc, entry) => { + acc[entry.family] = (acc[entry.family] ?? 0) + 1; + return acc; + }, {}); + } + + private isSnapshot(value: unknown): value is SemanticAssetSourceSnapshot { + return typeof value === "object" && value !== null && !Array.isArray(value); + } + + private entrySortKey(entry: SemanticAssetManifestEntry): string { + return [ + entry.family, + entry.sourceRef.type, + entry.sourceRef.ref, + entry.sourceVersion, + entry.id + ].join("|"); + } + + private unique(values: readonly T[]): T[] { + return Array.from(new Set(values.filter((item) => item.trim().length > 0))); + } + + private digest(value: unknown): string { + return createHash("sha256").update(stableStringify(value)).digest("hex"); + } +} + +function stableStringify(value: unknown): string { + if (Array.isArray(value)) { + return `[${value.map((item) => stableStringify(item)).join(",")}]`; + } + if (value && typeof value === "object") { + return `{${Object.entries(value) + .filter(([, item]) => item !== undefined) + .sort(([left], [right]) => left.localeCompare(right)) + .map(([key, item]) => `${JSON.stringify(key)}:${stableStringify(item)}`) + .join(",")}}`; + } + return JSON.stringify(value); +} diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-manifest.types.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-manifest.types.ts new file mode 100644 index 0000000..efc4f57 --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-manifest.types.ts @@ -0,0 +1,101 @@ +import type { SemanticAssetReasonCode } from "./semantic-asset-reason-codes"; +import type { SemanticAssetSourceSnapshot } from "./semantic-asset-source-snapshot.types"; + +export const SEMANTIC_ASSET_MANIFEST_VERSION = "semantic-asset-manifest/v1" as const; + +export const SEMANTIC_ASSET_FAMILIES = [ + "table_description", + "full_schema", + "column_batch", + "relationship_binding", + "metric", + "calculated_field", + "business_term", + "prompt_instruction", + "prior_question_sql", + "dialect_rule", + "project_metadata" +] as const; + +export type SemanticAssetFamily = (typeof SEMANTIC_ASSET_FAMILIES)[number]; + +export type SemanticAssetPreparationStatus = "prepared" | "skipped" | "degraded"; + +export type SemanticAssetVisibilityScope = + | "workspace" + | "datasource" + | "table_permissions" + | "public"; + +export type SemanticAssetPreparationTrigger = + | "schema" + | "modeling_revision" + | "examples" + | "instructions" + | "prior_sql" + | "embedding_model"; + +export interface SemanticAssetEmbeddingProfile { + provider: string; + model: string; + dimensions?: number; + vectorVersion: string; + configSource: "settings" | "env_fallback" | "missing"; +} + +export interface SemanticAssetSourceRef { + type: string; + ref: string; +} + +export interface SemanticAssetManifestEntry { + id: string; + family: SemanticAssetFamily; + sourceRef: SemanticAssetSourceRef; + sourceVersion: string; + sourceHash?: string; + workspaceId?: string; + datasourceId: string; + policyVersion?: string; + modelingRevision?: number; + visibilityScope: SemanticAssetVisibilityScope; + enabled: boolean; + status: SemanticAssetPreparationStatus; + reasonCodes: SemanticAssetReasonCode[]; + summary: Record; + generatedChunkRefs: string[]; +} + +export interface SemanticAssetManifest { + manifestVersion: typeof SEMANTIC_ASSET_MANIFEST_VERSION; + fingerprint: string; + datasourceId: string; + workspaceId?: string; + embeddingProfile: SemanticAssetEmbeddingProfile; + sourceSnapshotSummary: { + triggers: SemanticAssetPreparationTrigger[]; + modelingRevision?: number; + sourceVersion?: string; + sourceHash?: string; + }; + entries: SemanticAssetManifestEntry[]; + familyCounts: Partial>; + reasonCodes: SemanticAssetReasonCode[]; +} + +export interface SemanticAssetManifestSummary { + manifestVersion: typeof SEMANTIC_ASSET_MANIFEST_VERSION; + fingerprint: string; + datasourceId: string; + workspaceId?: string; + embeddingProfile: SemanticAssetEmbeddingProfile; + sourceSnapshotSummary: SemanticAssetManifest["sourceSnapshotSummary"]; + familyCounts: Partial>; + reasonCodes: SemanticAssetReasonCode[]; + entryCount: number; + preparedEntryCount: number; + skippedEntryCount: number; + degradedEntryCount: number; +} + +export type { SemanticAssetSourceSnapshot }; diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-preparer.service.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-preparer.service.ts new file mode 100644 index 0000000..9049754 --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-preparer.service.ts @@ -0,0 +1,311 @@ +import { createHash } from "node:crypto"; +import { Injectable } from "@nestjs/common"; +import { + type SemanticAssetEmbeddingProfile, + type SemanticAssetFamily, + type SemanticAssetManifest, + type SemanticAssetManifestSummary, + type SemanticAssetPreparationTrigger +} from "./semantic-asset-manifest.types"; +import { SemanticAssetManifestBuilder } from "./semantic-asset-manifest.builder"; +import { SEMANTIC_ASSET_REASON_CODES } from "./semantic-asset-reason-codes"; +import { SemanticAssetFamilyChunkMapper } from "./semantic-asset-family-chunk.mapper"; +import type { IngestionSourceInput } from "../../../rag/ingestion/ingestion-source.adapter"; +import type { + SemanticAssetExcludedSourceSnapshot, + SemanticAssetSourceSnapshot +} from "./semantic-asset-source-snapshot.types"; + +export interface PrepareSemanticAssetsInput { + datasourceId: string; + workspaceId?: string; + embeddingProfile: SemanticAssetEmbeddingProfile; + sourceVersion?: string; + sourceHash?: string; + modelingRevision?: number; + triggers?: readonly SemanticAssetPreparationTrigger[]; + reasonCodes?: readonly string[]; + sourceSnapshots?: readonly SemanticAssetSourceSnapshot[]; +} + +export interface SemanticAssetPreparationResult { + manifest: SemanticAssetManifest; + manifestSummary: SemanticAssetManifestSummary; + sourceSnapshots: SemanticAssetSourceSnapshot[]; + excludedSnapshots: SemanticAssetExcludedSourceSnapshot[]; + ingestionSources: IngestionSourceInput[]; +} + +@Injectable() +export class SemanticAssetPreparerService { + constructor( + private readonly manifestBuilder: SemanticAssetManifestBuilder, + private readonly chunkMapper: SemanticAssetFamilyChunkMapper + ) {} + + prepare(input: PrepareSemanticAssetsInput): SemanticAssetPreparationResult { + const normalizedSnapshots = (input.sourceSnapshots ?? []).map((snapshot) => + this.normalizeSnapshot(input, snapshot) + ); + const excludedSnapshots = this.excludedSnapshots(normalizedSnapshots); + const durableSnapshots = normalizedSnapshots.filter( + (snapshot) => !this.isExcludedCorrectionFeedback(snapshot) + ); + const triggers = + input.triggers && input.triggers.length > 0 + ? [...input.triggers] + : this.unique(durableSnapshots.map((snapshot) => this.triggerForSnapshot(snapshot))); + const manifest = this.manifestBuilder.build({ + datasourceId: input.datasourceId, + workspaceId: input.workspaceId, + triggers, + embeddingProfile: input.embeddingProfile, + sourceVersion: input.sourceVersion, + sourceHash: input.sourceHash, + modelingRevision: input.modelingRevision, + reasonCodes: input.reasonCodes, + sourceSnapshots: this.groupSnapshotsByTrigger(durableSnapshots) + }); + const snapshotsByEntry = this.snapshotsByEntryKey(durableSnapshots); + const ingestionSources = manifest.entries + .filter((entry) => entry.status === "prepared") + .map((entry) => { + const snapshot = snapshotsByEntry.get(this.entryKeyFromParts(entry.family, entry.sourceRef.ref)); + return snapshot + ? this.chunkMapper.toIngestionSource({ + manifestFingerprint: manifest.fingerprint, + entry, + snapshot + }) + : undefined; + }) + .filter((item): item is IngestionSourceInput => Boolean(item)); + + return { + manifest, + manifestSummary: this.manifestBuilder.summarize(manifest), + sourceSnapshots: durableSnapshots, + excludedSnapshots, + ingestionSources + }; + } + + private normalizeSnapshot( + input: PrepareSemanticAssetsInput, + snapshot: SemanticAssetSourceSnapshot + ): SemanticAssetSourceSnapshot { + const family = snapshot.family ?? this.familyForSourceKind(snapshot); + const sourceRef = snapshot.sourceRef ?? { + type: snapshot.sourceKind ?? this.sourceRefTypeForFamily(family), + ref: + snapshot.viewId ?? + snapshot.term ?? + snapshot.tableName ?? + snapshot.title ?? + `${family}:${this.digest(snapshot).slice(0, 12)}` + }; + const trigger = snapshot.trigger ?? this.triggerForFamily(family); + const reasonCodes = [...(snapshot.reasonCodes ?? [])]; + let status = snapshot.status; + let enabled = snapshot.enabled; + + if (snapshot.runtimeEligible === false) { + enabled = false; + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.skippedDisabled); + } + if (this.isInactiveGlossaryTerm(snapshot)) { + enabled = false; + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.skippedDisabled); + } + if (family === "relationship_binding" && !this.hasRelationshipPayload(snapshot)) { + status = "degraded"; + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.degradedMissingRelationshipPayload); + } + if (this.requiresModelingRevision(family) && !snapshot.modelingRevision && !input.modelingRevision) { + status = "degraded"; + reasonCodes.push(SEMANTIC_ASSET_REASON_CODES.degradedMissingModelingRevision); + } + + return { + ...snapshot, + trigger, + family, + sourceRef, + sourceVersion: snapshot.sourceVersion ?? `${sourceRef.type}:${sourceRef.ref}:latest`, + sourceHash: snapshot.sourceHash, + workspaceId: snapshot.workspaceId ?? input.workspaceId, + datasourceId: snapshot.datasourceId ?? input.datasourceId, + modelingRevision: snapshot.modelingRevision ?? input.modelingRevision, + visibilityScope: snapshot.visibilityScope ?? this.visibilityScopeForFamily(family), + enabled, + status, + reasonCodes: this.unique(reasonCodes) + }; + } + + private groupSnapshotsByTrigger( + snapshots: readonly SemanticAssetSourceSnapshot[] + ): Partial< + Record + > { + return snapshots.reduce< + Partial> + >((acc, snapshot) => { + const trigger = this.triggerForSnapshot(snapshot); + acc[trigger] = [...(acc[trigger] ?? []), snapshot]; + return acc; + }, {}); + } + + private snapshotsByEntryKey( + snapshots: readonly SemanticAssetSourceSnapshot[] + ): Map { + const result = new Map(); + for (const snapshot of snapshots) { + if (!snapshot.family || !snapshot.sourceRef) { + continue; + } + result.set(this.entryKeyFromParts(snapshot.family, snapshot.sourceRef.ref), snapshot); + } + return result; + } + + private excludedSnapshots( + snapshots: readonly SemanticAssetSourceSnapshot[] + ): SemanticAssetExcludedSourceSnapshot[] { + return snapshots + .filter((snapshot) => this.isExcludedCorrectionFeedback(snapshot)) + .map((snapshot) => ({ + snapshot, + reasonCodes: [SEMANTIC_ASSET_REASON_CODES.skippedCorrectionNotPromoted] + })); + } + + private isExcludedCorrectionFeedback(snapshot: SemanticAssetSourceSnapshot): boolean { + return snapshot.sourceKind === "correction_feedback" && snapshot.promoted !== true; + } + + private isInactiveGlossaryTerm(snapshot: SemanticAssetSourceSnapshot): boolean { + return ( + (snapshot.family === "business_term" || snapshot.sourceKind === "glossary_term") && + snapshot.summary?.status === "inactive" + ); + } + + private hasRelationshipPayload(snapshot: SemanticAssetSourceSnapshot): boolean { + return ( + Boolean(snapshot.content?.trim()) || + Boolean(snapshot.relationships && snapshot.relationships.length > 0) + ); + } + + private requiresModelingRevision(family: SemanticAssetFamily): boolean { + return family === "relationship_binding" || family === "metric" || family === "calculated_field"; + } + + private familyForSourceKind(snapshot: SemanticAssetSourceSnapshot): SemanticAssetFamily { + switch (snapshot.sourceKind) { + case "glossary_term": + return "business_term"; + case "prompt_instruction": + return "prompt_instruction"; + case "saved_prior_sql": + return "prior_question_sql"; + case "dialect_metadata": + return "dialect_rule"; + case "project_metadata": + return "project_metadata"; + case "modeling_graph": + return "relationship_binding"; + case "datasource_schema": + case undefined: + return snapshot.columns && snapshot.columns.length > 0 + ? "column_batch" + : "table_description"; + case "correction_feedback": + return "prior_question_sql"; + default: { + const exhaustive: never = snapshot.sourceKind; + return exhaustive; + } + } + } + + private triggerForSnapshot(snapshot: SemanticAssetSourceSnapshot): SemanticAssetPreparationTrigger { + return snapshot.trigger ?? this.triggerForFamily(snapshot.family ?? this.familyForSourceKind(snapshot)); + } + + private triggerForFamily(family: SemanticAssetFamily): SemanticAssetPreparationTrigger { + if (family === "prompt_instruction") { + return "instructions"; + } + if (family === "prior_question_sql") { + return "prior_sql"; + } + if (family === "relationship_binding" || family === "metric" || family === "calculated_field") { + return "modeling_revision"; + } + if (family === "project_metadata") { + return "embedding_model"; + } + return "schema"; + } + + private sourceRefTypeForFamily(family: SemanticAssetFamily): string { + if (family === "business_term") { + return "glossary_term"; + } + if (family === "prior_question_sql") { + return "saved_prior_sql"; + } + if (family === "prompt_instruction") { + return "prompt_instruction"; + } + if (family === "relationship_binding" || family === "metric" || family === "calculated_field") { + return "modeling_graph"; + } + if (family === "dialect_rule") { + return "dialect_metadata"; + } + if (family === "project_metadata") { + return "project_metadata"; + } + return "datasource_schema"; + } + + private visibilityScopeForFamily(family: SemanticAssetFamily): "workspace" | "datasource" | "table_permissions" | "public" { + if (family === "prior_question_sql" || family === "prompt_instruction") { + return "workspace"; + } + if (family === "business_term") { + return "workspace"; + } + return "datasource"; + } + + private entryKeyFromParts(family: SemanticAssetFamily, ref: string): string { + return `${family}|${ref}`; + } + + private unique(values: readonly (T | undefined)[]): T[] { + return Array.from(new Set(values.filter((item): item is T => Boolean(item?.trim())))); + } + + private digest(value: unknown): string { + return createHash("sha256").update(stableStringify(value)).digest("hex"); + } +} + +function stableStringify(value: unknown): string { + if (Array.isArray(value)) { + return `[${value.map((item) => stableStringify(item)).join(",")}]`; + } + if (value && typeof value === "object") { + return `{${Object.entries(value) + .filter(([, item]) => item !== undefined) + .sort(([left], [right]) => left.localeCompare(right)) + .map(([key, item]) => `${JSON.stringify(key)}:${stableStringify(item)}`) + .join(",")}}`; + } + return JSON.stringify(value); +} diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts new file mode 100644 index 0000000..c112adf --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts @@ -0,0 +1,217 @@ +import { Injectable } from "@nestjs/common"; +import { RagIndexRepository } from "../../../rag/index/rag-index.repository"; +import type { + SemanticAssetEmbeddingProfile, + SemanticAssetFamily, + SemanticAssetManifestSummary +} from "./semantic-asset-manifest.types"; + +export interface SemanticAssetReadinessActivationInput { + datasourceId: string; + workspaceId?: string; + runId?: string; + sourceVersion: string; + indexVersionId?: string; + manifestSummary: SemanticAssetManifestSummary; + activatedAt?: string; + previousActiveIndexVersionId?: string; + staleReasons?: string[]; +} + +export interface SemanticAssetReadinessSummary { + status: "ready" | "degraded" | "missing_active"; + datasourceId?: string; + workspaceId?: string; + activeManifestFingerprint?: string; + activeIndexVersionId?: string; + activeSourceVersion?: string; + embeddingProfile?: SemanticAssetEmbeddingProfile; + familyCounts: Partial>; + preparedEntryCount: number; + degradedEntryCount: number; + skippedEntryCount: number; + degradedFamilies: SemanticAssetFamily[]; + skippedFamilies: SemanticAssetFamily[]; + staleReasons: string[]; + rebuildable: boolean; + lastActivation?: { + runId?: string; + indexVersionId?: string; + manifestFingerprint: string; + activatedAt: string; + previousActiveIndexVersionId?: string; + }; + lastTrigger?: { + runId?: string; + sourceVersion: string; + manifestFingerprint: string; + }; + generatedAt: string; +} + +@Injectable() +export class SemanticAssetReadinessService { + private readonly summariesByDatasource = new Map(); + private latestDatasourceId?: string; + + constructor(private readonly indexRepository: RagIndexRepository) {} + + recordActivation(input: SemanticAssetReadinessActivationInput): SemanticAssetReadinessSummary { + const datasourceId = input.datasourceId.trim(); + const summary = input.manifestSummary; + const degradedFamilies = this.familiesForStatus(summary, "degraded"); + const skippedFamilies = this.familiesForStatus(summary, "skipped"); + const staleReasons = this.unique(input.staleReasons ?? []); + const status = + degradedFamilies.length > 0 || staleReasons.length > 0 ? "degraded" : "ready"; + const manifestFingerprint = summary.fingerprint; + const generatedAt = new Date().toISOString(); + const readiness: SemanticAssetReadinessSummary = { + status, + datasourceId, + workspaceId: input.workspaceId, + activeManifestFingerprint: manifestFingerprint, + activeIndexVersionId: input.indexVersionId, + activeSourceVersion: input.sourceVersion, + embeddingProfile: summary.embeddingProfile, + familyCounts: { ...summary.familyCounts }, + preparedEntryCount: summary.preparedEntryCount, + degradedEntryCount: summary.degradedEntryCount, + skippedEntryCount: summary.skippedEntryCount, + degradedFamilies, + skippedFamilies, + staleReasons, + rebuildable: staleReasons.length > 0, + lastActivation: { + runId: input.runId, + indexVersionId: input.indexVersionId, + manifestFingerprint, + activatedAt: input.activatedAt ?? generatedAt, + previousActiveIndexVersionId: input.previousActiveIndexVersionId + }, + lastTrigger: { + runId: input.runId, + sourceVersion: input.sourceVersion, + manifestFingerprint + }, + generatedAt + }; + this.summariesByDatasource.set(datasourceId, readiness); + this.latestDatasourceId = datasourceId; + return this.clone(readiness); + } + + async snapshot(input?: { datasourceId?: string }): Promise { + const datasourceId = input?.datasourceId?.trim() || this.latestDatasourceId; + if (!datasourceId) { + return this.emptySummary(); + } + + const current = this.summariesByDatasource.get(datasourceId); + const activeVersion = await this.indexRepository.getActiveVersion(datasourceId); + if (!activeVersion) { + return { + ...(current ?? this.emptySummary()), + status: "missing_active", + datasourceId, + activeIndexVersionId: undefined, + activeSourceVersion: undefined, + staleReasons: this.unique([...(current?.staleReasons ?? []), "no_active_index"]), + rebuildable: true, + generatedAt: new Date().toISOString() + }; + } + + if (!current) { + return { + ...this.emptySummary(), + status: "ready", + datasourceId, + activeIndexVersionId: activeVersion.id, + activeSourceVersion: activeVersion.sourceVersion, + activeManifestFingerprint: this.extractManifestFingerprint(activeVersion.sourceVersion), + lastActivation: activeVersion.activatedAt + ? { + indexVersionId: activeVersion.id, + manifestFingerprint: + this.extractManifestFingerprint(activeVersion.sourceVersion) ?? + activeVersion.sourceVersion, + activatedAt: activeVersion.activatedAt + } + : undefined, + generatedAt: new Date().toISOString() + }; + } + + const activeManifestFingerprint = this.extractManifestFingerprint(activeVersion.sourceVersion); + const staleReasons = + current.activeSourceVersion && current.activeSourceVersion !== activeVersion.sourceVersion + ? this.unique([...current.staleReasons, "active_index_changed"]) + : current.staleReasons; + return { + ...this.clone(current), + activeIndexVersionId: activeVersion.id, + activeSourceVersion: activeVersion.sourceVersion, + activeManifestFingerprint: activeManifestFingerprint ?? current.activeManifestFingerprint, + status: + staleReasons.length > 0 || current.degradedFamilies.length > 0 + ? "degraded" + : "ready", + staleReasons, + rebuildable: staleReasons.length > 0, + generatedAt: new Date().toISOString() + }; + } + + private emptySummary(): SemanticAssetReadinessSummary { + return { + status: "missing_active", + familyCounts: {}, + preparedEntryCount: 0, + degradedEntryCount: 0, + skippedEntryCount: 0, + degradedFamilies: [], + skippedFamilies: [], + staleReasons: [], + rebuildable: false, + generatedAt: new Date().toISOString() + }; + } + + private familiesForStatus( + summary: SemanticAssetManifestSummary, + status: "degraded" | "skipped" + ): SemanticAssetFamily[] { + const reasonMarker = status === "degraded" ? "degraded" : "skipped"; + const entryCount = + status === "degraded" ? summary.degradedEntryCount : summary.skippedEntryCount; + if (entryCount === 0) { + return []; + } + return Object.keys(summary.familyCounts) + .filter((family): family is SemanticAssetFamily => Boolean(family)) + .filter(() => summary.reasonCodes.some((reason) => reason.includes(reasonMarker))) + .sort(); + } + + private extractManifestFingerprint(sourceVersion: string): string | undefined { + const [fingerprint] = sourceVersion.trim().split(":"); + return fingerprint || undefined; + } + + private clone(summary: SemanticAssetReadinessSummary): SemanticAssetReadinessSummary { + return { + ...summary, + familyCounts: { ...summary.familyCounts }, + degradedFamilies: [...summary.degradedFamilies], + skippedFamilies: [...summary.skippedFamilies], + staleReasons: [...summary.staleReasons], + lastActivation: summary.lastActivation ? { ...summary.lastActivation } : undefined, + lastTrigger: summary.lastTrigger ? { ...summary.lastTrigger } : undefined + }; + } + + private unique(values: readonly string[]): string[] { + return Array.from(new Set(values.filter((item) => item.trim().length > 0))).sort(); + } +} diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-reason-codes.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-reason-codes.ts new file mode 100644 index 0000000..ea7b2d3 --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-reason-codes.ts @@ -0,0 +1,19 @@ +export const SEMANTIC_ASSET_REASON_CODES = { + prepared: "prepared", + semanticAssetReindexRequested: "semantic_asset_reindex_requested", + skippedNoTriggers: "skipped_no_triggers", + skippedDisabled: "skipped_disabled", + skippedAlreadyActive: "skip:already_active", + degradedInvalidSourceSnapshot: "degraded_invalid_source_snapshot", + degradedMissingSourceHash: "degraded_missing_source_hash", + degradedMissingRelationshipPayload: "degraded_missing_relationship_payload", + degradedMissingModelingRevision: "degraded_missing_modeling_revision", + skippedCorrectionNotPromoted: "skipped_correction_not_promoted", + staleSourceVersion: "stale_source_version", + activationReplacedPrevious: "activation_replaced_previous" +} as const; + +export type SemanticAssetReasonCode = + (typeof SEMANTIC_ASSET_REASON_CODES)[keyof typeof SEMANTIC_ASSET_REASON_CODES] | + `trigger:${string}` | + string; diff --git a/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-source-snapshot.types.ts b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-source-snapshot.types.ts new file mode 100644 index 0000000..89d6c8a --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-source-snapshot.types.ts @@ -0,0 +1,87 @@ +import type { + SemanticAssetFamily, + SemanticAssetPreparationStatus, + SemanticAssetPreparationTrigger, + SemanticAssetSourceRef, + SemanticAssetVisibilityScope +} from "./semantic-asset-manifest.types"; +import type { SemanticAssetReasonCode } from "./semantic-asset-reason-codes"; + +export type SemanticAssetSourceSnapshotKind = + | "datasource_schema" + | "modeling_graph" + | "glossary_term" + | "prompt_instruction" + | "saved_prior_sql" + | "dialect_metadata" + | "project_metadata" + | "correction_feedback"; + +export interface SemanticAssetColumnSnapshot { + name: string; + type?: string; + description?: string; +} + +export interface SemanticAssetRelationshipSnapshot { + name?: string; + fromTable: string; + fromColumn?: string; + toTable: string; + toColumn?: string; + joinType?: string; + description?: string; +} + +export interface SemanticAssetSourceSnapshot { + trigger?: SemanticAssetPreparationTrigger; + sourceKind?: SemanticAssetSourceSnapshotKind; + family?: SemanticAssetFamily; + sourceRef?: SemanticAssetSourceRef; + sourceVersion?: string; + sourceHash?: string; + policyVersion?: string; + modelingRevision?: number; + visibilityScope?: SemanticAssetVisibilityScope; + enabled?: boolean; + status?: SemanticAssetPreparationStatus; + reasonCodes?: SemanticAssetReasonCode[]; + summary?: Record; + generatedChunkRefs?: string[]; + title?: string; + content?: string; + tableName?: string; + columnName?: string; + columns?: SemanticAssetColumnSnapshot[]; + relationships?: SemanticAssetRelationshipSnapshot[]; + term?: string; + definition?: string; + synonyms?: string[]; + question?: string; + sql?: string; + rationale?: string; + scene?: string; + scope?: string; + instructionSummary?: string; + dialect?: string; + rules?: string[]; + projectName?: string; + workspaceId?: string; + datasourceId?: string; + tableNames?: string[]; + columnNames?: string[]; + trusted?: boolean; + verified?: boolean; + promoted?: boolean; + runtimeEligible?: boolean; + viewId?: string; + viewName?: string; + viewStatus?: string; + compatibilitySignals?: Record; + metadata?: Record; +} + +export interface SemanticAssetExcludedSourceSnapshot { + snapshot: SemanticAssetSourceSnapshot; + reasonCodes: SemanticAssetReasonCode[]; +} diff --git a/apps/backend/src/modules/knowledge/rag/rag-owner-classification.ts b/apps/backend/src/modules/knowledge/rag/rag-owner-classification.ts new file mode 100644 index 0000000..e65683f --- /dev/null +++ b/apps/backend/src/modules/knowledge/rag/rag-owner-classification.ts @@ -0,0 +1,153 @@ +const RAG_OWNER_CLASSIFICATION_ENTRIES = [ + { + file: "apps/backend/src/modules/rag/rag.module.ts", + classification: "compatibility-support", + reason: + "Legacy rag module path is compat-only alias wiring and must not be treated as an active owner entry." + }, + { + file: "apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts", + classification: "legacy-active", + reason: + "Legacy retrieval service path is pending retirement and must not be used as active owner." + }, + { + file: "apps/backend/src/modules/rag/rerank/rag-rerank.service.ts", + classification: "legacy-active", + reason: + "Legacy rerank service path is pending retirement and must not be used as active owner." + }, + { + file: "apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts", + classification: "legacy-active", + reason: + "Legacy retrieval payload type path is pending retirement and must not be imported by new production code." + }, + { + file: "apps/backend/src/modules/rag/index/rag-index-builder.service.ts", + classification: "shared-internal", + reason: "Shared internal index builder still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/index/rag-index.repository.ts", + classification: "shared-internal", + reason: "Shared internal index repository still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/ingestion/rag-document.factory.ts", + classification: "shared-internal", + reason: "Shared internal ingestion factory still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/ingestion/rag-document.repository.ts", + classification: "shared-internal", + reason: "Shared internal ingestion repository still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/ingestion/ingestion-source.adapter.ts", + classification: "shared-internal", + reason: "Shared internal ingestion source adapter still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/ingestion/rag-chunking.service.ts", + classification: "shared-internal", + reason: "Shared internal chunking service still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/jobs/build-rag-index.job.ts", + classification: "shared-internal", + reason: "Shared internal index build job still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/observability/rag-replay.repository.ts", + classification: "shared-internal", + reason: "Shared internal replay persistence adapter still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/quality/rag-quality.service.ts", + classification: "shared-internal", + reason: "Shared internal quality service still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/perf/rag-budget-policy.ts", + classification: "shared-internal", + reason: "Shared internal budget policy still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/perf/rag-cache-key.factory.ts", + classification: "shared-internal", + reason: "Shared internal cache key factory still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/perf/rag-query-cache.service.ts", + classification: "shared-internal", + reason: "Shared internal cache service still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/orchestration/rag-datasource-orchestrator.service.ts", + classification: "shared-internal", + reason: "Shared internal datasource orchestration still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/orchestration/rag-datasource-quota.policy.ts", + classification: "shared-internal", + reason: "Shared internal quota policy still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/audit/rag-audit-replay.service.ts", + classification: "shared-internal", + reason: "Shared internal replay audit service still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/events/rag-event-consumer.service.ts", + classification: "shared-internal", + reason: "Shared internal event consumer still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/rerank/model-reranker.adapter.ts", + classification: "shared-internal", + reason: "Shared internal model reranker adapter still assembled by canonical knowledge/rag owner." + }, + { + file: "apps/backend/src/modules/rag/retrieval/fusion/rrf-fusion.ts", + classification: "shared-internal", + reason: "Shared internal RRF fusion utility remains reused by canonical owner." + } +] as const; + +export type RagOwnerClassificationEntry = (typeof RAG_OWNER_CLASSIFICATION_ENTRIES)[number]; +export type RagOwnerClassification = RagOwnerClassificationEntry["classification"]; + +const RAG_OWNER_CLASSIFICATION_MAP: Map = new Map( + RAG_OWNER_CLASSIFICATION_ENTRIES.map((entry) => [entry.file, entry]) +); + +export const KNOWLEDGE_RAG_OWNER_MODULE_PATH = + "apps/backend/src/modules/knowledge/rag/rag.module.ts"; + +export function getRagOwnerClassification( + relativeFilePath: string +): RagOwnerClassificationEntry | undefined { + return RAG_OWNER_CLASSIFICATION_MAP.get(relativeFilePath); +} + +export function isRagSharedInternalPath(relativeFilePath: string): boolean { + return getRagOwnerClassification(relativeFilePath)?.classification === "shared-internal"; +} + +export function listRagOwnerClassifications(): readonly RagOwnerClassificationEntry[] { + return RAG_OWNER_CLASSIFICATION_ENTRIES; +} + +export function isKnowledgeRagOwnerClassificationReady(): boolean { + const hasCompatibilitySupportCoverage = RAG_OWNER_CLASSIFICATION_ENTRIES.some( + (entry) => entry.classification === "compatibility-support" + ); + const hasLegacyActiveCoverage = RAG_OWNER_CLASSIFICATION_ENTRIES.some( + (entry) => entry.classification === "legacy-active" + ); + const hasSharedInternalCoverage = RAG_OWNER_CLASSIFICATION_ENTRIES.some( + (entry) => entry.classification === "shared-internal" + ); + return hasCompatibilitySupportCoverage && hasLegacyActiveCoverage && hasSharedInternalCoverage; +} diff --git a/apps/backend/src/modules/knowledge/rag/rag.module.ts b/apps/backend/src/modules/knowledge/rag/rag.module.ts index 58983f4..c13a8e2 100644 --- a/apps/backend/src/modules/knowledge/rag/rag.module.ts +++ b/apps/backend/src/modules/knowledge/rag/rag.module.ts @@ -26,11 +26,16 @@ import { RagQueryCacheService } from "../../rag/perf/rag-query-cache.service"; import { RagQualityController } from "../../rag/quality/rag-quality.controller"; import { RagQualityService } from "../../rag/quality/rag-quality.service"; import { RagRetrievalService as LegacyRagRetrievalService } from "../../rag/retrieval/rag-retrieval.service"; +import { SemanticAssetFamilyChunkMapper } from "./preparation/semantic-asset-family-chunk.mapper"; +import { SemanticAssetManifestBuilder } from "./preparation/semantic-asset-manifest.builder"; +import { SemanticAssetPreparerService } from "./preparation/semantic-asset-preparer.service"; +import { SemanticAssetReadinessService } from "./preparation/semantic-asset-readiness.service"; import { SemanticAssetReindexService } from "./retrieval/semantic-asset-reindex.service"; import { RagRetrievalService } from "./retrieval/rag-retrieval.service"; import { ModelRerankerAdapter } from "../../rag/rerank/model-reranker.adapter"; import { RagRerankService as LegacyRagRerankService } from "../../rag/rerank/rag-rerank.service"; import { RagRerankService } from "./rerank/rag-rerank.service"; +import { isKnowledgeRagOwnerClassificationReady } from "./rag-owner-classification"; export const KNOWLEDGE_COMPAT_BRIDGE_RETIREMENT_WINDOW = "next-milestone"; @@ -52,6 +57,9 @@ export function assertKnowledgeCompatBridgeRetirementReady( checks: KnowledgeCompatBridgeRetirementChecks ): void { const blockers: string[] = []; + if (!isKnowledgeRagOwnerClassificationReady()) { + blockers.push("rag owner classification map is incomplete"); + } if (!checks.conversationImportsClosed) { blockers.push("conversation imports are not fully migrated to knowledge facade contracts"); } @@ -97,6 +105,10 @@ export function assertKnowledgeCompatBridgeRetirementReady( RagAuditReplayService, LegacyRagReplayRepository, RagQualityService, + SemanticAssetFamilyChunkMapper, + SemanticAssetManifestBuilder, + SemanticAssetPreparerService, + SemanticAssetReadinessService, RagRetrievalService, SemanticAssetReindexService, ModelRerankerAdapter, @@ -135,6 +147,10 @@ export function assertKnowledgeCompatBridgeRetirementReady( LegacyRagReplayRepository, RagReplayRepository, RagQualityService, + SemanticAssetFamilyChunkMapper, + SemanticAssetManifestBuilder, + SemanticAssetPreparerService, + SemanticAssetReadinessService, LegacyRagRetrievalService, SemanticAssetReindexService, RagRetrievalService, diff --git a/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts b/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts index 1d1d63b..16f2993 100644 --- a/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts +++ b/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts @@ -222,7 +222,13 @@ export class RagRerankService { const degradeReasons = this.unique([...bundle.degrade_reasons, ...rerankDegradeReasons]); const selectedContext = reranked .slice(0, selectedContextLimit) - .map((item) => item.chunk); + .map((item) => ({ + ...item.chunk, + metadata: { + ...item.chunk.metadata, + lifecycleState: "selected" as const + } + })); const responseBundle: RagRetrievalBundle = { ...bundle, diff --git a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts index 1d15810..cb82f10 100644 --- a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts +++ b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts @@ -21,6 +21,7 @@ import { type RagContextPackPruningDecision, type RagPriorSqlLaneEvidence, type RagPriorSqlShortcutDecision, + type RagRetrievalBundle, type RagRetrievalCandidate, type RagRetrievalChunkMetadata, type RagRetrievalChunkPayload, @@ -108,6 +109,31 @@ interface PermissionFilteringResult { evidence: PermissionFilteringEvidence; } +interface ContextPermissionFilteringResult { + contexts: RagRetrievalEntryContext[]; + evidence: PermissionFilteringEvidence; +} + +interface TwoPassSchemaRecallEvidence { + status: "applied" | "skipped"; + selected_table_names: string[]; + table_description_evidence_ids: string[]; + supplemental_evidence_ids: string[]; + supplemental_families: string[]; + reason_codes: string[]; + selectedTableNames?: string[]; + tableDescriptionEvidenceIds?: string[]; + supplementalEvidenceIds?: string[]; + supplementalFamilies?: string[]; + reasonCodes?: string[]; +} + +interface TwoPassSchemaRecallResult { + candidates: RagRetrievalCandidate[]; + evidence: TwoPassSchemaRecallEvidence; + reasonCodes: string[]; +} + interface WideTableProfile { tableName: string; normalizedTableName: string; @@ -268,9 +294,13 @@ export class RagRetrievalService { const entries = await this.indexRepository.listEntriesByVersion(activeVersion.id); const contexts = entries.map((entry) => this.toEntryContext(activeVersion.id, entry)); + const preRankingPermissionFiltering = this.applyPermissionFilteringToContexts({ + contexts, + allowedTables + }); const laneResults = await this.collectLaneResults({ query, - contexts, + contexts: preRankingPermissionFiltering.contexts, perLaneLimit, laneTimeoutMs, enabledLanes: budgetDecision.enabledLanes, @@ -302,8 +332,13 @@ export class RagRetrievalService { priorSqlFiltered, priorSqlSelection.selectedCandidates ); + const schemaRecall = this.applyTwoPassSchemaRecall({ + candidates: fusedWithPrior, + contexts: preRankingPermissionFiltering.contexts, + limit: finalCandidateLimit + }); const coveredCandidates = this.applyDomainCoverage( - fusedWithPrior, + schemaRecall.candidates, finalCandidateLimit, REQUIRED_DOMAIN_COVERAGE ); @@ -317,11 +352,16 @@ export class RagRetrievalService { allowedTables }); const candidates = permissionFiltering.candidates; + const combinedPermissionFiltering = this.mergePermissionFilteringEvidence([ + preRankingPermissionFiltering.evidence, + permissionFiltering.evidence + ]); const skillContext = await this.resolveSkillContext(query, candidates); const degradeReasons = this.collectDegradeReasons(laneResults); degradeReasons.push(...this.collectSemanticLinkageDegradeReasons(candidates)); - degradeReasons.push(...permissionFiltering.evidence.reason_codes); + degradeReasons.push(...combinedPermissionFiltering.reason_codes); + degradeReasons.push(...schemaRecall.reasonCodes); if (skillContext.degrade_reason) { degradeReasons.push(skillContext.degrade_reason); } @@ -352,8 +392,9 @@ export class RagRetrievalService { this.attachColumnPruningEvidence(response.retrieval_bundle, columnPruning.evidence); this.attachPermissionFilteringEvidence( response.retrieval_bundle, - permissionFiltering.evidence + combinedPermissionFiltering ); + this.attachTwoPassSchemaRecallEvidence(response.retrieval_bundle, schemaRecall.evidence); response.retrieval_bundle.context_pack = await this.buildContextPack({ bundle: response.retrieval_bundle, workspaceId, @@ -390,6 +431,17 @@ export class RagRetrievalService { cacheHit: false, budgetDegraded: budgetDecision.degraded }); + this.ragQualityService.recordPreparationPlane({ + runId, + datasourceId, + manifestFingerprint: this.extractManifestFingerprint(activeVersion.sourceVersion), + activeIndexVersionId: activeVersion.id, + familyCounts: this.countCandidateAssetFamilies(candidates), + permissionFilteredAssetCount: combinedPermissionFiltering.denied_evidence_ids.length, + selectedAssetCount: candidates.length, + staleReasons: schemaRecall.reasonCodes.filter((reason) => reason.includes("stale")), + lifecycleStatus: "retrieved" + }); await this.persistReplay(response.retrieval_bundle); return response; } @@ -405,6 +457,7 @@ export class RagRetrievalService { const priorSqlLane = this.readPriorSqlLaneEvidence(input.cachedBundle); const columnPruning = this.readColumnPruningEvidence(input.cachedBundle); const permissionFiltering = this.readPermissionFilteringEvidence(input.cachedBundle); + const twoPassSchemaRecall = this.readTwoPassSchemaRecallEvidence(input.cachedBundle); const hydratedBundle: RagRetrievalResponse["retrieval_bundle"] = { ...input.cachedBundle, query: input.query, @@ -424,6 +477,7 @@ export class RagRetrievalService { }; this.attachColumnPruningEvidence(hydratedBundle, columnPruning); this.attachPermissionFilteringEvidence(hydratedBundle, permissionFiltering); + this.attachTwoPassSchemaRecallEvidence(hydratedBundle, twoPassSchemaRecall); hydratedBundle.context_pack = await this.buildContextPack({ bundle: hydratedBundle, workspaceId: input.workspaceId, @@ -806,6 +860,21 @@ export class RagRetrievalService { indexVersionId: context.indexVersionId, chunkId: context.entry.chunkId, domain: context.entry.domain, + assetFamily: this.readString(context.parsedMetadata.sourceMetadata.assetFamily), + manifestFingerprint: this.readString( + context.parsedMetadata.sourceMetadata.manifestFingerprint + ), + manifestEntryId: this.readString(context.parsedMetadata.sourceMetadata.manifestEntryId), + sourceRef: context.parsedMetadata.sourceMetadata.semanticAssetSourceRef, + sourceVersion: this.readString(context.parsedMetadata.sourceMetadata.sourceVersion), + policyVersion: this.readString(context.parsedMetadata.sourceMetadata.policyVersion), + modelingRevision: this.readNumber(context.parsedMetadata.sourceMetadata.modelingRevision), + visibilityScope: this.readString(context.parsedMetadata.sourceMetadata.visibilityScope), + preparationStatus: this.readString( + context.parsedMetadata.sourceMetadata.preparationStatus + ), + reasonCodes: this.readStringArray(context.parsedMetadata.sourceMetadata.reasonCodes), + lifecycleState: "retrieved", chunkProfile: this.readString(context.parsedMetadata.chunkProfile), startOffset: this.readNumber(context.parsedMetadata.startOffset), endOffset: this.readNumber(context.parsedMetadata.endOffset), @@ -1626,6 +1695,247 @@ export class RagRetrievalService { }; } + private applyPermissionFilteringToContexts(input: { + contexts: RagRetrievalEntryContext[]; + allowedTables: string[]; + }): ContextPermissionFilteringResult { + if (input.allowedTables.length === 0) { + return { + contexts: input.contexts, + evidence: { + status: "skipped", + denied_evidence_ids: [], + denied_table_names: [], + denied_column_names: [], + reason_codes: [], + kept_candidate_count: input.contexts.length, + deniedEvidenceIds: [], + deniedTableNames: [], + deniedColumnNames: [], + reasonCodes: [], + keptCandidateCount: input.contexts.length + } + }; + } + + const allowedTableSet = new Set(input.allowedTables.map((table) => table.trim().toLowerCase())); + const keptContexts: RagRetrievalEntryContext[] = []; + const deniedEvidenceIds: string[] = []; + const deniedTableNames: string[] = []; + const deniedColumnNames: string[] = []; + + for (const context of input.contexts) { + const tableNames = context.parsedMetadata.tableNames + .map((tableName) => tableName.trim().toLowerCase()) + .filter((tableName) => tableName.length > 0); + if (tableNames.length === 0) { + keptContexts.push(context); + continue; + } + const deniedTables = tableNames.filter((tableName) => !allowedTableSet.has(tableName)); + if (deniedTables.length === 0) { + keptContexts.push(context); + continue; + } + deniedEvidenceIds.push(context.entry.chunkId); + deniedTableNames.push(...deniedTables); + deniedColumnNames.push(...context.parsedMetadata.columnNames); + } + + const uniqueDeniedEvidenceIds = this.unique(deniedEvidenceIds).slice(0, 128); + const uniqueDeniedTableNames = this.unique(deniedTableNames).slice(0, 64); + const uniqueDeniedColumnNames = this.unique(deniedColumnNames).slice(0, 128); + const reasonCodes = + uniqueDeniedEvidenceIds.length > 0 + ? ["permission_filtered_before_ranking", "permission_filtered_not_in_allowed_tables"] + : []; + + return { + contexts: keptContexts, + evidence: { + status: uniqueDeniedEvidenceIds.length > 0 ? "applied" : "skipped", + denied_evidence_ids: uniqueDeniedEvidenceIds, + denied_table_names: uniqueDeniedTableNames, + denied_column_names: uniqueDeniedColumnNames, + reason_codes: reasonCodes, + kept_candidate_count: keptContexts.length, + deniedEvidenceIds: uniqueDeniedEvidenceIds, + deniedTableNames: uniqueDeniedTableNames, + deniedColumnNames: uniqueDeniedColumnNames, + reasonCodes, + keptCandidateCount: keptContexts.length + } + }; + } + + private mergePermissionFilteringEvidence( + evidences: PermissionFilteringEvidence[] + ): PermissionFilteringEvidence { + const applied = evidences.some((evidence) => evidence.status === "applied"); + const deniedEvidenceIds = this.unique( + evidences.flatMap((evidence) => evidence.denied_evidence_ids ?? []) + ).slice(0, 128); + const deniedTableNames = this.unique( + evidences.flatMap((evidence) => evidence.denied_table_names ?? []) + ).slice(0, 64); + const deniedColumnNames = this.unique( + evidences.flatMap((evidence) => evidence.denied_column_names ?? []) + ).slice(0, 128); + const reasonCodes = this.unique( + evidences.flatMap((evidence) => evidence.reason_codes ?? []) + ); + const keptCandidateCount = evidences[evidences.length - 1]?.kept_candidate_count ?? 0; + return { + status: applied ? "applied" : "skipped", + denied_evidence_ids: deniedEvidenceIds, + denied_table_names: deniedTableNames, + denied_column_names: deniedColumnNames, + reason_codes: reasonCodes, + kept_candidate_count: keptCandidateCount, + deniedEvidenceIds, + deniedTableNames, + deniedColumnNames, + reasonCodes, + keptCandidateCount + }; + } + + private applyTwoPassSchemaRecall(input: { + candidates: RagRetrievalCandidate[]; + contexts: RagRetrievalEntryContext[]; + limit: number; + }): TwoPassSchemaRecallResult { + const tableDescriptionCandidates = input.candidates.filter( + (candidate) => this.readAssetFamily(candidate.chunk.metadata) === "table_description" + ); + const selectedTableNames = this.unique( + tableDescriptionCandidates + .flatMap((candidate) => candidate.chunk.metadata.tableNames) + .map((tableName) => tableName.trim().toLowerCase()) + .filter((tableName) => tableName.length > 0) + ); + if (selectedTableNames.length === 0) { + const evidence = this.emptyTwoPassSchemaRecallEvidence("two_pass_schema_recall_no_table_description"); + return { + candidates: input.candidates, + evidence, + reasonCodes: [] + }; + } + + const supplementalFamilies = new Set(["full_schema", "column_batch", "relationship_binding"]); + const existingSupplementalCandidates = input.candidates.filter((candidate) => { + const assetFamily = this.readAssetFamily(candidate.chunk.metadata); + if (!assetFamily || !supplementalFamilies.has(assetFamily)) { + return false; + } + const tableNames = candidate.chunk.metadata.tableNames.map((tableName) => + tableName.trim().toLowerCase() + ); + return tableNames.some((tableName) => selectedTableNames.includes(tableName)); + }); + const existingChunkIds = new Set(input.candidates.map((candidate) => candidate.chunk_id)); + const supplementalCandidates: RagRetrievalCandidate[] = []; + for (const context of input.contexts) { + if (existingChunkIds.has(context.entry.chunkId)) { + continue; + } + const chunk = this.toChunkPayload(context); + const assetFamily = this.readAssetFamily(chunk.metadata); + if (!assetFamily || !supplementalFamilies.has(assetFamily)) { + continue; + } + const tableNames = chunk.metadata.tableNames.map((tableName) => + tableName.trim().toLowerCase() + ); + const overlapsSelectedTable = tableNames.some((tableName) => + selectedTableNames.includes(tableName) + ); + const isRelationshipSupplement = + assetFamily === "relationship_binding" && + tableNames.some((tableName) => selectedTableNames.includes(tableName)); + if (!overlapsSelectedTable && !isRelationshipSupplement) { + continue; + } + supplementalCandidates.push({ + chunk_id: chunk.chunk_id, + source_lane: "graph", + evidence: this.unique([ + "two_pass_schema_recall", + `asset_family:${assetFamily}`, + ...chunk.metadata.tableNames.map((tableName) => `table:${tableName}`) + ]), + score: 0.01, + lane_scores: { + graph: 0.01 + }, + lane_ranks: { + graph: input.candidates.length + supplementalCandidates.length + 1 + }, + chunk + }); + existingChunkIds.add(chunk.chunk_id); + if (input.candidates.length + supplementalCandidates.length >= input.limit) { + break; + } + } + + const allSupplementalCandidates = [ + ...existingSupplementalCandidates, + ...supplementalCandidates + ]; + const supplementalEvidenceIds = this.unique( + allSupplementalCandidates.map((candidate) => candidate.chunk_id) + ); + const supplementalFamilyList = this.unique( + allSupplementalCandidates + .map((candidate) => this.readAssetFamily(candidate.chunk.metadata)) + .filter((family): family is string => Boolean(family)) + ).sort(); + const reasonCodes = + supplementalEvidenceIds.length > 0 + ? ["two_pass_schema_recall_applied"] + : ["two_pass_schema_recall_no_supplement"]; + const evidence: TwoPassSchemaRecallEvidence = { + status: supplementalEvidenceIds.length > 0 ? "applied" : "skipped", + selected_table_names: selectedTableNames, + table_description_evidence_ids: tableDescriptionCandidates.map( + (candidate) => candidate.chunk_id + ), + supplemental_evidence_ids: supplementalEvidenceIds, + supplemental_families: supplementalFamilyList, + reason_codes: reasonCodes, + selectedTableNames: selectedTableNames, + tableDescriptionEvidenceIds: tableDescriptionCandidates.map( + (candidate) => candidate.chunk_id + ), + supplementalEvidenceIds: supplementalEvidenceIds, + supplementalFamilies: supplementalFamilyList, + reasonCodes + }; + return { + candidates: [...input.candidates, ...supplementalCandidates], + evidence, + reasonCodes + }; + } + + private emptyTwoPassSchemaRecallEvidence(reasonCode: string): TwoPassSchemaRecallEvidence { + return { + status: "skipped", + selected_table_names: [], + table_description_evidence_ids: [], + supplemental_evidence_ids: [], + supplemental_families: [], + reason_codes: [reasonCode], + selectedTableNames: [], + tableDescriptionEvidenceIds: [], + supplementalEvidenceIds: [], + supplementalFamilies: [], + reasonCodes: [reasonCode] + }; + } + private buildFieldIntentTokens( queryTokens: string[], normalizedTableName: string @@ -1850,12 +2160,63 @@ export class RagRetrievalService { return source.permission_filtering ?? source.permissionFiltering; } + private attachTwoPassSchemaRecallEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"], + evidence: TwoPassSchemaRecallEvidence | undefined + ): void { + const target = bundle as RagRetrievalResponse["retrieval_bundle"] & { + two_pass_schema_recall?: TwoPassSchemaRecallEvidence; + twoPassSchemaRecall?: TwoPassSchemaRecallEvidence; + }; + if (!evidence) { + delete target.two_pass_schema_recall; + delete target.twoPassSchemaRecall; + return; + } + target.two_pass_schema_recall = evidence; + target.twoPassSchemaRecall = evidence; + } + + private readTwoPassSchemaRecallEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"] + ): TwoPassSchemaRecallEvidence | undefined { + const source = bundle as RagRetrievalResponse["retrieval_bundle"] & { + two_pass_schema_recall?: TwoPassSchemaRecallEvidence; + twoPassSchemaRecall?: TwoPassSchemaRecallEvidence; + }; + return source.two_pass_schema_recall ?? source.twoPassSchemaRecall; + } + private readDenseVectorMetadata( context: RagRetrievalEntryContext ): DenseVectorMetadata | undefined { return context.parsedMetadata.denseMetadata as DenseVectorMetadata | undefined; } + private readAssetFamily(metadata: RagRetrievalChunkMetadata): string | undefined { + return ( + this.readString(metadata.assetFamily) ?? + this.readString(metadata.sourceMetadata?.assetFamily) + ); + } + + private countCandidateAssetFamilies( + candidates: RagRetrievalBundle["candidates"] + ): Record { + return candidates.reduce>((accumulator, candidate) => { + const family = this.readAssetFamily(candidate.chunk.metadata); + if (family) { + accumulator[family] = (accumulator[family] ?? 0) + 1; + } + return accumulator; + }, {}); + } + + private extractManifestFingerprint(sourceVersion: string): string | undefined { + const [fingerprint] = sourceVersion.trim().split(":"); + return fingerprint || undefined; + } + private readDenseVectorMetadataFromParsed( parsed: Record, sourceMetadata: Record @@ -2125,6 +2486,16 @@ export class RagRetrievalService { const priorSqlLane = bundle.prior_sql_lane ?? bundle.priorSqlLane; const permissionFiltering = this.readPermissionFilteringEvidence(bundle); const permissionReasonCodes = this.unique(permissionFiltering?.reason_codes ?? []); + const twoPassSchemaRecall = this.readTwoPassSchemaRecallEvidence(bundle); + const semanticAssetCandidates = bundle.candidates.filter((candidate) => + Boolean(this.readAssetFamily(candidate.chunk.metadata)) + ); + const semanticAssetFamilyReasonCodes = this.unique([ + ...(twoPassSchemaRecall?.reason_codes ?? []), + ...semanticAssetCandidates.flatMap( + (candidate) => candidate.chunk.metadata.reasonCodes ?? [] + ) + ]); laneMetadata.push( { @@ -2205,6 +2576,24 @@ export class RagRetrievalService { reason_codes: this.unique(priorSqlLane?.degrade_reasons ?? []), reasonCodes: this.unique(priorSqlLane?.degrade_reasons ?? []) }, + { + lane: "semantic_asset_family", + state: semanticAssetCandidates.length > 0 ? "ready" : "skipped", + input_count: semanticAssetCandidates.length, + inputCount: semanticAssetCandidates.length, + output_count: semanticAssetCandidates.length, + outputCount: semanticAssetCandidates.length, + selected_count: selectedContext.filter((chunk) => + Boolean(this.readAssetFamily(chunk.metadata)) + ).length, + selectedCount: selectedContext.filter((chunk) => + Boolean(this.readAssetFamily(chunk.metadata)) + ).length, + evidence_ids: semanticAssetCandidates.map((candidate) => candidate.chunk_id), + evidenceIds: semanticAssetCandidates.map((candidate) => candidate.chunk_id), + reason_codes: semanticAssetFamilyReasonCodes, + reasonCodes: semanticAssetFamilyReasonCodes + }, { lane: "dialect_function", state: @@ -2288,6 +2677,22 @@ export class RagRetrievalService { }); } + const twoPassSchemaRecall = this.readTwoPassSchemaRecallEvidence(bundle); + if (twoPassSchemaRecall && twoPassSchemaRecall.reason_codes.length > 0) { + const reasonCodes = this.unique(twoPassSchemaRecall.reason_codes); + decisions.push({ + budget_source: "context_pack", + removed_evidence_ids: [], + kept_evidence_ids: twoPassSchemaRecall.supplemental_evidence_ids, + reason_codes: reasonCodes, + summary: `context_pack:two_pass_schema_recall:${reasonCodes.join("|")}`, + budgetSource: "context_pack", + removedEvidenceIds: [], + keptEvidenceIds: twoPassSchemaRecall.supplemental_evidence_ids, + reasonCodes + }); + } + return decisions; } @@ -2528,12 +2933,17 @@ export class RagRetrievalService { bundle.context_pack?.pruningDecisions ?? [], permissionFiltering: this.readPermissionFilteringEvidence(bundle), + twoPassSchemaRecall: this.readTwoPassSchemaRecallEvidence(bundle), skillContext: bundle.skill_context, candidates: bundle.candidates.map((candidate) => ({ chunkId: candidate.chunk_id, sourceLane: candidate.source_lane, score: candidate.score, - domain: candidate.chunk.metadata.domain + domain: candidate.chunk.metadata.domain, + assetFamily: this.readAssetFamily(candidate.chunk.metadata), + manifestFingerprint: candidate.chunk.metadata.manifestFingerprint, + sourceVersion: candidate.chunk.metadata.sourceVersion, + lifecycleState: candidate.chunk.metadata.lifecycleState })) } }); diff --git a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts index 76a501c..5653aa3 100644 --- a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts +++ b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts @@ -28,6 +28,17 @@ export interface RagRetrievalChunkMetadata { indexVersionId: string; chunkId: string; domain: string; + assetFamily?: string; + manifestFingerprint?: string; + manifestEntryId?: string; + sourceRef?: unknown; + sourceVersion?: string; + policyVersion?: string; + modelingRevision?: number; + visibilityScope?: string; + preparationStatus?: string; + reasonCodes?: string[]; + lifecycleState?: "retrieved" | "selected" | "pruned" | "filtered" | "unused"; chunkProfile?: string; startOffset?: number; endOffset?: number; @@ -121,6 +132,7 @@ export interface RagContextPackLaneMetadata { | "instruction" | "saved_prior_sql" | "schema_ddl_supplement" + | "semantic_asset_family" | "dialect_function"; state: "ready" | "degraded" | "unavailable" | "skipped"; provider?: string; @@ -291,6 +303,34 @@ export interface RagRetrievalBundle { laneMetadata?: RagContextPackLaneMetadata[]; pruning_decisions?: RagContextPackPruningDecision[]; pruningDecisions?: RagContextPackPruningDecision[]; + permission_filtering?: { + status?: "applied" | "skipped"; + denied_evidence_ids?: string[]; + denied_table_names?: string[]; + denied_column_names?: string[]; + reason_codes?: string[]; + kept_candidate_count?: number; + deniedEvidenceIds?: string[]; + deniedTableNames?: string[]; + deniedColumnNames?: string[]; + reasonCodes?: string[]; + keptCandidateCount?: number; + }; + permissionFiltering?: RagRetrievalBundle["permission_filtering"]; + two_pass_schema_recall?: { + status?: "applied" | "skipped"; + selected_table_names?: string[]; + table_description_evidence_ids?: string[]; + supplemental_evidence_ids?: string[]; + supplemental_families?: string[]; + reason_codes?: string[]; + selectedTableNames?: string[]; + tableDescriptionEvidenceIds?: string[]; + supplementalEvidenceIds?: string[]; + supplementalFamilies?: string[]; + reasonCodes?: string[]; + }; + twoPassSchemaRecall?: RagRetrievalBundle["two_pass_schema_recall"]; } export interface RagRetrievalResponse { diff --git a/apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts b/apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts index 5b52bb0..0218dfd 100644 --- a/apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts +++ b/apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts @@ -1,10 +1,21 @@ -import { createHash } from "node:crypto"; import { Injectable } from "@nestjs/common"; import { AppConfigService } from "../../../config/app-config.service"; import { RagTaskConfigService } from "../../../llm/rag-task-config.service"; import { ModelingGraphRepository } from "../../../platform/data/persistence/modeling-graph.repository"; import { RagIndexRepository } from "../../../rag/index/rag-index.repository"; +import type { RagChunkBuildInput } from "../../../rag/index/rag-index.repository"; +import type { RagIndexBuildManifestContext } from "../../../rag/index/rag-index-builder.service"; +import { RagDocumentFactory } from "../../../rag/ingestion/rag-document.factory"; import { BuildRagIndexJob } from "../../../rag/jobs/build-rag-index.job"; +import { RagQualityService } from "../../../rag/quality/rag-quality.service"; +import type { + SemanticAssetManifest, + SemanticAssetManifestSummary +} from "../preparation/semantic-asset-manifest.types"; +import { SemanticAssetReadinessService } from "../preparation/semantic-asset-readiness.service"; +import { SemanticAssetPreparerService } from "../preparation/semantic-asset-preparer.service"; +import type { SemanticAssetSourceSnapshot } from "../preparation/semantic-asset-source-snapshot.types"; +import { SEMANTIC_ASSET_REASON_CODES } from "../preparation/semantic-asset-reason-codes"; export const SEMANTIC_ASSET_REINDEX_TRIGGERS = [ "schema", @@ -25,6 +36,7 @@ export interface SemanticAssetReindexRequest { runId?: string; force?: boolean; sourceVersion?: string; + sourceSnapshots?: SemanticAssetSourceSnapshot[]; } export interface SemanticAssetReindexResult { @@ -43,8 +55,31 @@ export interface SemanticAssetReindexResult { triggerSummary: Record; modelingRevision?: number; reasonCodes: string[]; + runId?: string; + manifestSummary: SemanticAssetManifestSummary; + preparedSourceCount?: number; + typedChunkInputCount?: number; + excludedSourceCount?: number; indexVersionId?: string; entryCount?: number; + lifecycle: SemanticAssetReindexLifecycleSummary; +} + +export interface SemanticAssetReindexLifecycleSummary { + status: + | "skipped_no_triggers" + | "missing_active" + | "already_active" + | "stale_rebuildable" + | "activated"; + latestManifestFingerprint: string; + activeManifestFingerprint?: string; + activeIndexVersionId?: string; + activeSourceVersion?: string; + activatedIndexVersionId?: string; + previousActiveIndexVersionId?: string; + rebuildable: boolean; + staleReasons: string[]; } @Injectable() @@ -54,7 +89,11 @@ export class SemanticAssetReindexService { private readonly ragTaskConfigService: RagTaskConfigService, private readonly indexRepository: RagIndexRepository, private readonly buildIndexJob: BuildRagIndexJob, - private readonly modelingGraphRepository: ModelingGraphRepository + private readonly modelingGraphRepository: ModelingGraphRepository, + private readonly semanticAssetPreparer: SemanticAssetPreparerService, + private readonly ragDocumentFactory: RagDocumentFactory, + private readonly semanticAssetReadiness: SemanticAssetReadinessService, + private readonly ragQualityService: RagQualityService ) {} async reindex(input: SemanticAssetReindexRequest): Promise { @@ -68,14 +107,23 @@ export class SemanticAssetReindexService { includeModelingRevision: triggerSummary.modeling_revision }); const embeddingProfile = await this.resolveEmbeddingProfile(); - - const semanticAssetVersion = this.buildSemanticAssetVersion({ + const reasonCodes = this.unique([ + input.reason?.trim() || SEMANTIC_ASSET_REASON_CODES.semanticAssetReindexRequested, + ...triggers.map((trigger) => `trigger:${trigger}`) + ]); + const preparation = this.semanticAssetPreparer.prepare({ datasourceId, workspaceId, triggers, modelingRevision, - embeddingProfile + embeddingProfile, + sourceVersion: input.sourceVersion?.trim(), + reasonCodes, + sourceSnapshots: input.sourceSnapshots ?? [] }); + const manifestSummary = preparation.manifestSummary; + const manifest = preparation.manifest; + const semanticAssetVersion = manifest.fingerprint; const sourceVersion = input.sourceVersion?.trim() || this.buildSourceVersion({ @@ -84,12 +132,8 @@ export class SemanticAssetReindexService { embeddingProfile }); - const reasonCodes = this.unique([ - input.reason?.trim() || "semantic_asset_reindex_requested", - ...triggers.map((trigger) => `trigger:${trigger}`) - ]); if (triggers.length === 0) { - return { + const result: SemanticAssetReindexResult = { status: "skipped", datasourceId, workspaceId, @@ -98,13 +142,31 @@ export class SemanticAssetReindexService { embeddingProfile, triggerSummary, modelingRevision, - reasonCodes: [...reasonCodes, "skip:no_triggers"] + reasonCodes: [...reasonCodes, "skip:no_triggers"], + runId: input.runId?.trim() || undefined, + manifestSummary, + preparedSourceCount: preparation.sourceSnapshots.length, + typedChunkInputCount: preparation.ingestionSources.length, + excludedSourceCount: preparation.excludedSnapshots.length, + lifecycle: { + status: "skipped_no_triggers", + latestManifestFingerprint: semanticAssetVersion, + rebuildable: false, + staleReasons: [] + } }; + this.recordReadinessAndQuality(result); + return result; } const activeVersion = await this.indexRepository.getActiveVersion(datasourceId); + const lifecycleBeforeBuild = this.resolveLifecycle({ + activeVersion, + sourceVersion, + semanticAssetVersion + }); if (!input.force && activeVersion?.sourceVersion === sourceVersion) { - return { + const result: SemanticAssetReindexResult = { status: "skipped", datasourceId, workspaceId, @@ -113,8 +175,20 @@ export class SemanticAssetReindexService { embeddingProfile, triggerSummary, modelingRevision, - reasonCodes: [...reasonCodes, "skip:already_active"] + reasonCodes: [...reasonCodes, SEMANTIC_ASSET_REASON_CODES.skippedAlreadyActive], + runId: input.runId?.trim() || undefined, + manifestSummary, + preparedSourceCount: preparation.sourceSnapshots.length, + typedChunkInputCount: preparation.ingestionSources.length, + excludedSourceCount: preparation.excludedSnapshots.length, + lifecycle: { + ...lifecycleBeforeBuild, + status: "already_active", + rebuildable: false + } }; + this.recordReadinessAndQuality(result); + return result; } const runId = @@ -124,10 +198,12 @@ export class SemanticAssetReindexService { datasourceId, sourceVersion, buildReason: `semantic_asset_reindex:${reasonCodes.join("|")}`, - runId + runId, + chunks: this.buildTypedChunks(preparation.ingestionSources), + manifest: this.toBuildManifestContext(manifest, manifestSummary) }); - return { + const reindexResult: SemanticAssetReindexResult = { status: "reindexed", datasourceId, workspaceId, @@ -137,9 +213,141 @@ export class SemanticAssetReindexService { triggerSummary, modelingRevision, reasonCodes, + runId, + manifestSummary, + preparedSourceCount: preparation.sourceSnapshots.length, + typedChunkInputCount: preparation.ingestionSources.length, + excludedSourceCount: preparation.excludedSnapshots.length, indexVersionId: result.indexVersionId, - entryCount: result.entryCount + entryCount: result.entryCount, + lifecycle: { + ...lifecycleBeforeBuild, + status: "activated", + activatedIndexVersionId: result.indexVersionId, + previousActiveIndexVersionId: activeVersion?.id, + rebuildable: false + } }; + this.recordReadinessAndQuality(reindexResult); + return reindexResult; + } + + private recordReadinessAndQuality(result: SemanticAssetReindexResult): void { + if (result.lifecycle.status === "activated" || result.lifecycle.status === "already_active") { + this.semanticAssetReadiness.recordActivation({ + datasourceId: result.datasourceId, + workspaceId: result.workspaceId, + runId: result.runId, + sourceVersion: result.sourceVersion, + indexVersionId: + result.lifecycle.activatedIndexVersionId ?? + result.lifecycle.activeIndexVersionId ?? + result.indexVersionId, + manifestSummary: result.manifestSummary, + previousActiveIndexVersionId: result.lifecycle.previousActiveIndexVersionId, + staleReasons: result.lifecycle.staleReasons + }); + } + this.ragQualityService.recordPreparationPlane({ + runId: + result.runId ?? + result.lifecycle.activatedIndexVersionId ?? + result.lifecycle.activeIndexVersionId ?? + result.indexVersionId ?? + result.datasourceId, + datasourceId: result.datasourceId, + manifestFingerprint: result.semanticAssetVersion, + activeIndexVersionId: + result.lifecycle.activatedIndexVersionId ?? + result.lifecycle.activeIndexVersionId ?? + result.indexVersionId, + familyCounts: result.manifestSummary.familyCounts as Record, + preparedEntryCount: result.manifestSummary.preparedEntryCount, + degradedEntryCount: result.manifestSummary.degradedEntryCount, + skippedEntryCount: result.manifestSummary.skippedEntryCount, + staleReasons: result.lifecycle.staleReasons, + lifecycleStatus: result.lifecycle.status + }); + } + + private buildTypedChunks( + ingestionSources: ReturnType["ingestionSources"] + ): RagChunkBuildInput[] { + return ingestionSources.flatMap((source) => { + const build = this.ragDocumentFactory.create(source); + return build.chunks.map((chunk) => ({ + id: chunk.id, + datasourceId: chunk.datasourceId, + domain: chunk.domain, + content: chunk.content, + metadata: chunk.metadata + })); + }); + } + + private toBuildManifestContext( + manifest: SemanticAssetManifest, + summary: SemanticAssetManifestSummary + ): RagIndexBuildManifestContext { + return { + fingerprint: manifest.fingerprint, + summary: { + entryCount: summary.entryCount, + preparedEntryCount: summary.preparedEntryCount, + skippedEntryCount: summary.skippedEntryCount, + degradedEntryCount: summary.degradedEntryCount, + familyCounts: summary.familyCounts as Record, + reasonCodes: summary.reasonCodes, + embeddingProfile: summary.embeddingProfile, + sourceSnapshotSummary: summary.sourceSnapshotSummary + }, + entries: manifest.entries.map((entry) => ({ + id: entry.id, + family: entry.family, + status: entry.status, + sourceRef: entry.sourceRef, + sourceVersion: entry.sourceVersion, + reasonCodes: entry.reasonCodes + })) + }; + } + + private resolveLifecycle(input: { + activeVersion?: { id: string; sourceVersion: string }; + sourceVersion: string; + semanticAssetVersion: string; + }): SemanticAssetReindexLifecycleSummary { + if (!input.activeVersion) { + return { + status: "missing_active", + latestManifestFingerprint: input.semanticAssetVersion, + rebuildable: true, + staleReasons: [] + }; + } + const activeManifestFingerprint = this.extractManifestFingerprint( + input.activeVersion.sourceVersion + ); + const alreadyActive = input.activeVersion.sourceVersion === input.sourceVersion; + return { + status: alreadyActive ? "already_active" : "stale_rebuildable", + latestManifestFingerprint: input.semanticAssetVersion, + activeManifestFingerprint, + activeIndexVersionId: input.activeVersion.id, + activeSourceVersion: input.activeVersion.sourceVersion, + rebuildable: !alreadyActive, + staleReasons: alreadyActive + ? [] + : [SEMANTIC_ASSET_REASON_CODES.staleSourceVersion] + }; + } + + private extractManifestFingerprint(sourceVersion: string): string | undefined { + const normalized = sourceVersion.trim(); + if (!normalized) { + return undefined; + } + return normalized.split(":")[0]; } private normalizeTriggers(triggers: SemanticAssetReindexTrigger[]): SemanticAssetReindexTrigger[] { @@ -168,34 +376,6 @@ export class SemanticAssetReindexService { }; } - private buildSemanticAssetVersion(input: { - datasourceId: string; - workspaceId?: string; - triggers: SemanticAssetReindexTrigger[]; - modelingRevision?: number; - embeddingProfile: { - provider: string; - model: string; - dimensions?: number; - vectorVersion: string; - configSource: "settings" | "env_fallback" | "missing"; - }; - }): string { - const fingerprint = createHash("sha256") - .update( - JSON.stringify({ - datasourceId: input.datasourceId, - workspaceId: input.workspaceId, - triggers: [...input.triggers].sort(), - modelingRevision: input.modelingRevision, - embeddingProfile: input.embeddingProfile - }) - ) - .digest("hex") - .slice(0, 16); - return `semantic-assets-${fingerprint}`; - } - private buildSourceVersion(input: { semanticAssetVersion: string; triggers: SemanticAssetReindexTrigger[]; diff --git a/apps/backend/src/modules/llm/llm-gateway.interface.ts b/apps/backend/src/modules/llm/llm-gateway.interface.ts index c049184..1b98a09 100644 --- a/apps/backend/src/modules/llm/llm-gateway.interface.ts +++ b/apps/backend/src/modules/llm/llm-gateway.interface.ts @@ -59,6 +59,7 @@ export interface LlmGateway { prompt: LlmGatewayPrompt, runtime: LlmGatewayRuntimeConfig, options?: { + abortSignal?: AbortSignal; tools?: Record; onEvent?: (event: LlmGatewayStreamEvent) => Promise | void; } diff --git a/apps/backend/src/modules/llm/llm-gateway.service.ts b/apps/backend/src/modules/llm/llm-gateway.service.ts index 48b2776..2a326a0 100644 --- a/apps/backend/src/modules/llm/llm-gateway.service.ts +++ b/apps/backend/src/modules/llm/llm-gateway.service.ts @@ -1,5 +1,11 @@ import { Injectable } from "@nestjs/common"; import { generateText, streamText, tool } from "ai"; +import { + composeAbortSignals, + createUserCancelledError, + isUserCancelledError, + throwIfAborted +} from "../../common/abort-utils"; import { DomainError } from "../../common/domain-error"; import { AppConfigService } from "../config/app-config.service"; import { LlmModelFactory } from "./llm-model-factory"; @@ -31,6 +37,32 @@ const isTimeoutAbortError = (error: unknown): boolean => { ); }; +const isUserAbortError = (error: unknown): boolean => { + if (isUserCancelledError(error)) { + return true; + } + if (!(error instanceof Error)) { + return false; + } + const normalized = `${error.name} ${error.message}`.toLowerCase(); + return normalized.includes("abort") && !normalized.includes("timeout"); +}; + +const isInvalidJsonResponseError = (error: unknown): boolean => { + if (!(error instanceof Error)) { + return false; + } + return `${error.name} ${error.message}`.toLowerCase().includes("invalid json response"); +}; + +const isRetriableGenerateError = (error: unknown): boolean => + isTimeoutAbortError(error) || isInvalidJsonResponseError(error); + +const isRecoverableStreamTransportError = (error: unknown): boolean => + isTimeoutAbortError(error) || isInvalidJsonResponseError(error); + +const MAX_GENERATE_RETRY = 1; + @Injectable() export class LlmGatewayService implements LlmGateway { constructor( @@ -84,26 +116,7 @@ export class LlmGatewayService implements LlmGateway { } try { - const model = this.modelFactory.createChatModel(runtime) as never; - const result = await generateText({ - model, - system: prompt.systemPrompt, - prompt: prompt.userPrompt, - abortSignal: AbortSignal.timeout(runtime.timeoutMs), - temperature: 0.2 - }); - - const content = result.text?.trim(); - if (!content) { - throw new DomainError( - "LLM_EMPTY_RESPONSE", - "LLM 返回为空,无法生成 SQL。", - 502, - { - provider: runtime.provider - } - ); - } + const content = await this.generateWithRetry(prompt, runtime); return { provider: runtime.provider, @@ -127,19 +140,84 @@ export class LlmGatewayService implements LlmGateway { } } + private async generateWithRetry( + prompt: LlmGatewayPrompt, + runtime: LlmGatewayRuntimeConfig + ): Promise { + let lastError: unknown; + for (let attempt = 0; attempt <= MAX_GENERATE_RETRY; attempt += 1) { + try { + return await this.executeGenerate(prompt, runtime, attempt); + } catch (error) { + lastError = error; + if (error instanceof DomainError) { + throw error; + } + if (!isRetriableGenerateError(error) || attempt >= MAX_GENERATE_RETRY) { + throw error; + } + } + } + + throw lastError; + } + + private async executeGenerate( + prompt: LlmGatewayPrompt, + runtime: LlmGatewayRuntimeConfig, + attempt: number + ): Promise { + const model = this.modelFactory.createChatModel(runtime) as never; + const timeoutMs = + attempt === 0 + ? runtime.timeoutMs + : Math.min( + Math.max(runtime.timeoutMs * 2, runtime.timeoutMs + 15000), + 120000 + ); + const result = await generateText({ + model, + system: prompt.systemPrompt, + prompt: prompt.userPrompt, + abortSignal: AbortSignal.timeout(timeoutMs), + temperature: 0.2 + }); + + const content = result.text?.trim(); + if (!content) { + throw new DomainError( + "LLM_EMPTY_RESPONSE", + "LLM 返回为空,无法生成 SQL。", + 502, + { + provider: runtime.provider + } + ); + } + + return content; + } + async stream( prompt: LlmGatewayPrompt, runtime: LlmGatewayRuntimeConfig, options?: { + abortSignal?: AbortSignal; tools?: Record; onEvent?: (event: LlmGatewayStreamEvent) => Promise | void; } ): Promise { if (this.config.llmMockMode) { + throwIfAborted(options?.abortSignal, { + phase: "llm_mock_stream_start" + }); const simulated = ( await this.generate(prompt, runtime) ).rawText; for (const line of simulated.split("\n")) { + throwIfAborted(options?.abortSignal, { + phase: "llm_mock_stream_chunk" + }); await options?.onEvent?.({ type: "text-delta", text: `${line}\n` @@ -161,16 +239,26 @@ export class LlmGatewayService implements LlmGateway { const model = this.modelFactory.createChatModel(runtime) as never; const normalizedTools = this.normalizeTools(options?.tools); const streamTimeoutMs = runtime.streamTimeoutMs ?? runtime.timeoutMs; + const streamAbortSignal = composeAbortSignals([ + AbortSignal.timeout(streamTimeoutMs), + options?.abortSignal + ]); + throwIfAborted(options?.abortSignal, { + phase: "llm_stream_start" + }); const result = streamText({ model, system: prompt.systemPrompt, prompt: prompt.userPrompt, temperature: 0.2, - abortSignal: AbortSignal.timeout(streamTimeoutMs), + abortSignal: streamAbortSignal, tools: normalizedTools }); for await (const chunk of result.fullStream) { + throwIfAborted(options?.abortSignal, { + phase: "llm_stream_chunk" + }); if (chunk.type === "text-delta") { streamedText += chunk.text; await options?.onEvent?.({ @@ -225,6 +313,9 @@ export class LlmGatewayService implements LlmGateway { } } + throwIfAborted(options?.abortSignal, { + phase: "llm_stream_finish" + }); const fullText = (await result.text).trim() || streamedText.trim(); if (!fullText) { if (toolCallSql) { @@ -255,6 +346,11 @@ export class LlmGatewayService implements LlmGateway { rawText: fullText }; } catch (error) { + if (isUserAbortError(error) || options?.abortSignal?.aborted) { + throw createUserCancelledError({ + provider: runtime.provider + }); + } if (error instanceof DomainError) { throw error; } @@ -276,8 +372,8 @@ export class LlmGatewayService implements LlmGateway { }; } - if (isTimeoutAbortError(error)) { - const recoveredText = await this.tryRecoverFromStreamTimeout( + if (isRecoverableStreamTransportError(error)) { + const recoveredText = await this.tryRecoverFromStreamTransportError( prompt, runtime, streamedText @@ -310,7 +406,7 @@ export class LlmGatewayService implements LlmGateway { } } - private async tryRecoverFromStreamTimeout( + private async tryRecoverFromStreamTransportError( prompt: LlmGatewayPrompt, runtime: LlmGatewayRuntimeConfig, streamedText: string diff --git a/apps/backend/src/modules/llm/provider-router.service.ts b/apps/backend/src/modules/llm/provider-router.service.ts index 4e3229a..e3a9c6d 100644 --- a/apps/backend/src/modules/llm/provider-router.service.ts +++ b/apps/backend/src/modules/llm/provider-router.service.ts @@ -144,6 +144,7 @@ export class ProviderRouterService { modelCatalogId?: string; }, options?: { + abortSignal?: AbortSignal; tools?: Record; onEvent?: (event: LlmGatewayStreamEvent) => Promise | void; } diff --git a/apps/backend/src/modules/rag/audit/rag-audit-replay.service.ts b/apps/backend/src/modules/rag/audit/rag-audit-replay.service.ts index e7421c4..fc29f56 100644 --- a/apps/backend/src/modules/rag/audit/rag-audit-replay.service.ts +++ b/apps/backend/src/modules/rag/audit/rag-audit-replay.service.ts @@ -65,6 +65,19 @@ export interface RagAuditReplayChain { runId: string; requestId?: string; runTrace?: ExecutionTrace; + preparationPlane?: { + manifestFingerprints: string[]; + activeManifestFingerprint?: string; + activatedIndexVersionIds: string[]; + preparedFamilies: string[]; + degradedFamilies: string[]; + skippedFamilies: string[]; + filteredAssetCount: number; + selectedAssetCount: number; + unusedAssetCount: number; + lifecycleStates: string[]; + staleReasons: string[]; + }; events: RagAuditReplayEventRecord[]; generatedAt: string; } @@ -211,11 +224,118 @@ export class RagAuditReplayService { runId: resolvedRunId, requestId: requestedRequestId || undefined, runTrace, + preparationPlane: this.buildPreparationPlaneSummary(replayRows), events, generatedAt: new Date().toISOString() }; } + private buildPreparationPlaneSummary(rows: RagReplayRecord[]): + | RagAuditReplayChain["preparationPlane"] + | undefined { + if (rows.length === 0) { + return undefined; + } + const manifestFingerprints: string[] = []; + const activatedIndexVersionIds: string[] = []; + const preparedFamilies: string[] = []; + const degradedFamilies: string[] = []; + const skippedFamilies: string[] = []; + const lifecycleStates: string[] = []; + const staleReasons: string[] = []; + let activeManifestFingerprint: string | undefined; + let filteredAssetCount = 0; + let selectedAssetCount = 0; + let candidateAssetCount = 0; + + for (const row of rows) { + const payload = this.parsePayload(row.payload); + const manifestFingerprint = this.readOptionalString(payload.manifestFingerprint); + if (manifestFingerprint) { + manifestFingerprints.push(manifestFingerprint); + } + if (row.stage === "index_activation_completed") { + activeManifestFingerprint = manifestFingerprint ?? activeManifestFingerprint; + if (row.indexVersionId) { + activatedIndexVersionIds.push(row.indexVersionId); + } + } + if (row.stage === "manifest_prepared") { + const summary = this.readRecord(payload.summary); + const familyCounts = this.readRecord(summary?.familyCounts); + preparedFamilies.push(...Object.keys(familyCounts ?? {})); + degradedFamilies.push(...this.readStringArray(payload.degradedFamilies)); + skippedFamilies.push(...this.readStringArray(payload.skippedFamilies)); + } + if (row.stage === "retrieval_fused") { + const permissionFiltering = this.readRecord( + payload.permissionFiltering ?? payload.permission_filtering + ); + filteredAssetCount += this.readNumber( + permissionFiltering?.filtered_count ?? permissionFiltering?.filteredCount, + 0 + ); + const candidates = Array.isArray(payload.candidates) ? payload.candidates : []; + candidateAssetCount += candidates.filter((candidate) => { + if (!this.isRecord(candidate)) { + return false; + } + return Boolean( + this.readOptionalString(candidate.assetFamily) ?? + this.readOptionalString(candidate.asset_family) + ); + }).length; + staleReasons.push( + ...this.readStringArray(payload.degradeReasons ?? payload.degrade_reasons).filter( + (reason) => reason.includes("stale") + ) + ); + } + if (row.stage === "rerank_finalized") { + const selectedContexts = Array.isArray(payload.selectedContext) + ? payload.selectedContext + : Array.isArray(payload.selected_context) + ? payload.selected_context + : []; + selectedAssetCount += selectedContexts.filter((item) => this.isRecord(item)).length; + for (const item of selectedContexts) { + if (!this.isRecord(item)) { + continue; + } + const metadata = this.readRecord(item.metadata); + const lifecycleState = this.readOptionalString( + metadata?.lifecycleState ?? metadata?.lifecycle_state + ); + if (lifecycleState) { + lifecycleStates.push(lifecycleState); + } + } + } + } + + if ( + manifestFingerprints.length === 0 && + candidateAssetCount === 0 && + selectedAssetCount === 0 && + filteredAssetCount === 0 + ) { + return undefined; + } + return { + manifestFingerprints: this.unique(manifestFingerprints), + activeManifestFingerprint, + activatedIndexVersionIds: this.unique(activatedIndexVersionIds), + preparedFamilies: this.unique(preparedFamilies), + degradedFamilies: this.unique(degradedFamilies), + skippedFamilies: this.unique(skippedFamilies), + filteredAssetCount, + selectedAssetCount, + unusedAssetCount: Math.max(0, candidateAssetCount - selectedAssetCount), + lifecycleStates: this.unique(lifecycleStates), + staleReasons: this.unique(staleReasons) + }; + } + private buildEventNodes(rows: RagReplayRecord[]): EventNode[] { const nodes = new Map(); for (const row of rows) { @@ -429,6 +549,29 @@ export class RagAuditReplayService { } } + private readRecord(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; + } + + private isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); + } + + private readStringArray(value: unknown): string[] { + if (!Array.isArray(value)) { + return []; + } + return value + .map((item) => (typeof item === "string" ? item.trim() : "")) + .filter((item): item is string => Boolean(item)); + } + + private unique(values: readonly string[]): string[] { + return Array.from(new Set(values.filter((item) => item.trim().length > 0))).sort(); + } + private resolveEventId(replayKey: string, payloadEventId: unknown): string | undefined { if (typeof payloadEventId === "string" && payloadEventId.trim()) { return payloadEventId.trim(); diff --git a/apps/backend/src/modules/rag/index/rag-index-builder.service.ts b/apps/backend/src/modules/rag/index/rag-index-builder.service.ts index 60063b4..d5da084 100644 --- a/apps/backend/src/modules/rag/index/rag-index-builder.service.ts +++ b/apps/backend/src/modules/rag/index/rag-index-builder.service.ts @@ -15,6 +15,29 @@ export interface RagIndexBuildRequest { createdByRunId?: string; activatedByRunId?: string; chunks?: RagChunkBuildInput[]; + manifest?: RagIndexBuildManifestContext; +} + +export interface RagIndexBuildManifestContext { + fingerprint: string; + summary?: { + entryCount?: number; + preparedEntryCount?: number; + skippedEntryCount?: number; + degradedEntryCount?: number; + familyCounts?: Record; + reasonCodes?: string[]; + embeddingProfile?: unknown; + sourceSnapshotSummary?: unknown; + }; + entries?: Array<{ + id?: string; + family?: string; + status?: string; + sourceRef?: unknown; + sourceVersion?: string; + reasonCodes?: string[]; + }>; } export interface RagIndexBuildResult { @@ -31,6 +54,13 @@ export interface RagIndexBuildResult { denseDimensions?: number; vectorVersion?: string; indexVersion?: string; + manifestFingerprint?: string; + manifestSummary?: RagIndexBuildManifestContext["summary"]; + activation: { + activatedAt?: string; + replacedVersionIds: string[]; + replacedSourceVersions: string[]; + }; } @Injectable() @@ -78,11 +108,12 @@ export class RagIndexBuilderService { await this.repository.replaceEntriesForVersion(version.id, entries); await this.repository.markVersionReady(version.id); - const active = await this.repository.activateVersion({ + const activation = await this.repository.activateVersionWithEvidence({ datasourceId: input.datasourceId, indexVersionId: version.id, activatedByRunId: input.activatedByRunId }); + const active = activation.activated; const archived = await this.repository.listEntriesByVersion(version.id); return { @@ -98,7 +129,14 @@ export class RagIndexBuilderService { denseConfigSource: denseEmbedding.metadata?.configSource, denseDimensions: denseEmbedding.metadata?.dimensions, vectorVersion: denseEmbedding.metadata?.vectorVersion, - indexVersion: denseEmbedding.metadata?.indexVersion + indexVersion: denseEmbedding.metadata?.indexVersion, + manifestFingerprint: input.manifest?.fingerprint, + manifestSummary: input.manifest?.summary, + activation: { + activatedAt: active.activatedAt, + replacedVersionIds: activation.replacedVersions.map((item) => item.id), + replacedSourceVersions: activation.replacedVersions.map((item) => item.sourceVersion) + } }; } catch (error) { await this.safeDeprecate(version.id); diff --git a/apps/backend/src/modules/rag/index/rag-index.repository.ts b/apps/backend/src/modules/rag/index/rag-index.repository.ts index 4517ed4..338c4c0 100644 --- a/apps/backend/src/modules/rag/index/rag-index.repository.ts +++ b/apps/backend/src/modules/rag/index/rag-index.repository.ts @@ -53,6 +53,11 @@ export interface ActivateVersionInput { simulateFailure?: "after_deprecating_current_active"; } +export interface RagIndexActivationResult { + activated: RagIndexVersionRecord; + replacedVersions: RagIndexVersionRecord[]; +} + type PrismaClientLike = { ragChunk: { findMany: (args: Record) => Promise; @@ -321,6 +326,13 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { } async activateVersion(input: ActivateVersionInput): Promise { + const result = await this.activateVersionWithEvidence(input); + return result.activated; + } + + async activateVersionWithEvidence( + input: ActivateVersionInput + ): Promise { return this.withDatasourceActivationLock(input.datasourceId, async () => { if (this.isPrimaryPersistenceConfigured() && this.prisma) { return this.activateWithPrisma(input); @@ -446,9 +458,7 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { return entries; } - private async activateInMemory( - input: ActivateVersionInput - ): Promise { + private async activateInMemory(input: ActivateVersionInput): Promise { const target = this.indexVersions.get(input.indexVersionId); if (!target || target.datasourceId !== input.datasourceId) { throw new DomainError("RAG_INDEX_VERSION_NOT_FOUND", "待激活索引版本不存在。", 404, { @@ -471,8 +481,10 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { const snapshot = this.snapshotDatasourceVersions(input.datasourceId); try { const now = new Date().toISOString(); + const replacedVersions: RagIndexVersionRecord[] = []; for (const version of this.indexVersions.values()) { if (version.datasourceId === input.datasourceId && version.status === "active") { + replacedVersions.push({ ...version }); version.status = "deprecated"; version.updatedAt = now; this.indexVersions.set(version.id, { ...version }); @@ -491,7 +503,10 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { updatedAt: now }; this.indexVersions.set(next.id, next); - return { ...next }; + return { + activated: { ...next }, + replacedVersions + }; } catch (error) { this.restoreDatasourceVersions(input.datasourceId, snapshot); throw error; @@ -500,7 +515,7 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { private async activateWithPrisma( input: ActivateVersionInput - ): Promise { + ): Promise { const result = await this.tryPrismaWrite(async () => this.prisma?.$transaction(async (tx) => { const target = (await tx.ragIndexVersion.findUnique({ @@ -525,6 +540,13 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { } const now = new Date(); + const replacedRows = (await tx.ragIndexVersion.findMany({ + where: { + datasourceId: input.datasourceId, + status: "active" + }, + orderBy: [{ activatedAt: "desc" }, { updatedAt: "desc" }] + })) as RagIndexVersionRow[]; await tx.ragIndexVersion.updateMany({ where: { datasourceId: input.datasourceId, @@ -549,7 +571,10 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { updatedAt: now } })) as RagIndexVersionRow; - return this.fromRagIndexVersionRow(activated); + return { + activated: this.fromRagIndexVersionRow(activated), + replacedVersions: replacedRows.map((row) => this.fromRagIndexVersionRow(row)) + }; }) ); @@ -557,8 +582,17 @@ export class RagIndexRepository implements OnModuleInit, OnModuleDestroy { throw new Error("RAG 激活事务返回空结果"); } - this.indexVersions.set(result.id, result); - return { ...result }; + this.indexVersions.set(result.activated.id, result.activated); + for (const replaced of result.replacedVersions) { + this.indexVersions.set(replaced.id, { + ...replaced, + status: "deprecated" + }); + } + return { + activated: { ...result.activated }, + replacedVersions: result.replacedVersions.map((item) => ({ ...item })) + }; } private async updateVersionStatus( diff --git a/apps/backend/src/modules/rag/ingestion/chunk-profiles.ts b/apps/backend/src/modules/rag/ingestion/chunk-profiles.ts index 18dd303..92b65d3 100644 --- a/apps/backend/src/modules/rag/ingestion/chunk-profiles.ts +++ b/apps/backend/src/modules/rag/ingestion/chunk-profiles.ts @@ -2,7 +2,18 @@ export type RagChunkProfile = | "schema_table" | "schema_column" | "sql_example" - | "semantic_term"; + | "semantic_term" + | "semantic_asset_table_description" + | "semantic_asset_full_schema" + | "semantic_asset_column_batch" + | "semantic_asset_relationship" + | "semantic_asset_metric" + | "semantic_asset_calculated_field" + | "semantic_asset_business_term" + | "semantic_asset_prompt_instruction" + | "semantic_asset_prior_question_sql" + | "semantic_asset_dialect_rule" + | "semantic_asset_project_metadata"; export interface ChunkProfileConfig { maxCharacters: number; @@ -30,6 +41,61 @@ const CHUNK_PROFILE_CONFIGS: Record = { maxCharacters: 800, overlapCharacters: 120, hardChunkLimit: 5000 + }, + semantic_asset_table_description: { + maxCharacters: 900, + overlapCharacters: 120, + hardChunkLimit: 5000 + }, + semantic_asset_full_schema: { + maxCharacters: 1400, + overlapCharacters: 160, + hardChunkLimit: 5000 + }, + semantic_asset_column_batch: { + maxCharacters: 1000, + overlapCharacters: 120, + hardChunkLimit: 5000 + }, + semantic_asset_relationship: { + maxCharacters: 900, + overlapCharacters: 120, + hardChunkLimit: 5000 + }, + semantic_asset_metric: { + maxCharacters: 800, + overlapCharacters: 100, + hardChunkLimit: 5000 + }, + semantic_asset_calculated_field: { + maxCharacters: 800, + overlapCharacters: 100, + hardChunkLimit: 5000 + }, + semantic_asset_business_term: { + maxCharacters: 800, + overlapCharacters: 100, + hardChunkLimit: 5000 + }, + semantic_asset_prompt_instruction: { + maxCharacters: 700, + overlapCharacters: 100, + hardChunkLimit: 5000 + }, + semantic_asset_prior_question_sql: { + maxCharacters: 1100, + overlapCharacters: 140, + hardChunkLimit: 5000 + }, + semantic_asset_dialect_rule: { + maxCharacters: 700, + overlapCharacters: 100, + hardChunkLimit: 5000 + }, + semantic_asset_project_metadata: { + maxCharacters: 700, + overlapCharacters: 100, + hardChunkLimit: 5000 } }; diff --git a/apps/backend/src/modules/rag/ingestion/ingestion-source.adapter.ts b/apps/backend/src/modules/rag/ingestion/ingestion-source.adapter.ts index 75e62c1..0374f8d 100644 --- a/apps/backend/src/modules/rag/ingestion/ingestion-source.adapter.ts +++ b/apps/backend/src/modules/rag/ingestion/ingestion-source.adapter.ts @@ -1,7 +1,11 @@ import { Injectable } from "@nestjs/common"; import type { RagChunkProfile } from "./chunk-profiles"; -export type RagIngestionSourceType = "schema" | "sql_example" | "semantic_term"; +export type RagIngestionSourceType = + | "schema" + | "sql_example" + | "semantic_term" + | "semantic_asset"; interface IngestionBaseInput { datasourceId: string; @@ -39,10 +43,21 @@ export interface SemanticTermIngestionSourceInput extends IngestionBaseInput { columnNames?: string[]; } +export interface SemanticAssetIngestionSourceInput extends IngestionBaseInput { + sourceType: "semantic_asset"; + assetFamily: string; + title?: string; + content: string; + tableNames?: string[]; + columnNames?: string[]; + chunkProfile: RagChunkProfile; +} + export type IngestionSourceInput = | SchemaIngestionSourceInput | SqlExampleIngestionSourceInput - | SemanticTermIngestionSourceInput; + | SemanticTermIngestionSourceInput + | SemanticAssetIngestionSourceInput; export interface NormalizedRagDocumentInput { datasourceId: string; @@ -69,6 +84,8 @@ export class IngestionSourceAdapter { return this.normalizeSqlExampleSource(input); case "semantic_term": return this.normalizeSemanticTermSource(input); + case "semantic_asset": + return this.normalizeSemanticAssetSource(input); default: { const exhaustive: never = input; return exhaustive; @@ -166,6 +183,32 @@ export class IngestionSourceAdapter { chunkProfile: "semantic_term" }; } + + private normalizeSemanticAssetSource( + input: SemanticAssetIngestionSourceInput + ): NormalizedRagDocumentInput { + const sourceRef = input.sourceRef ?? input.assetFamily; + const assetFamily = input.assetFamily.trim(); + const title = input.title?.trim() || `Semantic Asset ${assetFamily}`; + + return { + datasourceId: input.datasourceId, + domain: "semantic_asset", + sourceType: input.sourceType, + sourceRef, + sourceVersion: input.sourceVersion, + contentChecksum: input.contentChecksum, + title, + content: input.content.trim(), + tableNames: uniqueNonEmpty(input.tableNames ?? []), + columnNames: uniqueNonEmpty(input.columnNames ?? []), + metadata: { + assetFamily, + ...(input.metadata ?? {}) + }, + chunkProfile: input.chunkProfile + }; + } } const uniqueNonEmpty = (values: string[]): string[] => { diff --git a/apps/backend/src/modules/rag/jobs/build-rag-index.job.ts b/apps/backend/src/modules/rag/jobs/build-rag-index.job.ts index edc7349..21d2302 100644 --- a/apps/backend/src/modules/rag/jobs/build-rag-index.job.ts +++ b/apps/backend/src/modules/rag/jobs/build-rag-index.job.ts @@ -1,6 +1,7 @@ import { Injectable } from "@nestjs/common"; import { RagIndexBuilderService, + type RagIndexBuildManifestContext, type RagIndexBuildResult } from "../index/rag-index-builder.service"; import { @@ -17,6 +18,7 @@ export interface BuildRagIndexJobInput { runId?: string; queuedAt?: string; chunks?: RagChunkBuildInput[]; + manifest?: RagIndexBuildManifestContext; } @Injectable() @@ -34,6 +36,38 @@ export class BuildRagIndexJob { const replayRunId = this.resolveReplayRunId(input.runId, input.datasourceId, startedAt); const baseChunks = await this.repository.listChunksForBuild(input.datasourceId); const chunks = this.mergeChunks(baseChunks, input.chunks ?? []); + if (input.manifest) { + await this.replayRepository.writeReplay({ + runId: replayRunId, + replayKey: `manifest:prepared:${input.manifest.fingerprint}`, + datasourceId: input.datasourceId, + stage: "manifest_prepared", + payload: { + manifestFingerprint: input.manifest.fingerprint, + sourceVersion: input.sourceVersion, + buildReason: input.buildReason ?? "scheduled_build", + summary: input.manifest.summary, + degradedFamilies: this.resolveFamiliesByStatus(input.manifest, "degraded"), + skippedFamilies: this.resolveFamiliesByStatus(input.manifest, "skipped"), + typedChunkInputCount: input.chunks?.length ?? 0 + } + }); + } + await this.replayRepository.writeReplay({ + runId: replayRunId, + replayKey: `index:build:started:${startedAt}`, + datasourceId: input.datasourceId, + stage: "index_build_started", + payload: { + sourceVersion: input.sourceVersion, + buildReason: input.buildReason ?? "scheduled_build", + manifestFingerprint: input.manifest?.fingerprint, + baseChunkCount: baseChunks.length, + incrementalChunkCount: input.chunks?.length ?? 0, + mergedChunkCount: chunks.length, + queueWaitMs + } + }); try { const result = await this.builder.buildAndActivate({ datasourceId: input.datasourceId, @@ -41,7 +75,8 @@ export class BuildRagIndexJob { buildReason: input.buildReason ?? "scheduled_build", createdByRunId: input.runId, activatedByRunId: input.runId, - chunks + chunks, + manifest: input.manifest }); const activationLatencyMs = Date.now() - startedAt; this.ingestionMetrics.recordBuild({ @@ -61,16 +96,51 @@ export class BuildRagIndexJob { payload: { sourceVersion: input.sourceVersion, buildReason: input.buildReason ?? "scheduled_build", + manifestFingerprint: input.manifest?.fingerprint, + manifestSummary: input.manifest?.summary, entryCount: result.entryCount, archivedChannels: result.archivedChannels, denseMode: result.denseMode, + activation: result.activation, activationLatencyMs, queueWaitMs } }); + await this.replayRepository.writeReplay({ + runId: replayRunId, + replayKey: `index:activation:completed:${result.indexVersionId}`, + datasourceId: input.datasourceId, + stage: "index_activation_completed", + indexVersionId: result.indexVersionId, + payload: { + sourceVersion: input.sourceVersion, + manifestFingerprint: input.manifest?.fingerprint, + activatedAt: result.activation.activatedAt, + replacedVersionIds: result.activation.replacedVersionIds, + replacedSourceVersions: result.activation.replacedSourceVersions + } + }); + + if (result.activation.replacedVersionIds.length > 0) { + await this.replayRepository.writeReplay({ + runId: replayRunId, + replayKey: `index:activation:replaced_previous:${result.indexVersionId}`, + datasourceId: input.datasourceId, + stage: "activation_replaced_previous", + indexVersionId: result.indexVersionId, + payload: { + sourceVersion: input.sourceVersion, + manifestFingerprint: input.manifest?.fingerprint, + replacedVersionIds: result.activation.replacedVersionIds, + replacedSourceVersions: result.activation.replacedSourceVersions + } + }); + } + const indexedChunks = await this.repository.listEntriesByVersion(result.indexVersionId); for (const entry of indexedChunks) { + const lineage = this.resolveChunkLineage(entry.metadata); await this.replayRepository.writeReplay({ runId: replayRunId, replayKey: `chunk:indexed:${entry.chunkId}`, @@ -80,7 +150,8 @@ export class BuildRagIndexJob { chunkId: entry.chunkId, payload: { domain: entry.domain, - lexicalContentLength: entry.lexicalContent.length + lexicalContentLength: entry.lexicalContent.length, + ...lineage } }); } @@ -101,6 +172,8 @@ export class BuildRagIndexJob { payload: { sourceVersion: input.sourceVersion, buildReason: input.buildReason ?? "scheduled_build", + manifestFingerprint: input.manifest?.fingerprint, + manifestSummary: input.manifest?.summary, error: error instanceof Error ? error.message : String(error), queueWaitMs } @@ -159,4 +232,60 @@ export class BuildRagIndexJob { } return [...merged.values()]; } + + private resolveFamiliesByStatus( + manifest: RagIndexBuildManifestContext, + status: "degraded" | "skipped" + ): string[] { + const families = (manifest.entries ?? []) + .filter((entry) => entry.status === status && entry.family) + .map((entry) => entry.family as string); + return Array.from(new Set(families)).sort(); + } + + private resolveChunkLineage(metadata: string | undefined): Record { + const parsed = this.safeParseJson(metadata); + const sourceMetadata = this.isRecord(parsed?.sourceMetadata) + ? parsed.sourceMetadata + : this.isRecord(parsed) + ? parsed + : undefined; + if (!sourceMetadata) { + return {}; + } + return { + assetFamily: this.readString(sourceMetadata.assetFamily), + manifestFingerprint: this.readString(sourceMetadata.manifestFingerprint), + manifestEntryId: this.readString(sourceMetadata.manifestEntryId), + sourceRef: sourceMetadata.semanticAssetSourceRef, + sourceVersion: this.readString(sourceMetadata.sourceVersion), + visibilityScope: this.readString(sourceMetadata.visibilityScope), + preparationStatus: this.readString(sourceMetadata.preparationStatus), + reasonCodes: Array.isArray(sourceMetadata.reasonCodes) + ? sourceMetadata.reasonCodes.filter((item): item is string => typeof item === "string") + : undefined, + chunkProfile: + this.readString(sourceMetadata.chunkProfile) ?? this.readString(parsed?.chunkProfile) + }; + } + + private safeParseJson(value: string | undefined): Record | undefined { + if (!value) { + return undefined; + } + try { + const parsed = JSON.parse(value) as unknown; + return this.isRecord(parsed) ? parsed : undefined; + } catch { + return undefined; + } + } + + private isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); + } + + private readString(value: unknown): string | undefined { + return typeof value === "string" && value.trim().length > 0 ? value.trim() : undefined; + } } diff --git a/apps/backend/src/modules/rag/quality/rag-quality.service.ts b/apps/backend/src/modules/rag/quality/rag-quality.service.ts index bf3b05a..d37f317 100644 --- a/apps/backend/src/modules/rag/quality/rag-quality.service.ts +++ b/apps/backend/src/modules/rag/quality/rag-quality.service.ts @@ -23,6 +23,22 @@ export interface RagQualityEvaluationInput { recordedAt?: string; } +export interface RagPreparationPlaneMetricsInput { + runId: string; + datasourceId: string; + manifestFingerprint?: string; + activeIndexVersionId?: string; + familyCounts?: Record; + preparedEntryCount?: number; + degradedEntryCount?: number; + skippedEntryCount?: number; + permissionFilteredAssetCount?: number; + selectedAssetCount?: number; + staleReasons?: string[]; + lifecycleStatus?: string; + recordedAt?: string; +} + export interface RagPriorSqlLaneMetricsInput { totalCount: number; hitCount: number; @@ -52,6 +68,37 @@ interface RagPriorSqlLaneMetricsRecord { fallbackToGenerationCount: number; } +interface RagPreparationPlaneMetricsRecord { + runId: string; + datasourceId: string; + manifestFingerprint?: string; + activeIndexVersionId?: string; + familyCounts: Record; + preparedEntryCount: number; + degradedEntryCount: number; + skippedEntryCount: number; + permissionFilteredAssetCount: number; + selectedAssetCount: number; + staleReasons: string[]; + lifecycleStatus?: string; + recordedAt: string; +} + +export interface RagPreparationPlaneGateReport { + sampleSize: number; + latestManifestFingerprint?: string; + latestActiveIndexVersionId?: string; + familyCounts: Record; + preparedEntryCount: number; + degradedEntryCount: number; + skippedEntryCount: number; + permissionFilteredAssetCount: number; + selectedAssetCount: number; + staleReasons: string[]; + lifecycleStatuses: string[]; + completenessReady: boolean; +} + export interface RagDatasourceOrchestrationSample { datasourceId: string; workspaceId: string; @@ -135,6 +182,7 @@ export interface RagQualityGateReport { datasourceOrchestration: RagDatasourceOrchestrationReport; cacheBudget: RagCacheBudgetReport; priorSqlLane: RagPriorSqlLaneGateReport; + preparationPlane: RagPreparationPlaneGateReport; latest?: { runId: string; datasourceId: string; @@ -331,6 +379,7 @@ export class RagQualityService { private readonly records: RagQualityEvaluationRecord[] = []; private readonly datasourceRecords: RagDatasourceOrchestrationRecord[] = []; private readonly cacheBudgetRecords: RagCacheBudgetRecord[] = []; + private readonly preparationPlaneRecords: RagPreparationPlaneMetricsRecord[] = []; constructor(private readonly replayRepository: RagReplayRepository) {} @@ -353,6 +402,7 @@ export class RagQualityService { this.records.length = 0; this.datasourceRecords.length = 0; this.cacheBudgetRecords.length = 0; + this.preparationPlaneRecords.length = 0; } recordDatasourceOrchestration(input: RagDatasourceOrchestrationSample): void { @@ -379,6 +429,36 @@ export class RagQualityService { }); } + recordPreparationPlane(input: RagPreparationPlaneMetricsInput): void { + const runId = input.runId.trim(); + const datasourceId = input.datasourceId.trim(); + if (!runId || !datasourceId) { + return; + } + const familyCounts = Object.fromEntries( + Object.entries(input.familyCounts ?? {}) + .filter(([family, count]) => family.trim().length > 0 && Number.isFinite(count)) + .map(([family, count]) => [family, this.normalizeCount(count)]) + ); + this.preparationPlaneRecords.push({ + runId, + datasourceId, + manifestFingerprint: input.manifestFingerprint?.trim() || undefined, + activeIndexVersionId: input.activeIndexVersionId?.trim() || undefined, + familyCounts, + preparedEntryCount: this.normalizeCount(input.preparedEntryCount), + degradedEntryCount: this.normalizeCount(input.degradedEntryCount), + skippedEntryCount: this.normalizeCount(input.skippedEntryCount), + permissionFilteredAssetCount: this.normalizeCount( + input.permissionFilteredAssetCount + ), + selectedAssetCount: this.normalizeCount(input.selectedAssetCount), + staleReasons: this.unique(input.staleReasons ?? []), + lifecycleStatus: input.lifecycleStatus?.trim() || undefined, + recordedAt: this.normalizeIsoTimestamp(input.recordedAt) + }); + } + snapshot(): RagQualityGateReport { const latest = this.records.at(-1); const thresholds = DEFAULT_THRESHOLDS; @@ -419,6 +499,7 @@ export class RagQualityService { datasourceOrchestration: this.snapshotDatasourceOrchestration(), cacheBudget: this.snapshotCacheBudget(), priorSqlLane: this.snapshotPriorSqlLane(), + preparationPlane: this.snapshotPreparationPlane(), latest: latest ? { runId: latest.runId, @@ -767,6 +848,51 @@ export class RagQualityService { }; } + private snapshotPreparationPlane(): RagPreparationPlaneGateReport { + const latest = this.preparationPlaneRecords.at(-1); + const familyCounts = new Map(); + let preparedEntryCount = 0; + let degradedEntryCount = 0; + let skippedEntryCount = 0; + let permissionFilteredAssetCount = 0; + let selectedAssetCount = 0; + const staleReasons: string[] = []; + const lifecycleStatuses: string[] = []; + + for (const record of this.preparationPlaneRecords) { + for (const [family, count] of Object.entries(record.familyCounts)) { + familyCounts.set(family, (familyCounts.get(family) ?? 0) + count); + } + preparedEntryCount += record.preparedEntryCount; + degradedEntryCount += record.degradedEntryCount; + skippedEntryCount += record.skippedEntryCount; + permissionFilteredAssetCount += record.permissionFilteredAssetCount; + selectedAssetCount += record.selectedAssetCount; + staleReasons.push(...record.staleReasons); + if (record.lifecycleStatus) { + lifecycleStatuses.push(record.lifecycleStatus); + } + } + + return { + sampleSize: this.preparationPlaneRecords.length, + latestManifestFingerprint: latest?.manifestFingerprint, + latestActiveIndexVersionId: latest?.activeIndexVersionId, + familyCounts: Object.fromEntries([...familyCounts.entries()].sort()), + preparedEntryCount, + degradedEntryCount, + skippedEntryCount, + permissionFilteredAssetCount, + selectedAssetCount, + staleReasons: this.unique(staleReasons), + lifecycleStatuses: this.unique(lifecycleStatuses), + completenessReady: + this.preparationPlaneRecords.length > 0 && + preparedEntryCount > 0 && + this.unique(staleReasons).length === 0 + }; + } + private percentile(values: number[], quantile: number): number { if (values.length === 0) { return 0; @@ -780,6 +906,10 @@ export class RagQualityService { return Number.isFinite(value) ? Number(value.toFixed(3)) : 0; } + private unique(values: readonly string[]): string[] { + return Array.from(new Set(values.filter((item) => item.trim().length > 0))).sort(); + } + private snapshotR6( latest: RagQualityEvaluationRecord | undefined ): RagR6GateReport { diff --git a/apps/backend/src/modules/rag/rag.module.ts b/apps/backend/src/modules/rag/rag.module.ts index ad5d8d4..d9dc59c 100644 --- a/apps/backend/src/modules/rag/rag.module.ts +++ b/apps/backend/src/modules/rag/rag.module.ts @@ -1,77 +1,8 @@ import { Module } from "@nestjs/common"; -import { AppConfigModule } from "../config/config.module"; -import { PlatformDataPersistenceModule } from "../platform/data/persistence.module"; -import { GraphAccelerationAdapter } from "../knowledge/graph/adapter/graph-acceleration.adapter"; -import { GraphAccelerationCircuitBreaker } from "../knowledge/graph/adapter/graph-acceleration-circuit-breaker"; -import { GraphService } from "../knowledge/graph/graph.service"; -import { LlmModule } from "../llm/llm.module"; -import { ObservabilityModule } from "../observability/observability.module"; -import { SkillRegistryModule } from "../skill-registry/skill-registry.module"; -import { RagAuditReplayService } from "./audit/rag-audit-replay.service"; -import { RagEventConsumerService } from "./events/rag-event-consumer.service"; -import { RagIndexBuilderService } from "./index/rag-index-builder.service"; -import { RagIndexRepository } from "./index/rag-index.repository"; -import { BuildRagIndexJob } from "./jobs/build-rag-index.job"; -import { RagReplayRepository } from "./observability/rag-replay.repository"; -import { RagDatasourceOrchestratorService } from "./orchestration/rag-datasource-orchestrator.service"; -import { RagDatasourceQuotaPolicy } from "./orchestration/rag-datasource-quota.policy"; -import { RagBudgetPolicy } from "./perf/rag-budget-policy"; -import { RagCacheKeyFactory } from "./perf/rag-cache-key.factory"; -import { RagQueryCacheService } from "./perf/rag-query-cache.service"; -import { RagQualityController } from "./quality/rag-quality.controller"; -import { RagQualityService } from "./quality/rag-quality.service"; -import { RagRetrievalService } from "./retrieval/rag-retrieval.service"; -import { ModelRerankerAdapter } from "./rerank/model-reranker.adapter"; -import { RagRerankService } from "./rerank/rag-rerank.service"; +import { RagModule as KnowledgeRagModule } from "../knowledge/rag/rag.module"; @Module({ - imports: [ - AppConfigModule, - PlatformDataPersistenceModule, - LlmModule, - ObservabilityModule, - SkillRegistryModule - ], - controllers: [RagQualityController], - providers: [ - RagIndexRepository, - RagIndexBuilderService, - BuildRagIndexJob, - RagDatasourceQuotaPolicy, - RagDatasourceOrchestratorService, - GraphAccelerationAdapter, - GraphAccelerationCircuitBreaker, - GraphService, - RagBudgetPolicy, - RagCacheKeyFactory, - RagQueryCacheService, - RagEventConsumerService, - RagAuditReplayService, - RagReplayRepository, - RagQualityService, - RagRetrievalService, - ModelRerankerAdapter, - RagRerankService - ], - exports: [ - RagIndexRepository, - RagIndexBuilderService, - BuildRagIndexJob, - RagDatasourceQuotaPolicy, - RagDatasourceOrchestratorService, - GraphAccelerationAdapter, - GraphAccelerationCircuitBreaker, - GraphService, - RagBudgetPolicy, - RagCacheKeyFactory, - RagQueryCacheService, - RagEventConsumerService, - RagAuditReplayService, - RagReplayRepository, - RagQualityService, - RagRetrievalService, - ModelRerankerAdapter, - RagRerankService - ] + imports: [KnowledgeRagModule], + exports: [KnowledgeRagModule] }) export class RagModule {} diff --git a/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts b/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts index 8783c60..12427c7 100644 --- a/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts +++ b/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts @@ -263,7 +263,13 @@ export class RagRerankService { const degradeReasons = this.unique([...bundle.degrade_reasons, ...rerankDegradeReasons]); const selectedContext = reranked .slice(0, selectedContextLimit) - .map((item) => item.chunk); + .map((item) => ({ + ...item.chunk, + metadata: { + ...item.chunk.metadata, + lifecycleState: "selected" as const + } + })); const responseBundle: RagRetrievalBundle = { ...bundle, diff --git a/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts b/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts index 81d0f1b..6b01dbd 100644 --- a/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts +++ b/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts @@ -28,6 +28,17 @@ export interface RagRetrievalChunkMetadata { indexVersionId: string; chunkId: string; domain: string; + assetFamily?: string; + manifestFingerprint?: string; + manifestEntryId?: string; + sourceRef?: unknown; + sourceVersion?: string; + policyVersion?: string; + modelingRevision?: number; + visibilityScope?: string; + preparationStatus?: string; + reasonCodes?: string[]; + lifecycleState?: "retrieved" | "selected" | "pruned" | "filtered" | "unused"; chunkProfile?: string; startOffset?: number; endOffset?: number; diff --git a/apps/backend/src/modules/system/health.controller.ts b/apps/backend/src/modules/system/health.controller.ts index 368c21f..fbb6707 100644 --- a/apps/backend/src/modules/system/health.controller.ts +++ b/apps/backend/src/modules/system/health.controller.ts @@ -11,6 +11,7 @@ import { DatasourceRegistryService } from "../governance/datasource/datasource-r import { GateMetricsService } from "../observability/gate-metrics.service"; import { RagIngestionMetricsService } from "../rag/observability/rag-ingestion-metrics.service"; import { RagQualityService } from "../rag/quality/rag-quality.service"; +import { SemanticAssetReadinessService } from "../knowledge/rag/preparation/semantic-asset-readiness.service"; import { SemanticSpineShadowGateService } from "../observability/semantic-spine-shadow-gate.service"; import { RagTaskConfigService } from "../llm/rag-task-config.service"; @@ -26,6 +27,7 @@ export class HealthController { private readonly gateMetrics: GateMetricsService, private readonly ragIngestionMetrics: RagIngestionMetricsService, private readonly ragQuality: RagQualityService, + private readonly semanticAssetReadiness: SemanticAssetReadinessService, private readonly semanticSpineShadow: SemanticSpineShadowGateService, private readonly ragTaskConfigService: RagTaskConfigService ) {} @@ -39,6 +41,7 @@ export class HealthController { }); const postgresEnabled = Boolean(this.config.databaseUrl); const ragQualityGate = this.ragQuality.snapshot(); + const semanticAssetReadiness = await this.semanticAssetReadiness.snapshot(); const semanticSpineShadowGate = this.semanticSpineShadow.snapshot(); const ragConfigView = await this.ragTaskConfigService.listSettingsView({ id: "system-health", @@ -116,7 +119,8 @@ export class HealthController { configSource: rerankConfig.configSource, healthStatus: rerankConfig.healthStatus } - : null + : null, + semanticAssetReadiness }, semanticSpineShadow: { gate: semanticSpineShadowGate diff --git a/apps/backend/test/e2e/chat-stream-api.spec.ts b/apps/backend/test/e2e/chat-stream-api.spec.ts index df0d61a..2054ff7 100644 --- a/apps/backend/test/e2e/chat-stream-api.spec.ts +++ b/apps/backend/test/e2e/chat-stream-api.spec.ts @@ -1,16 +1,12 @@ import { resolve } from "node:path"; import { INestApplication } from "@nestjs/common"; +import { parseSseEventsForTest } from "@text2sql/chat-stream-protocol"; import type { ChatStreamEvent } from "@text2sql/shared-types"; import { Test } from "@nestjs/testing"; import request from "supertest"; import { AppModule } from "../../src/app.module"; -interface ParsedSseEvent { - eventType: string; - event: ChatStreamEvent; -} - -const INTERMEDIATE_EVENT_TYPES = new Set([ +const INTERMEDIATE_EVENT_TYPES = new Set([ "state", "text-delta", "tool-call", @@ -18,38 +14,6 @@ const INTERMEDIATE_EVENT_TYPES = new Set([ "tool-error" ]); -function parseSseEvents(payload: string): ParsedSseEvent[] { - const blocks = payload - .split(/\n\n+/) - .map((block) => block.trim()) - .filter(Boolean); - - const events: ParsedSseEvent[] = []; - for (const block of blocks) { - const lines = block - .split("\n") - .map((line) => line.trim()) - .filter(Boolean); - const eventLine = lines.find((line) => line.startsWith("event:")); - const dataLines = lines.filter((line) => line.startsWith("data:")); - if (!eventLine || dataLines.length === 0) { - continue; - } - - const eventType = eventLine.replace(/^event:\s*/, "").trim(); - const dataText = dataLines - .map((line) => line.replace(/^data:\s*/, "")) - .join("\n"); - const event = JSON.parse(dataText) as ChatStreamEvent; - events.push({ - eventType, - event - }); - } - - return events; -} - describe("chat stream api (e2e)", () => { let app: INestApplication; @@ -85,7 +49,7 @@ describe("chat stream api (e2e)", () => { expect(streamRes.status).toBe(200); expect(streamRes.headers["content-type"]).toContain("text/event-stream"); - const parsedEvents = parseSseEvents(streamRes.text); + const parsedEvents = parseSseEventsForTest(streamRes.text); expect(parsedEvents.length).toBeGreaterThanOrEqual(2); const eventTypes = parsedEvents.map(({ eventType }) => eventType); @@ -360,7 +324,7 @@ describe("chat stream api (e2e)", () => { .send({ message: "DELETE orders where id = 1" }); expect(streamRes.status).toBe(200); - const parsedEvents = parseSseEvents(streamRes.text); + const parsedEvents = parseSseEventsForTest(streamRes.text); const finishEvent = [...parsedEvents] .reverse() .find(({ eventType }) => eventType === "finish")?.event; @@ -420,7 +384,7 @@ describe("chat stream api (e2e)", () => { expect(streamRes.status).toBe(200); expect(streamRes.headers["content-type"]).toContain("text/event-stream"); - const parsedEvents = parseSseEvents(streamRes.text); + const parsedEvents = parseSseEventsForTest(streamRes.text); expect(parsedEvents.length).toBeGreaterThanOrEqual(2); expect(parsedEvents.some(({ eventType }) => eventType === "start")).toBe(true); expect(parsedEvents.some(({ eventType }) => eventType === "finish")).toBe(true); diff --git a/apps/backend/test/fixtures/text2sql-v2-closeout-flow-matrix.json b/apps/backend/test/fixtures/text2sql-v2-closeout-flow-matrix.json index 19481d0..1b66d61 100644 --- a/apps/backend/test/fixtures/text2sql-v2-closeout-flow-matrix.json +++ b/apps/backend/test/fixtures/text2sql-v2-closeout-flow-matrix.json @@ -1356,5 +1356,79 @@ "behaviorTestStatus": "covered", "coverageOwnerStatus": "covered" } + ], + "preparationPlaneRows": [ + { + "id": "manifest-completeness", + "implementationOwners": [ + "apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts", + "apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts" + ], + "evidenceOwners": ["dependencies.ragConfig.semanticAssetReadiness"], + "expectedTestFiles": ["apps/backend/test/integration/settings-rag-readiness.spec.ts"], + "behaviorTestStatus": "covered", + "coverageOwnerStatus": "covered", + "requiredEvidenceSignals": ["activeManifestFingerprint", "familyCounts"] + }, + { + "id": "typed-family-generation", + "implementationOwners": [ + "apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts" + ], + "evidenceOwners": ["manifestSummary.familyCounts"], + "expectedTestFiles": [ + "apps/backend/test/unit/semantic-asset-preparer.service.spec.ts" + ], + "behaviorTestStatus": "covered", + "coverageOwnerStatus": "covered", + "requiredEvidenceSignals": ["assetFamily", "manifestFingerprint"] + }, + { + "id": "activation-stale-behavior", + "implementationOwners": [ + "apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts", + "apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts" + ], + "evidenceOwners": ["semanticAssetReadiness.staleReasons"], + "expectedTestFiles": [ + "apps/backend/test/integration/text2sql-semantic-asset-reindex.spec.ts" + ], + "behaviorTestStatus": "covered", + "coverageOwnerStatus": "covered", + "requiredEvidenceSignals": ["staleReasons", "rebuildable"] + }, + { + "id": "permission-filtering-before-ranking", + "implementationOwners": [ + "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts" + ], + "evidenceOwners": ["retrieval:fused.permissionFiltering"], + "expectedTestFiles": ["apps/backend/test/integration/rag-retrieval.service.spec.ts"], + "behaviorTestStatus": "covered", + "coverageOwnerStatus": "covered", + "requiredEvidenceSignals": ["permission_filtered_before_ranking"] + }, + { + "id": "two-pass-schema-recall", + "implementationOwners": [ + "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts" + ], + "evidenceOwners": ["retrieval:fused.twoPassSchemaRecall"], + "expectedTestFiles": ["apps/backend/test/integration/rag-retrieval.service.spec.ts"], + "behaviorTestStatus": "covered", + "coverageOwnerStatus": "covered", + "requiredEvidenceSignals": ["table_description", "full_schema"] + }, + { + "id": "prior-sql-diagnostics", + "implementationOwners": [ + "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts" + ], + "evidenceOwners": ["retrieval:fused.priorSqlLane"], + "expectedTestFiles": ["apps/backend/test/integration/rag-retrieval.service.spec.ts"], + "behaviorTestStatus": "covered", + "coverageOwnerStatus": "covered", + "requiredEvidenceSignals": ["stale", "ambiguous", "filtered"] + } ] } diff --git a/apps/backend/test/integration/delivery-contract.spec.ts b/apps/backend/test/integration/delivery-contract.spec.ts index 17341bb..5f7240b 100644 --- a/apps/backend/test/integration/delivery-contract.spec.ts +++ b/apps/backend/test/integration/delivery-contract.spec.ts @@ -1,46 +1,11 @@ import { INestApplication } from "@nestjs/common"; -import type { ChatStreamEvent } from "@text2sql/shared-types"; +import { parseSseEventsForTest } from "@text2sql/chat-stream-protocol"; import { Test } from "@nestjs/testing"; import request from "supertest"; import { AppModule } from "../../src/app.module"; import { DeliveryContractMapper } from "../../src/modules/conversation/delivery/delivery-contract.mapper"; import { createSeededSqliteFixture } from "../support/sqlite-fixture"; -interface ParsedSseEvent { - eventType: string; - event: ChatStreamEvent; -} - -function parseSseEvents(payload: string): ParsedSseEvent[] { - const blocks = payload - .split(/\n\n+/) - .map((block) => block.trim()) - .filter(Boolean); - - const events: ParsedSseEvent[] = []; - for (const block of blocks) { - const lines = block - .split("\n") - .map((line) => line.trim()) - .filter(Boolean); - const eventLine = lines.find((line) => line.startsWith("event:")); - const dataLines = lines.filter((line) => line.startsWith("data:")); - if (!eventLine || dataLines.length === 0) { - continue; - } - const eventType = eventLine.replace(/^event:\s*/, "").trim(); - const dataText = dataLines - .map((line) => line.replace(/^data:\s*/, "")) - .join("\n"); - const event = JSON.parse(dataText) as ChatStreamEvent; - events.push({ - eventType, - event - }); - } - return events; -} - describe("delivery contract integration", () => { let app: INestApplication; let cleanupFixture: (() => Promise) | undefined; @@ -109,7 +74,7 @@ describe("delivery contract integration", () => { .post(`/api/v1/sessions/${sessionId}/messages/stream`) .send({ message: "统计订单状态分布" }); expect(streamRes.status).toBe(200); - const events = parseSseEvents(streamRes.text); + const events = parseSseEventsForTest(streamRes.text); const finish = [...events].reverse().find((item) => item.eventType === "finish"); expect(finish).toBeDefined(); @@ -190,7 +155,7 @@ describe("delivery contract integration", () => { .post(`/api/v1/sessions/${sessionId}/messages/stream`) .send({ message: "统计订单状态分布" }); expect(streamRes.status).toBe(200); - const events = parseSseEvents(streamRes.text); + const events = parseSseEventsForTest(streamRes.text); const finish = [...events].reverse().find((item) => item.eventType === "finish"); expect(finish).toBeDefined(); const finishData = (finish?.event.data ?? {}) as { diff --git a/apps/backend/test/integration/rag-audit-replay.spec.ts b/apps/backend/test/integration/rag-audit-replay.spec.ts index 27f9885..32a1d29 100644 --- a/apps/backend/test/integration/rag-audit-replay.spec.ts +++ b/apps/backend/test/integration/rag-audit-replay.spec.ts @@ -181,6 +181,93 @@ describe("rag audit replay integration", () => { }); }); + it("summarizes preparation-plane manifest, filtering, and selected lifecycle replay", async () => { + const runId = "run-rag-audit-preparation-plane"; + await replayRepository.writeReplay({ + runId, + replayKey: "manifest:prepared:semantic-assets-audit-v1", + datasourceId: "ds-rag-audit-preparation", + stage: "manifest_prepared", + payload: { + manifestFingerprint: "semantic-assets-audit-v1", + summary: { + familyCounts: { + table_description: 1, + full_schema: 1 + } + }, + degradedFamilies: ["relationship_binding"], + skippedFamilies: ["prompt_instruction"] + }, + createdAt: "2026-04-18T03:10:00.000Z" + }); + await replayRepository.writeReplay({ + runId, + replayKey: "index:activation:completed:idx-rag-audit-preparation", + datasourceId: "ds-rag-audit-preparation", + stage: "index_activation_completed", + indexVersionId: "idx-rag-audit-preparation", + payload: { + manifestFingerprint: "semantic-assets-audit-v1" + }, + createdAt: "2026-04-18T03:10:01.000Z" + }); + await replayRepository.writeReplay({ + runId, + replayKey: "retrieval:fused", + datasourceId: "ds-rag-audit-preparation", + stage: "retrieval_fused", + payload: { + permissionFiltering: { + filteredCount: 1 + }, + candidates: [ + { + chunkId: "chunk-table-description", + assetFamily: "table_description" + }, + { + chunkId: "chunk-full-schema", + assetFamily: "full_schema" + } + ] + }, + createdAt: "2026-04-18T03:10:02.000Z" + }); + await replayRepository.writeReplay({ + runId, + replayKey: "rerank:final", + datasourceId: "ds-rag-audit-preparation", + stage: "rerank_finalized", + payload: { + selectedContext: [ + { + chunkId: "chunk-table-description", + metadata: { + lifecycleState: "selected" + } + } + ] + }, + createdAt: "2026-04-18T03:10:03.000Z" + }); + + const chain = await auditReplayService.queryChain({ runId }); + + expect(chain.preparationPlane).toMatchObject({ + manifestFingerprints: ["semantic-assets-audit-v1"], + activeManifestFingerprint: "semantic-assets-audit-v1", + activatedIndexVersionIds: ["idx-rag-audit-preparation"], + preparedFamilies: ["full_schema", "table_description"], + degradedFamilies: ["relationship_binding"], + skippedFamilies: ["prompt_instruction"], + filteredAssetCount: 1, + selectedAssetCount: 1, + unusedAssetCount: 1, + lifecycleStates: ["selected"] + }); + }); + it("returns dlq status and failure reason for unrecoverable event", async () => { const runId = "run-rag-audit-chain-dlq"; await chatRepository.persistRun(createRun(runId)); diff --git a/apps/backend/test/integration/rag-document-factory.spec.ts b/apps/backend/test/integration/rag-document-factory.spec.ts index 1fdd0e7..feb0caa 100644 --- a/apps/backend/test/integration/rag-document-factory.spec.ts +++ b/apps/backend/test/integration/rag-document-factory.spec.ts @@ -98,4 +98,43 @@ describe("RagDocumentFactory integration", () => { second.chunks.map((item) => item.chunkOrder) ); }); + + it("preserves semantic asset family provenance in normalized metadata", () => { + const result = factory.create({ + sourceType: "semantic_asset", + datasourceId: "ds-main", + sourceVersion: "semantic-assets-v1", + contentChecksum: "asset-checksum", + sourceRef: "datasource_schema:orders.description", + assetFamily: "table_description", + title: "Orders description", + content: "Table: orders\nDescription: Paid order facts", + tableNames: ["orders"], + metadata: { + manifestFingerprint: "semantic-assets-abc", + sourceHash: "source-hash", + visibilityScope: "datasource", + preparationStatus: "prepared", + reasonCodes: ["prepared"] + }, + chunkProfile: "semantic_asset_table_description" + }); + + expect(result.document.domain).toBe("semantic_asset"); + expect(result.chunks[0]?.chunkProfile).toBe("semantic_asset_table_description"); + expect(JSON.parse(result.document.metadata ?? "{}")).toEqual( + expect.objectContaining({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-abc", + chunkProfile: "semantic_asset_table_description" + }) + ); + expect(JSON.parse(result.chunks[0]?.metadata ?? "{}")).toEqual( + expect.objectContaining({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-abc", + visibilityScope: "datasource" + }) + ); + }); }); diff --git a/apps/backend/test/integration/rag-foundation-replay.spec.ts b/apps/backend/test/integration/rag-foundation-replay.spec.ts index 68cdb8d..ffe06a6 100644 --- a/apps/backend/test/integration/rag-foundation-replay.spec.ts +++ b/apps/backend/test/integration/rag-foundation-replay.spec.ts @@ -170,13 +170,41 @@ describe("rag foundation replay repository", () => { const result = await buildJob.run({ datasourceId: "ds-rag-replay-job", sourceVersion: "source-replay-job-v1", - runId: "run-rag-replay-job" + runId: "run-rag-replay-job", + manifest: { + fingerprint: "semantic-assets-replay-v1", + summary: { + entryCount: 1, + preparedEntryCount: 1, + familyCounts: { + table_description: 1 + }, + reasonCodes: ["prepared"] + }, + entries: [ + { + id: "manifest-entry-replay-orders", + family: "table_description", + status: "prepared", + sourceRef: { type: "datasource_schema", ref: "orders.description" }, + sourceVersion: "schema-v1", + reasonCodes: ["prepared"] + } + ] + } }); const replayEvents = await replayRepository.listByRunId("run-rag-replay-job"); expect(result.status).toBe("active"); + expect(replayEvents.some((item) => item.stage === "manifest_prepared")).toBe(true); + expect(replayEvents.some((item) => item.stage === "index_build_started")).toBe(true); expect(replayEvents.some((item) => item.stage === "index_build_completed")).toBe(true); + expect(replayEvents.some((item) => item.stage === "index_activation_completed")).toBe(true); expect(replayEvents.filter((item) => item.stage === "chunk_indexed").length).toBe(2); + const completedPayload = JSON.parse( + replayEvents.find((item) => item.stage === "index_build_completed")?.payload ?? "{}" + ); + expect(completedPayload.manifestFingerprint).toBe("semantic-assets-replay-v1"); expect( replayEvents.some((item) => item.replayKey === "chunk:indexed:chunk-replay-job-1") ).toBe(true); diff --git a/apps/backend/test/integration/rag-index-builder.spec.ts b/apps/backend/test/integration/rag-index-builder.spec.ts index 96c07e8..fe44e08 100644 --- a/apps/backend/test/integration/rag-index-builder.spec.ts +++ b/apps/backend/test/integration/rag-index-builder.spec.ts @@ -53,6 +53,7 @@ describe("rag index builder integration", () => { expect(result.status).toBe("active"); expect(result.archivedChannels).toEqual(["lexical", "dense"]); + expect(result.activation.replacedVersionIds).toEqual([]); expect(["mock_provider", "external_provider", "dense_unavailable"]).toContain( result.denseMode ); @@ -71,4 +72,64 @@ describe("rag index builder integration", () => { await moduleRef.close(); }); + + it("returns activation evidence when replacing a previous active version", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + + const repository = moduleRef.get(RagIndexRepository); + const builder = moduleRef.get(RagIndexBuilderService); + const datasourceId = "ds-rag-index-builder-activation-evidence"; + + repository.seedChunksForDatasource(datasourceId, [ + { + id: "chunk-orders-activation-1", + datasourceId, + domain: "schema", + content: "table orders(id, amount)" + } + ]); + + const first = await builder.buildAndActivate({ + datasourceId, + sourceVersion: "semantic-assets-builder-v1:mock", + buildReason: "initial_build", + createdByRunId: "run-builder-activation-v1", + activatedByRunId: "run-builder-activation-v1", + manifest: { + fingerprint: "semantic-assets-builder-v1", + summary: { + entryCount: 1, + preparedEntryCount: 1, + familyCounts: { full_schema: 1 }, + reasonCodes: ["prepared"] + } + } + }); + const second = await builder.buildAndActivate({ + datasourceId, + sourceVersion: "semantic-assets-builder-v2:mock", + buildReason: "refresh_build", + createdByRunId: "run-builder-activation-v2", + activatedByRunId: "run-builder-activation-v2", + manifest: { + fingerprint: "semantic-assets-builder-v2", + summary: { + entryCount: 1, + preparedEntryCount: 1, + familyCounts: { full_schema: 1 }, + reasonCodes: ["prepared"] + } + } + }); + + expect(second.manifestFingerprint).toBe("semantic-assets-builder-v2"); + expect(second.activation.replacedVersionIds).toContain(first.indexVersionId); + expect(second.activation.replacedSourceVersions).toContain( + "semantic-assets-builder-v1:mock" + ); + + await moduleRef.close(); + }); }); diff --git a/apps/backend/test/integration/rag-quality.spec.ts b/apps/backend/test/integration/rag-quality.spec.ts index 14495bb..1fccc7a 100644 --- a/apps/backend/test/integration/rag-quality.spec.ts +++ b/apps/backend/test/integration/rag-quality.spec.ts @@ -49,6 +49,18 @@ describe("rag quality gate integration", () => { priorSqlSafetyRejectedCount: 0, priorSqlFallbackToGenerationCount: 0 }); + expect(snapshot.preparationPlane).toMatchObject({ + sampleSize: 0, + familyCounts: {}, + preparedEntryCount: 0, + degradedEntryCount: 0, + skippedEntryCount: 0, + permissionFilteredAssetCount: 0, + selectedAssetCount: 0, + staleReasons: [], + lifecycleStatuses: [], + completenessReady: false + }); await moduleRef.close(); }); @@ -188,4 +200,47 @@ describe("rag quality gate integration", () => { await moduleRef.close(); }); + + it("aggregates preparation-plane completeness counters", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + const quality = moduleRef.get(RagQualityService); + quality.reset(); + + quality.recordPreparationPlane({ + runId: "run-rag-quality-preparation-v1", + datasourceId: "ds-rag-quality-preparation", + manifestFingerprint: "semantic-assets-quality-v1", + activeIndexVersionId: "idx-quality-v1", + familyCounts: { + table_description: 2, + full_schema: 1 + }, + preparedEntryCount: 3, + permissionFilteredAssetCount: 1, + selectedAssetCount: 2, + lifecycleStatus: "retrieved" + }); + + const snapshot = quality.snapshot(); + + expect(snapshot.preparationPlane).toMatchObject({ + sampleSize: 1, + latestManifestFingerprint: "semantic-assets-quality-v1", + latestActiveIndexVersionId: "idx-quality-v1", + familyCounts: { + full_schema: 1, + table_description: 2 + }, + preparedEntryCount: 3, + permissionFilteredAssetCount: 1, + selectedAssetCount: 2, + staleReasons: [], + lifecycleStatuses: ["retrieved"], + completenessReady: true + }); + + await moduleRef.close(); + }); }); diff --git a/apps/backend/test/integration/rag-retrieval.service.spec.ts b/apps/backend/test/integration/rag-retrieval.service.spec.ts index b0847ed..e0a1d1b 100644 --- a/apps/backend/test/integration/rag-retrieval.service.spec.ts +++ b/apps/backend/test/integration/rag-retrieval.service.spec.ts @@ -403,14 +403,17 @@ describe("rag retrieval service integration", () => { const priorSqlLane = response.retrieval_bundle.prior_sql_lane; expect(priorSqlLane).toBeDefined(); expect(priorSqlLane?.status).toBe("filtered"); - expect(priorSqlLane?.matched_count).toBe(2); + expect(priorSqlLane?.matched_count).toBe(1); expect(priorSqlLane?.selected_count).toBe(0); - expect(priorSqlLane?.filtered_count).toBe(2); + expect(priorSqlLane?.filtered_count).toBe(1); expect(priorSqlLane?.degrade_reasons).toEqual( - expect.arrayContaining([ - "prior_sql_filtered_workspace_mismatch", - "prior_sql_filtered_not_in_allowed_tables" - ]) + expect.arrayContaining(["prior_sql_filtered_workspace_mismatch"]) + ); + expect(response.retrieval_bundle.permission_filtering).toMatchObject( + expect.objectContaining({ + denied_evidence_ids: expect.arrayContaining(["chunk-prior-filter-tables"]), + reason_codes: expect.arrayContaining(["permission_filtered_before_ranking"]) + }) ); expect( response.retrieval_bundle.candidates.some( @@ -430,6 +433,209 @@ describe("rag retrieval service integration", () => { await moduleRef.close(); }); + it("filters permission-scoped semantic assets before lane ranking and fusion", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + + const indexRepository = moduleRef.get(RagIndexRepository); + const indexBuilder = moduleRef.get(RagIndexBuilderService); + const retrievalService = moduleRef.get(RagRetrievalService); + + const datasourceId = "ds-rag-retrieval-permission-pre-rank"; + indexRepository.seedChunksForDatasource(datasourceId, [ + { + id: "chunk-allowed-orders-description", + datasourceId, + domain: "semantic_asset", + content: "Orders table contains amount and status facts.", + metadata: JSON.stringify({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-permission-v1", + manifestEntryId: "entry-orders-description", + sourceVersion: "schema-v1", + tableNames: ["orders"], + columnNames: ["amount", "status"], + visibilityScope: "table_permissions", + preparationStatus: "prepared" + }) + }, + { + id: "chunk-forbidden-secret-description", + datasourceId, + domain: "semantic_asset", + content: "secret_orders contains confidential revenue and margin facts.", + metadata: JSON.stringify({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-permission-v1", + manifestEntryId: "entry-secret-description", + sourceVersion: "schema-v1", + tableNames: ["secret_orders"], + columnNames: ["revenue", "margin"], + visibilityScope: "table_permissions", + preparationStatus: "prepared" + }) + } + ]); + await indexBuilder.buildAndActivate({ + datasourceId, + sourceVersion: "semantic-assets-permission-v1:mock", + createdByRunId: "run-rag-retrieval-permission-build-v1", + activatedByRunId: "run-rag-retrieval-permission-build-v1" + }); + + const response = await retrievalService.retrieve({ + query: "secret_orders revenue orders amount", + datasourceId, + allowedTables: ["orders"], + runId: "run-rag-retrieval-permission-pre-rank-v1" + }); + + const laneHitIds = Object.values(response.retrieval_bundle.lane_results).flatMap((lane) => + lane.hits.map((hit) => hit.chunk_id) + ); + expect(laneHitIds).not.toContain("chunk-forbidden-secret-description"); + expect(response.retrieval_bundle.candidates.map((item) => item.chunk_id)).not.toContain( + "chunk-forbidden-secret-description" + ); + expect(response.retrieval_bundle.candidates[0]?.chunk.metadata).toEqual( + expect.objectContaining({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-permission-v1", + sourceVersion: "schema-v1" + }) + ); + expect(response.retrieval_bundle.permission_filtering).toMatchObject({ + status: "applied", + denied_evidence_ids: ["chunk-forbidden-secret-description"], + reason_codes: expect.arrayContaining([ + "permission_filtered_before_ranking", + "permission_filtered_not_in_allowed_tables" + ]) + }); + + await moduleRef.close(); + }); + + it("supplements table-description-first recall with schema and relationship asset families", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + + const indexRepository = moduleRef.get(RagIndexRepository); + const indexBuilder = moduleRef.get(RagIndexBuilderService); + const retrievalService = moduleRef.get(RagRetrievalService); + const replayRepository = moduleRef.get(RagReplayRepository); + + const datasourceId = "ds-rag-retrieval-two-pass-schema"; + indexRepository.seedChunksForDatasource(datasourceId, [ + { + id: "chunk-two-pass-orders-description", + datasourceId, + domain: "semantic_asset", + content: "Orders table stores paid order facts for GMV analysis.", + metadata: JSON.stringify({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-two-pass-v1", + manifestEntryId: "entry-orders-description", + sourceVersion: "schema-v1", + tableNames: ["orders"], + columnNames: [], + visibilityScope: "datasource", + preparationStatus: "prepared" + }) + }, + { + id: "chunk-two-pass-orders-full-schema", + datasourceId, + domain: "semantic_asset", + content: "Schema: orders(id, amount, status, customer_id)", + metadata: JSON.stringify({ + assetFamily: "full_schema", + manifestFingerprint: "semantic-assets-two-pass-v1", + manifestEntryId: "entry-orders-full-schema", + sourceVersion: "schema-v1", + tableNames: ["orders"], + columnNames: ["id", "amount", "status", "customer_id"], + visibilityScope: "datasource", + preparationStatus: "prepared" + }) + }, + { + id: "chunk-two-pass-orders-relationship", + datasourceId, + domain: "semantic_asset", + content: "Relationship: orders.customer_id -> customers.id", + metadata: JSON.stringify({ + assetFamily: "relationship_binding", + manifestFingerprint: "semantic-assets-two-pass-v1", + manifestEntryId: "entry-orders-customers-rel", + sourceVersion: "modeling-v1", + modelingRevision: 3, + tableNames: ["orders", "customers"], + columnNames: ["customer_id", "id"], + visibilityScope: "datasource", + preparationStatus: "prepared" + }) + } + ]); + await indexBuilder.buildAndActivate({ + datasourceId, + sourceVersion: "semantic-assets-two-pass-v1:mock", + createdByRunId: "run-rag-retrieval-two-pass-build-v1", + activatedByRunId: "run-rag-retrieval-two-pass-build-v1" + }); + + const response = await retrievalService.retrieve({ + query: "paid orders GMV", + datasourceId, + runId: "run-rag-retrieval-two-pass-v1", + perLaneLimit: 2, + finalCandidateLimit: 6 + }); + + const candidateIds = response.retrieval_bundle.candidates.map((item) => item.chunk_id); + expect(candidateIds).toEqual( + expect.arrayContaining([ + "chunk-two-pass-orders-description", + "chunk-two-pass-orders-full-schema", + "chunk-two-pass-orders-relationship" + ]) + ); + expect(response.retrieval_bundle.two_pass_schema_recall).toMatchObject({ + status: "applied", + selected_table_names: ["orders"], + table_description_evidence_ids: ["chunk-two-pass-orders-description"], + supplemental_evidence_ids: expect.arrayContaining([ + "chunk-two-pass-orders-full-schema", + "chunk-two-pass-orders-relationship" + ]), + supplemental_families: expect.arrayContaining(["full_schema", "relationship_binding"]) + }); + + const replayEvents = await replayRepository.listByRunId("run-rag-retrieval-two-pass-v1"); + const fusedPayload = JSON.parse( + replayEvents.find((item) => item.replayKey === "retrieval:fused")?.payload ?? "{}" + ) as { + twoPassSchemaRecall?: { status?: string; supplementalFamilies?: string[] }; + candidates?: Array<{ assetFamily?: string; manifestFingerprint?: string }>; + }; + expect(fusedPayload.twoPassSchemaRecall?.status).toBe("applied"); + expect(fusedPayload.twoPassSchemaRecall?.supplementalFamilies).toEqual( + expect.arrayContaining(["full_schema", "relationship_binding"]) + ); + expect(fusedPayload.candidates).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-two-pass-v1" + }) + ]) + ); + + await moduleRef.close(); + }); + it("marks trusted prior SQL as ambiguous when multiple high-confidence candidates remain", async () => { const moduleRef = await Test.createTestingModule({ imports: [AppModule] diff --git a/apps/backend/test/integration/saved-prior-sql-capture.spec.ts b/apps/backend/test/integration/saved-prior-sql-capture.spec.ts index e6b415e..0c1793c 100644 --- a/apps/backend/test/integration/saved-prior-sql-capture.spec.ts +++ b/apps/backend/test/integration/saved-prior-sql-capture.spec.ts @@ -77,7 +77,11 @@ const createService = (override?: { status: "active", entryCount: 1, archivedChannels: ["lexical", "dense"], - denseMode: "mock_provider" + denseMode: "mock_provider", + activation: { + replacedVersionIds: [], + replacedSourceVersions: [] + } }) }; const service = new SavedPriorSqlService( diff --git a/apps/backend/test/integration/saved-prior-sql-ingestion.spec.ts b/apps/backend/test/integration/saved-prior-sql-ingestion.spec.ts index fc73fff..3be1c9b 100644 --- a/apps/backend/test/integration/saved-prior-sql-ingestion.spec.ts +++ b/apps/backend/test/integration/saved-prior-sql-ingestion.spec.ts @@ -104,6 +104,20 @@ describe("saved prior sql ingestion integration", () => { const activeEntries = await indexRepository.listEntriesByVersion(activeVersion?.id ?? ""); const priorChunkIds = activeEntries.filter((item) => item.domain === "sql_example"); expect(priorChunkIds).toHaveLength(1); + const priorMetadata = JSON.parse(priorChunkIds[0]?.metadata ?? "{}"); + expect(priorMetadata.sourceMetadata).toEqual( + expect.objectContaining({ + assetFamily: "prior_question_sql", + preparationStatus: "prepared", + visibilityScope: "workspace", + trusted: true, + verified: true, + viewStatus: "active", + compatibilitySignals: expect.objectContaining({ + viewStatus: "active" + }) + }) + ); await moduleRef.close(); }); diff --git a/apps/backend/test/integration/settings-rag-readiness.spec.ts b/apps/backend/test/integration/settings-rag-readiness.spec.ts index 9f5d45f..db1314f 100644 --- a/apps/backend/test/integration/settings-rag-readiness.spec.ts +++ b/apps/backend/test/integration/settings-rag-readiness.spec.ts @@ -2,6 +2,7 @@ import { INestApplication } from "@nestjs/common"; import { Test } from "@nestjs/testing"; import request from "supertest"; import { AppModule } from "../../src/app.module"; +import { SemanticAssetReindexService } from "../../src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service"; import { requestActorMiddleware } from "../../src/modules/auth/request-actor.middleware"; import { requestIdMiddleware } from "../../src/modules/middleware/request-id.middleware"; import { createSeededSqliteFixture } from "../support/sqlite-fixture"; @@ -100,4 +101,44 @@ describe("settings rag readiness integration", () => { "siliconflow" ); }); + + it("exposes semantic asset readiness summary in /health after reindex", async () => { + const reindex = app.get(SemanticAssetReindexService, { strict: false }); + const result = await reindex.reindex({ + datasourceId: "ds-rag-readiness-semantic-assets", + workspaceId: "ws-rag-readiness", + triggers: ["schema"], + reason: "health_readiness", + runId: "run-rag-readiness-semantic-assets", + sourceSnapshots: [ + { + family: "table_description", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.description" }, + sourceVersion: "schema-health-v1", + sourceHash: "hash-schema-health-v1", + tableName: "orders", + content: "Orders table stores paid and pending order facts." + } + ] + }); + + const healthRes = await request(app.getHttpServer()).get("/health"); + + expect(healthRes.status).toBe(200); + expect(healthRes.body.data.dependencies.ragConfig.semanticAssetReadiness).toMatchObject({ + status: "ready", + datasourceId: "ds-rag-readiness-semantic-assets", + activeManifestFingerprint: result.semanticAssetVersion, + activeIndexVersionId: result.indexVersionId, + familyCounts: { + table_description: 1 + }, + staleReasons: [] + }); + expect( + healthRes.body.data.dependencies.ragConfig.semanticAssetReadiness.embeddingProfile + .model + ).toBeTruthy(); + }); }); diff --git a/apps/backend/test/integration/text2sql-semantic-asset-reindex.spec.ts b/apps/backend/test/integration/text2sql-semantic-asset-reindex.spec.ts index 5f19d52..da5420a 100644 --- a/apps/backend/test/integration/text2sql-semantic-asset-reindex.spec.ts +++ b/apps/backend/test/integration/text2sql-semantic-asset-reindex.spec.ts @@ -54,8 +54,20 @@ describe("text2sql semantic asset reindex integration", () => { expect(result.status).toBe("reindexed"); expect(result.indexVersionId).toBeDefined(); + expect(result.lifecycle.status).toBe("activated"); + expect(result.lifecycle.latestManifestFingerprint).toBe(result.semanticAssetVersion); + expect(result.lifecycle.rebuildable).toBe(false); expect(result.semanticAssetVersion).toContain("semantic-assets-"); + expect(result.manifestSummary.fingerprint).toBe(result.semanticAssetVersion); + expect(result.manifestSummary.entryCount).toBe(3); + expect(result.manifestSummary.familyCounts).toMatchObject({ + full_schema: 1, + prior_question_sql: 1, + project_metadata: 1 + }); + expect(result.manifestSummary.preparedEntryCount).toBe(3); expect(result.sourceVersion).toContain("embedding_model"); + expect(result.sourceVersion).toContain(result.manifestSummary.fingerprint); expect(result.embeddingProfile.provider).toBe("volcengine"); expect(result.embeddingProfile.model).toBe("embedding-v2"); expect(result.embeddingProfile.vectorVersion).toBe("v3"); @@ -66,6 +78,59 @@ describe("text2sql semantic asset reindex integration", () => { await moduleRef.close(); }); + it("builds active index entries from manifest typed chunk inputs", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + const service = moduleRef.get(SemanticAssetReindexService); + const repository = moduleRef.get(RagIndexRepository); + + const result = await service.reindex({ + datasourceId: "ds-semantic-manifest-chunks", + workspaceId: "ws-semantic-reindex", + triggers: ["schema"], + reason: "unit3_manifest_build", + runId: "run-semantic-asset-reindex-manifest-build", + sourceSnapshots: [ + { + family: "table_description", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.description" }, + sourceVersion: "schema-v3", + sourceHash: "hash-orders-description-v3", + tableName: "orders", + content: "Orders table stores paid and pending order facts.", + summary: { description: "Paid and pending order facts" } + } + ] + }); + + expect(result.status).toBe("reindexed"); + expect(result.preparedSourceCount).toBe(1); + expect(result.typedChunkInputCount).toBe(1); + expect(result.entryCount).toBe(1); + + const active = await repository.getActiveVersion("ds-semantic-manifest-chunks"); + expect(active?.id).toBe(result.indexVersionId); + expect(active?.sourceVersion).toContain(result.semanticAssetVersion); + + const entries = await repository.listEntriesByVersion(result.indexVersionId ?? ""); + expect(entries).toHaveLength(1); + expect(entries[0]?.domain).toBe("semantic_asset"); + const metadata = JSON.parse(entries[0]?.metadata ?? "{}"); + expect(metadata.sourceMetadata).toEqual( + expect.objectContaining({ + assetFamily: "table_description", + manifestFingerprint: result.semanticAssetVersion, + preparationStatus: "prepared", + sourceVersion: "schema-v3", + visibilityScope: "datasource" + }) + ); + + await moduleRef.close(); + }); + it("skips reindex when the same source version is already active", async () => { const moduleRef = await Test.createTestingModule({ imports: [AppModule] @@ -99,7 +164,91 @@ describe("text2sql semantic asset reindex integration", () => { expect(first.status).toBe("reindexed"); expect(second.status).toBe("skipped"); + expect(second.lifecycle.status).toBe("already_active"); + expect(second.lifecycle.activeIndexVersionId).toBe(first.indexVersionId); expect(second.reasonCodes).toContain("skip:already_active"); + expect(second.manifestSummary.fingerprint).toBe(second.semanticAssetVersion); + expect(second.manifestSummary.familyCounts).toMatchObject({ + full_schema: 1, + project_metadata: 1 + }); + + await moduleRef.close(); + }); + + it("reports stale rebuildable lifecycle before replacing an old active manifest", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + const service = moduleRef.get(SemanticAssetReindexService); + + const datasourceId = "ds-semantic-manifest-stale"; + const first = await service.reindex({ + datasourceId, + workspaceId: "ws-semantic-reindex", + triggers: ["schema"], + reason: "unit3_manifest_stale_v1", + runId: "run-semantic-asset-reindex-stale-v1", + sourceSnapshots: [ + { + family: "table_description", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.description" }, + sourceVersion: "schema-v1", + sourceHash: "hash-orders-description-v1", + tableName: "orders", + content: "Orders table stores order facts." + } + ] + }); + const second = await service.reindex({ + datasourceId, + workspaceId: "ws-semantic-reindex", + triggers: ["schema"], + reason: "unit3_manifest_stale_v2", + runId: "run-semantic-asset-reindex-stale-v2", + sourceSnapshots: [ + { + family: "table_description", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.description" }, + sourceVersion: "schema-v2", + sourceHash: "hash-orders-description-v2", + tableName: "orders", + content: "Orders table stores paid order facts." + } + ] + }); + + expect(first.status).toBe("reindexed"); + expect(second.status).toBe("reindexed"); + expect(second.lifecycle.status).toBe("activated"); + expect(second.lifecycle.previousActiveIndexVersionId).toBe(first.indexVersionId); + expect(second.lifecycle.activeManifestFingerprint).toBe(first.semanticAssetVersion); + expect(second.lifecycle.staleReasons).toContain("stale_source_version"); + + await moduleRef.close(); + }); + + it("returns manifest skip evidence when no reindex triggers are present", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + const service = moduleRef.get(SemanticAssetReindexService); + + const result = await service.reindex({ + datasourceId: "sqlite_main", + workspaceId: "ws-semantic-reindex", + triggers: [], + reason: "empty_trigger_guard", + runId: "run-semantic-asset-reindex-empty" + }); + + expect(result.status).toBe("skipped"); + expect(result.reasonCodes).toContain("skip:no_triggers"); + expect(result.manifestSummary.entryCount).toBe(0); + expect(result.manifestSummary.reasonCodes).toContain("skipped_no_triggers"); + expect(result.manifestSummary.fingerprint).toBe(result.semanticAssetVersion); await moduleRef.close(); }); diff --git a/apps/backend/test/integration/text2sql-v2-focused-coverage-gate.spec.ts b/apps/backend/test/integration/text2sql-v2-focused-coverage-gate.spec.ts index f002f6e..5ff11be 100644 --- a/apps/backend/test/integration/text2sql-v2-focused-coverage-gate.spec.ts +++ b/apps/backend/test/integration/text2sql-v2-focused-coverage-gate.spec.ts @@ -36,6 +36,12 @@ const LEGACY_STAGE_OWNER = "apps/backend/src/modules/conversation/text2sql/stages/run-v2-state-machine.stage.ts"; const ARTIFACT_REF_OWNER = "apps/backend/src/modules/conversation/artifacts/text2sql-v2-artifact-ref.service.ts"; +const PREPARATION_READINESS_OWNER = + "apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts"; +const PREPARATION_RETRIEVAL_OWNER = + "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts"; +const PREPARATION_REINDEX_OWNER = + "apps/backend/src/modules/knowledge/rag/retrieval/semantic-asset-reindex.service.ts"; const ALL_COVERAGE_FILES = [ ...LEGACY_CRITICAL_FILES, @@ -45,7 +51,10 @@ const ALL_COVERAGE_FILES = [ PRE_LAYERED_LANGGRAPH_GRAPH_OWNER, PRE_LAYERED_LANGGRAPH_RESULT_MAPPER_OWNER, PRE_LAYERED_INTAKE_NODE_OWNER, - ARTIFACT_REF_OWNER + ARTIFACT_REF_OWNER, + PREPARATION_READINESS_OWNER, + PREPARATION_RETRIEVAL_OWNER, + PREPARATION_REINDEX_OWNER ] as const; function fileCoverage(params: { @@ -266,6 +275,64 @@ function coveredMatrix(): CloseoutFlowMatrix { behaviorTestStatus: "covered" as const, coverageOwnerStatus: "covered" as const })), + preparationPlaneRows: [ + { + id: "manifest-completeness", + implementationOwners: [PREPARATION_READINESS_OWNER, PREPARATION_REINDEX_OWNER], + evidenceOwners: ["dependencies.ragConfig.semanticAssetReadiness"], + expectedTestFiles: ["apps/backend/test/integration/settings-rag-readiness.spec.ts"], + behaviorTestStatus: "covered", + coverageOwnerStatus: "covered", + requiredEvidenceSignals: ["activeManifestFingerprint", "familyCounts"] + }, + { + id: "typed-family-generation", + implementationOwners: [PREPARATION_REINDEX_OWNER], + evidenceOwners: ["manifestSummary.familyCounts"], + expectedTestFiles: ["apps/backend/test/unit/semantic-asset-preparer.service.spec.ts"], + behaviorTestStatus: "covered", + coverageOwnerStatus: "covered", + requiredEvidenceSignals: ["assetFamily", "manifestFingerprint"] + }, + { + id: "activation-stale-behavior", + implementationOwners: [PREPARATION_READINESS_OWNER, PREPARATION_REINDEX_OWNER], + evidenceOwners: ["semanticAssetReadiness.staleReasons"], + expectedTestFiles: [ + "apps/backend/test/integration/text2sql-semantic-asset-reindex.spec.ts" + ], + behaviorTestStatus: "covered", + coverageOwnerStatus: "covered", + requiredEvidenceSignals: ["staleReasons", "rebuildable"] + }, + { + id: "permission-filtering-before-ranking", + implementationOwners: [PREPARATION_RETRIEVAL_OWNER], + evidenceOwners: ["retrieval:fused.permissionFiltering"], + expectedTestFiles: ["apps/backend/test/integration/rag-retrieval.service.spec.ts"], + behaviorTestStatus: "covered", + coverageOwnerStatus: "covered", + requiredEvidenceSignals: ["permission_filtered_before_ranking"] + }, + { + id: "two-pass-schema-recall", + implementationOwners: [PREPARATION_RETRIEVAL_OWNER], + evidenceOwners: ["retrieval:fused.twoPassSchemaRecall"], + expectedTestFiles: ["apps/backend/test/integration/rag-retrieval.service.spec.ts"], + behaviorTestStatus: "covered", + coverageOwnerStatus: "covered", + requiredEvidenceSignals: ["table_description", "full_schema"] + }, + { + id: "prior-sql-diagnostics", + implementationOwners: [PREPARATION_RETRIEVAL_OWNER], + evidenceOwners: ["retrieval:fused.priorSqlLane"], + expectedTestFiles: ["apps/backend/test/integration/rag-retrieval.service.spec.ts"], + behaviorTestStatus: "covered", + coverageOwnerStatus: "covered", + requiredEvidenceSignals: ["stale", "ambiguous", "filtered"] + } + ], runtimePaths: { currentActivePath: [ "apps/backend/src/modules/conversation/text2sql/text2sql-workflow-runner.service.ts", @@ -313,6 +380,7 @@ describe("text2sql v2 focused coverage gate", () => { expect(report.flowMatrix.incompleteRuntimeCoverageRows).toEqual([]); expect(report.flowMatrix.incompleteRuntimeArtifactProducerRows).toEqual([]); expect(report.flowMatrix.incompleteStreamLifecycleRows).toEqual([]); + expect(report.flowMatrix.incompletePreparationPlaneRows).toEqual([]); expect(report.rollout).toMatchObject({ gatePass: true, recommendedStage: "closeout_ready", @@ -506,6 +574,34 @@ describe("text2sql v2 focused coverage gate", () => { ]); }); + it("fails when preparation-plane rows lose required evidence signals", () => { + const matrix = coveredMatrix(); + matrix.preparationPlaneRows = matrix.preparationPlaneRows?.map((row) => + row.id === "permission-filtering-before-ranking" + ? { + ...row, + requiredEvidenceSignals: [] + } + : row + ); + + const report = evaluateFocusedCoverageGate({ + coverage: coverageFor(ALL_COVERAGE_FILES), + matrix + }); + + expect(report.rollout.gatePass).toBe(false); + expect(report.flowMatrix.incompletePreparationPlaneRows).toEqual([ + { + id: "permission-filtering-before-ranking", + reasons: ["missing_preparation_evidence_signals"] + } + ]); + expect(report.rollout.reasons).toContain( + "preparation_plane:permission-filtering-before-ranking:missing_preparation_evidence_signals" + ); + }); + it("requires explicit owner migration metadata when a critical file is replaced", () => { const replacementFile = "apps/backend/src/modules/conversation/adapters/sql-correction-decision.service.ts"; diff --git a/apps/backend/test/jest-e2e.json b/apps/backend/test/jest-e2e.json index c120a9b..baec191 100644 --- a/apps/backend/test/jest-e2e.json +++ b/apps/backend/test/jest-e2e.json @@ -8,6 +8,8 @@ }, "testEnvironment": "node", "moduleNameMapper": { + "^@text2sql/chat-stream-protocol$": "/../../packages/chat-stream-protocol/src", + "^@text2sql/chat-stream-protocol/(.*)$": "/../../packages/chat-stream-protocol/src/$1", "^@text2sql/shared-types$": "/../../packages/shared-types/src" } } diff --git a/apps/backend/test/unit/capability-boundary-check.spec.ts b/apps/backend/test/unit/capability-boundary-check.spec.ts index f9e2f15..65bb8ec 100644 --- a/apps/backend/test/unit/capability-boundary-check.spec.ts +++ b/apps/backend/test/unit/capability-boundary-check.spec.ts @@ -208,13 +208,17 @@ export class Text2SqlV2ArtifactRefService { const report = await runCapabilityBoundaryCheck({ repoRoot }); expect(report.violations).toHaveLength(0); - expect((report as unknown as Record).conversationKnowledgeSubpath).toMatchObject({ + expect(report.conversationKnowledgeSubpath).toMatchObject({ currentCount: 0, - baselineCount: 15, - remainingFromBaseline: 15, + baselineCount: 9, + remainingFromBaseline: 9, overBaselineCount: 0, exceedsBaseline: false }); + expect(report.ragImportReport).toMatchObject({ + blockedLegacyActiveImportCount: 0, + blockedDirectKnowledgeImplementationImportCount: 0 + }); }); it("reports conversation/text2sql -> legacy modules/chat imports as violations", async () => { @@ -249,11 +253,11 @@ export class PrepareRunStage { await writeRepoFile( repoRoot, "apps/backend/src/modules/conversation/chat/application/shared/chat-post-run-hooks.service.ts", - `import { RagRetrievalService } from "../../../../knowledge/rag/retrieval/rag-retrieval.service"; + `import { KnowledgeFacadeContract } from "../../../../knowledge/contracts/knowledge-facade.contract"; import { MemoryPromotionService } from "../../../../knowledge/memory/memory-promotion.service"; export class ChatPostRunHooksService { constructor( - private readonly retrieval: RagRetrievalService, + private readonly facade: KnowledgeFacadeContract, private readonly promotion: MemoryPromotionService ) {} } @@ -261,8 +265,8 @@ export class ChatPostRunHooksService { ); await writeRepoFile( repoRoot, - "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts", - "export class RagRetrievalService {}" + "apps/backend/src/modules/knowledge/contracts/knowledge-facade.contract.ts", + "export interface KnowledgeFacadeContract {}" ); await writeRepoFile( repoRoot, @@ -277,8 +281,8 @@ export class ChatPostRunHooksService { { sourceFile: "apps/backend/src/modules/conversation/chat/application/shared/chat-post-run-hooks.service.ts", - targetFile: "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts", - reason: "Temporary bridge: keep retrieval import until facade migration lands." + targetFile: "apps/backend/src/modules/knowledge/contracts/knowledge-facade.contract.ts", + reason: "Temporary bridge: keep facade contract direct import until stable entry migration lands." } ] }); @@ -292,7 +296,7 @@ export class ChatPostRunHooksService { targetFile: "apps/backend/src/modules/knowledge/memory/memory-promotion.service.ts", line: 2 }); - expect((report as unknown as Record).conversationKnowledgeSubpath).toMatchObject({ + expect(report.conversationKnowledgeSubpath).toMatchObject({ currentCount: 2, baselineCount: 2, remainingFromBaseline: 0, @@ -300,4 +304,168 @@ export class ChatPostRunHooksService { exceedsBaseline: false }); }); + + it("reports blocked direct knowledge/rag implementation imports", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts", + `import { RagRetrievalService } from "../../../knowledge/rag/retrieval/rag-retrieval.service"; +export class RetrieveKnowledgeNode { + constructor(private readonly retrieval: RagRetrievalService) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts", + "export class RagRetrievalService {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.ragImportReport.blockedDirectKnowledgeImplementationImportCount).toBe(1); + expect(report.ragImportReport.blockedDirectKnowledgeImplementationImports[0]).toMatchObject({ + sourceFile: "apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts", + targetFile: "apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts" + }); + }); + + it("reports blocked direct semantic asset preparation imports from conversation", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts", + `import { SemanticAssetReadinessService } from "../../../knowledge/rag/preparation/semantic-asset-readiness.service"; +export class RetrieveKnowledgeNode { + constructor(private readonly readiness: SemanticAssetReadinessService) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts", + "export class SemanticAssetReadinessService {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.ragImportReport.blockedDirectKnowledgeImplementationImportCount).toBe(1); + expect(report.ragImportReport.blockedDirectKnowledgeImplementationImports[0]).toMatchObject({ + sourceFile: "apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts", + targetFile: + "apps/backend/src/modules/knowledge/rag/preparation/semantic-asset-readiness.service.ts" + }); + }); + + it("reports blocked legacy active rag service imports", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts", + `import { RagRetrievalService } from "../../../rag/retrieval/rag-retrieval.service"; +export class RetrieveKnowledgeNode { + constructor(private readonly retrieval: RagRetrievalService) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts", + "export class RagRetrievalService {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.ragImportReport.blockedLegacyActiveImportCount).toBe(1); + expect(report.ragImportReport.blockedLegacyActiveImports[0]).toMatchObject({ + sourceFile: "apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts", + targetFile: "apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts" + }); + }); + + it("reports blocked legacy rag compat alias entry imports outside rag compat wiring", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/system/system.module.ts", + `import { RagModule } from "../rag/rag.module"; +export class SystemModule { + constructor(private readonly ragModule: RagModule) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/rag/rag.module.ts", + "export class RagModule {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.ragImportReport.blockedLegacyCompatAliasEntryImportCount).toBe(1); + expect(report.ragImportReport.blockedLegacyCompatAliasEntryImports[0]).toMatchObject({ + sourceFile: "apps/backend/src/modules/system/system.module.ts", + targetFile: "apps/backend/src/modules/rag/rag.module.ts" + }); + }); + + it("allows legacy rag compat alias module imports inside rag namespace wiring", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/rag/compat/compat-bridge.module.ts", + `import { RagModule } from "../rag.module"; +export class CompatBridgeModule { + constructor(private readonly ragModule: RagModule) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/rag/rag.module.ts", + "export class RagModule {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.violations).toHaveLength(0); + expect(report.ragImportReport.blockedLegacyCompatAliasEntryImportCount).toBe(0); + }); + + it("allows the legacy rag module compat-only alias wiring into knowledge rag module", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/rag/rag.module.ts", + `import { RagModule as KnowledgeRagModule } from "../knowledge/rag/rag.module"; +export class RagModule { + constructor(private readonly knowledgeRagModule: KnowledgeRagModule) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/knowledge/rag/rag.module.ts", + "export class RagModule {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.violations).toHaveLength(0); + expect(report.ragImportReport.blockedDirectKnowledgeImplementationImportCount).toBe(0); + }); + + it("allows knowledge/rag owner imports to shared internals and reports classification reason", async () => { + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/knowledge/rag/rag.module.ts", + `import { RagIndexBuilderService } from "../../rag/index/rag-index-builder.service"; +export class RagModule { + constructor(private readonly builder: RagIndexBuilderService) {} +} +` + ); + await writeRepoFile( + repoRoot, + "apps/backend/src/modules/rag/index/rag-index-builder.service.ts", + "export class RagIndexBuilderService {}" + ); + + const report = await runCapabilityBoundaryCheck({ repoRoot }); + expect(report.violations).toHaveLength(0); + expect(report.ragImportReport.allowedSharedInternalImportCount).toBe(1); + expect(report.ragImportReport.allowedSharedInternalImports[0]).toMatchObject({ + sourceFile: "apps/backend/src/modules/knowledge/rag/rag.module.ts", + targetFile: "apps/backend/src/modules/rag/index/rag-index-builder.service.ts" + }); + }); }); diff --git a/apps/backend/test/unit/chat.service.spec.ts b/apps/backend/test/unit/chat.service.spec.ts index c4ef392..70c285e 100644 --- a/apps/backend/test/unit/chat.service.spec.ts +++ b/apps/backend/test/unit/chat.service.spec.ts @@ -64,6 +64,7 @@ describe("ChatService", () => { "近30天支付方式分布", "req-2", onEvent, + undefined, contextEnvelope ); diff --git a/apps/backend/test/unit/delivery-contract.mapper.spec.ts b/apps/backend/test/unit/delivery-contract.mapper.spec.ts index 3c49b5a..af20cf4 100644 --- a/apps/backend/test/unit/delivery-contract.mapper.spec.ts +++ b/apps/backend/test/unit/delivery-contract.mapper.spec.ts @@ -1254,6 +1254,67 @@ describe("DeliveryContractMapper", () => { }); }); + it("projects safe preparation-plane summary from retrieval replay evidence", () => { + const mapper = new DeliveryContractMapper(new SandboxRuntimeService()); + const run = createBaseRun({ + trace: { + runId: "run-delivery-unit-preparation", + provider: "volcengine", + retryCount: 0, + steps: [] + } as SqlRun["trace"] + }); + + const delivery = mapper.map({ + run, + replayRecords: [ + { + replayKey: "retrieval:fused", + stage: "retrieval_fused", + indexVersionId: "idx-preparation-v1", + payload: JSON.stringify({ + status: "ready", + permissionFiltering: { + filteredCount: 2, + deniedChunkIds: ["forbidden-secret-orders"] + }, + twoPassSchemaRecall: { + status: "applied" + }, + candidates: [ + { + chunkId: "chunk-orders-description", + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-delivery-v1", + lifecycleState: "retrieved" + }, + { + chunkId: "chunk-orders-schema", + assetFamily: "full_schema", + manifestFingerprint: "semantic-assets-delivery-v1", + lifecycleState: "retrieved" + } + ] + }), + createdAt: "2026-04-18T00:00:01.000Z" + } + ] + }); + + expect(delivery.evidence?.preparationPlane).toEqual({ + manifestFingerprints: ["semantic-assets-delivery-v1"], + assetFamilyCounts: { + full_schema: 1, + table_description: 1 + }, + permissionFilteredAssetCount: 2, + twoPassSchemaRecallApplied: true + }); + expect(JSON.stringify(delivery.evidence?.preparationPlane)).not.toContain( + "forbidden-secret-orders" + ); + }); + it("keeps contract complete when artifact is absent", () => { const mapper = new DeliveryContractMapper(new SandboxRuntimeService()); const run = createBaseRun({ diff --git a/apps/backend/test/unit/knowledge-facade-contract.spec.ts b/apps/backend/test/unit/knowledge-facade-contract.spec.ts index b1c1433..3c5640d 100644 --- a/apps/backend/test/unit/knowledge-facade-contract.spec.ts +++ b/apps/backend/test/unit/knowledge-facade-contract.spec.ts @@ -21,6 +21,11 @@ import { } from "../../src/modules/knowledge/contracts/knowledge-semantic-registry.contract"; import { KnowledgeChatSupportFacade } from "../../src/modules/knowledge/knowledge-chat-support.facade"; import { KnowledgeModule } from "../../src/modules/knowledge/knowledge.module"; +import { RagModule as LegacyRagModule } from "../../src/modules/rag/rag.module"; +import { RagRetrievalService as KnowledgeRagRetrievalService } from "../../src/modules/knowledge/rag/retrieval/rag-retrieval.service"; +import { RagRerankService as KnowledgeRagRerankService } from "../../src/modules/knowledge/rag/rerank/rag-rerank.service"; +import { RagRetrievalService as LegacyRagRetrievalService } from "../../src/modules/rag/retrieval/rag-retrieval.service"; +import { RagRerankService as LegacyRagRerankService } from "../../src/modules/rag/rerank/rag-rerank.service"; describe("KnowledgeModule contract providers", () => { it("exports facade + capability contract tokens", async () => { @@ -61,4 +66,28 @@ describe("KnowledgeModule contract providers", () => { expect(semanticContract.registry).toBe(facadeContract.semanticRegistry.registry); expect(memoryContract.promotion).toBe(facadeContract.memory.promotion); }); + + it("resolves legacy rag tokens to canonical knowledge implementations", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [LegacyRagModule] + }).compile(); + + const legacyRetrieval = moduleRef.get(LegacyRagRetrievalService, { + strict: false + }); + const canonicalRetrieval = moduleRef.get(KnowledgeRagRetrievalService, { + strict: false + }); + const legacyRerank = moduleRef.get(LegacyRagRerankService, { + strict: false + }); + const canonicalRerank = moduleRef.get(KnowledgeRagRerankService, { + strict: false + }); + + expect(canonicalRetrieval).toBeDefined(); + expect(canonicalRerank).toBeDefined(); + expect(legacyRetrieval).toBe(canonicalRetrieval); + expect(legacyRerank).toBe(canonicalRerank); + }); }); diff --git a/apps/backend/test/unit/llm-gateway.service.spec.ts b/apps/backend/test/unit/llm-gateway.service.spec.ts index 89e3369..dc328ba 100644 --- a/apps/backend/test/unit/llm-gateway.service.spec.ts +++ b/apps/backend/test/unit/llm-gateway.service.spec.ts @@ -108,6 +108,60 @@ describe("LlmGatewayService", () => { }); }); + it("should retry sync generate once for invalid JSON response", async () => { + mockedGenerateText + .mockRejectedValueOnce(new Error("Invalid JSON response")) + .mockResolvedValueOnce({ + text: "SELECT COUNT(*) AS total_count FROM orders;" + } as Awaited>); + + const service = new LlmGatewayService( + { + llmMockMode: false + } as AppConfigService, + new LlmModelFactory() + ); + + const output = await service.generate( + { + systemPrompt: "sys", + userPrompt: "统计订单总数" + }, + runtime + ); + + expect(output.rawText).toBe("SELECT COUNT(*) AS total_count FROM orders;"); + expect(mockedGenerateText).toHaveBeenCalledTimes(2); + }); + + it("should retry sync generate with expanded timeout after timeout abort", async () => { + mockedGenerateText + .mockRejectedValueOnce(new Error("The operation was aborted due to timeout")) + .mockResolvedValueOnce({ + text: "SELECT COUNT(*) AS total_count FROM orders;" + } as Awaited>); + + const timeoutSpy = jest.spyOn(AbortSignal, "timeout"); + const service = new LlmGatewayService( + { + llmMockMode: false + } as AppConfigService, + new LlmModelFactory() + ); + + const output = await service.generate( + { + systemPrompt: "sys", + userPrompt: "统计订单总数" + }, + runtime + ); + + expect(output.rawText).toBe("SELECT COUNT(*) AS total_count FROM orders;"); + expect(timeoutSpy).toHaveBeenNthCalledWith(1, 3000); + expect(timeoutSpy).toHaveBeenNthCalledWith(2, 18000); + }); + it("should stream text and tool events", async () => { mockedStreamText.mockReturnValue({ fullStream: (async function* () { @@ -179,6 +233,71 @@ describe("LlmGatewayService", () => { expect(timeoutSpy).toHaveBeenCalledWith(9000); }); + it("should pass a composed abort signal to streamText when a user signal exists", async () => { + mockedStreamText.mockReturnValue({ + fullStream: (async function* () { + yield { type: "text-delta", text: "SELECT 1" }; + })(), + text: Promise.resolve("SELECT 1") + } as unknown as ReturnType); + + const service = new LlmGatewayService( + { + llmMockMode: false + } as AppConfigService, + new LlmModelFactory() + ); + const abortController = new AbortController(); + + await service.stream( + { + systemPrompt: "sys", + userPrompt: "统计订单状态分布" + }, + runtime, + { + abortSignal: abortController.signal + } + ); + + expect(mockedStreamText).toHaveBeenCalledWith( + expect.objectContaining({ + abortSignal: expect.any(AbortSignal) + }) + ); + }); + + it("should map user-aborted streams to USER_CANCELLED without generate fallback", async () => { + const abortError = new Error("This operation was aborted by the user"); + abortError.name = "AbortError"; + mockedStreamText.mockImplementation(() => { + throw abortError; + }); + + const service = new LlmGatewayService( + { + llmMockMode: false + } as AppConfigService, + new LlmModelFactory() + ); + + await expect( + service.stream( + { + systemPrompt: "sys", + userPrompt: "统计订单状态分布" + }, + runtime, + { + abortSignal: new AbortController().signal + } + ) + ).rejects.toMatchObject>({ + code: "USER_CANCELLED" + }); + expect(mockedGenerateText).not.toHaveBeenCalled(); + }); + it("should raise tool-call-only response when stream has no final text", async () => { mockedStreamText.mockReturnValue({ fullStream: (async function* () { @@ -255,6 +374,44 @@ describe("LlmGatewayService", () => { expect(mockedGenerateText).toHaveBeenCalledTimes(1); }); + it("should recover from invalid JSON stream responses via non-stream retry", async () => { + mockedStreamText.mockReturnValue({ + fullStream: (async function* () { + throw new Error("Invalid JSON response"); + })(), + text: Promise.resolve("") + } as unknown as ReturnType); + + mockedGenerateText.mockResolvedValue({ + text: "SELECT COUNT(*) AS total_count FROM orders;" + } as Awaited>); + + const events: string[] = []; + const service = new LlmGatewayService( + { + llmMockMode: false + } as AppConfigService, + new LlmModelFactory() + ); + + const output = await service.stream( + { + systemPrompt: "sys", + userPrompt: "统计订单总数" + }, + runtime, + { + onEvent: (event) => { + events.push(event.type); + } + } + ); + + expect(output.rawText).toBe("SELECT COUNT(*) AS total_count FROM orders;"); + expect(events).toEqual(["text-delta"]); + expect(mockedGenerateText).toHaveBeenCalledTimes(1); + }); + it("should recover with successful tool SQL when provider stream breaks after tool result", async () => { mockedStreamText.mockReturnValue({ fullStream: (async function* () { diff --git a/apps/backend/test/unit/rag-event-consumer.spec.ts b/apps/backend/test/unit/rag-event-consumer.spec.ts index 803fc10..2ce7bae 100644 --- a/apps/backend/test/unit/rag-event-consumer.spec.ts +++ b/apps/backend/test/unit/rag-event-consumer.spec.ts @@ -12,7 +12,11 @@ describe("RagEventConsumerService", () => { status: "active", entryCount: 2, archivedChannels: ["lexical", "dense"], - denseMode: "mock_provider" + denseMode: "mock_provider", + activation: { + replacedVersionIds: [], + replacedSourceVersions: [] + } }) }; const replayRepository: Pick = { diff --git a/apps/backend/test/unit/rag-rerank.service.spec.ts b/apps/backend/test/unit/rag-rerank.service.spec.ts index 62ba3c4..c3091a1 100644 --- a/apps/backend/test/unit/rag-rerank.service.spec.ts +++ b/apps/backend/test/unit/rag-rerank.service.spec.ts @@ -125,6 +125,9 @@ describe("rag rerank service", () => { expect(adapter.rerankWithMetadata).not.toHaveBeenCalled(); expect(response.retrieval_bundle.reranked?.length).toBe(2); + expect(response.retrieval_bundle.selected_context?.[0]?.metadata.lifecycleState).toBe( + "selected" + ); expect(response.retrieval_bundle.degrade_reasons).toEqual( expect.arrayContaining(["secondary_rerank_skipped_low_candidates"]) ); diff --git a/apps/backend/test/unit/retrieve-knowledge.node.spec.ts b/apps/backend/test/unit/retrieve-knowledge.node.spec.ts index fdcd48d..8c98105 100644 --- a/apps/backend/test/unit/retrieve-knowledge.node.spec.ts +++ b/apps/backend/test/unit/retrieve-knowledge.node.spec.ts @@ -56,4 +56,63 @@ describe("RetrieveKnowledgeNode", () => { expect(result.typedSummary.selectedContextCount).toBe(1); expect(result.evidenceRefs).toContain("schema-supplement:ds-sqlite:payments"); }); + + it("grounds sales questions to orders schema supplement when retrieval is degraded", async () => { + const queryExecutorRouter = { + execute: jest.fn(async () => ({ + columns: ["columnName", "dataType"], + rows: [ + { columnName: "id", dataType: "INTEGER" }, + { columnName: "created_at", dataType: "TEXT" }, + { columnName: "total_amount", dataType: "REAL" } + ] + })) + }; + const node = new RetrieveKnowledgeNode( + { agentRagRetrievalEnabled: false } as never, + queryExecutorRouter as never, + {} as never + ); + + const result = await node.run({ + question: "最近的销售额", + datasourceId: "ds-sqlite", + datasource: { + id: "ds-sqlite", + name: "SQLite", + type: "sqlite", + status: "available", + readonly: true, + shared: true, + config: { path: "/tmp/text2sql.db" }, + fileMeta: null, + unavailableAt: null, + deletedAt: null, + createdAt: new Date().toISOString(), + updatedAt: new Date().toISOString() + }, + runId: "run-sales-schema-supplement", + allowedTables: ["orders", "payments", "refunds"] + }); + + expect(queryExecutorRouter.execute).toHaveBeenCalledWith( + expect.objectContaining({ + sql: expect.stringContaining("pragma_table_info('orders')") + }) + ); + expect(result.retrievalBundle?.selected_context).toEqual([ + expect.objectContaining({ + chunk_id: "schema-supplement:ds-sqlite:orders", + metadata: expect.objectContaining({ + tableNames: ["orders"], + columnNames: [ + "orders.id", + "orders.created_at", + "orders.total_amount" + ] + }) + }) + ]); + expect(result.evidenceRefs).toContain("schema-supplement:ds-sqlite:orders"); + }); }); diff --git a/apps/backend/test/unit/semantic-asset-manifest.builder.spec.ts b/apps/backend/test/unit/semantic-asset-manifest.builder.spec.ts new file mode 100644 index 0000000..5fed66e --- /dev/null +++ b/apps/backend/test/unit/semantic-asset-manifest.builder.spec.ts @@ -0,0 +1,97 @@ +import { SemanticAssetManifestBuilder } from "../../src/modules/knowledge/rag/preparation/semantic-asset-manifest.builder"; +import { SEMANTIC_ASSET_REASON_CODES } from "../../src/modules/knowledge/rag/preparation/semantic-asset-reason-codes"; + +const embeddingProfile = { + provider: "volcengine", + model: "embedding-v2", + dimensions: 1024, + vectorVersion: "v3", + configSource: "settings" as const +}; + +describe("semantic asset manifest builder", () => { + const builder = new SemanticAssetManifestBuilder(); + + it("builds deterministic manifest entries from semantic asset triggers", () => { + const first = builder.build({ + datasourceId: "sqlite_main", + workspaceId: "ws-semantic-assets", + triggers: ["schema", "modeling_revision", "embedding_model"], + embeddingProfile, + modelingRevision: 7, + reasonCodes: ["semantic_asset_reindex_requested"] + }); + const second = builder.build({ + datasourceId: "sqlite_main", + workspaceId: "ws-semantic-assets", + triggers: ["embedding_model", "schema", "modeling_revision"], + embeddingProfile, + modelingRevision: 7, + reasonCodes: ["semantic_asset_reindex_requested"] + }); + + expect(first.fingerprint).toBe(second.fingerprint); + expect(first.fingerprint).toContain("semantic-assets-"); + expect(first.embeddingProfile).toEqual(embeddingProfile); + expect(first.entries).toHaveLength(3); + expect(first.familyCounts).toMatchObject({ + full_schema: 1, + relationship_binding: 1, + project_metadata: 1 + }); + expect(first.entries.every((entry) => entry.status === "prepared")).toBe(true); + expect(first.reasonCodes).toEqual( + expect.arrayContaining([SEMANTIC_ASSET_REASON_CODES.prepared, "trigger:schema"]) + ); + }); + + it("returns a skipped manifest summary when no triggers are present", () => { + const manifest = builder.build({ + datasourceId: "sqlite_main", + workspaceId: "ws-semantic-assets", + triggers: [], + embeddingProfile + }); + const summary = builder.summarize(manifest); + + expect(summary.entryCount).toBe(0); + expect(summary.reasonCodes).toContain(SEMANTIC_ASSET_REASON_CODES.skippedNoTriggers); + expect(summary.familyCounts).toEqual({}); + }); + + it("degrades only the affected entry when a provided source snapshot lacks a hash", () => { + const manifest = builder.build({ + datasourceId: "sqlite_main", + workspaceId: "ws-semantic-assets", + triggers: ["schema", "instructions"], + embeddingProfile, + sourceSnapshots: { + schema: { + family: "full_schema", + sourceRef: { type: "datasource_schema", ref: "active-schema" }, + sourceVersion: "schema-v1", + sourceHash: "schema-hash", + summary: { tableCount: 1 } + }, + instructions: { + family: "prompt_instruction", + sourceRef: { type: "prompt_instruction", ref: "runtime-eligible" }, + sourceVersion: "prompt-v1", + summary: { scene: "text2sql" } + } + } + }); + + const schemaEntry = manifest.entries.find((entry) => entry.family === "full_schema"); + const promptEntry = manifest.entries.find( + (entry) => entry.family === "prompt_instruction" + ); + + expect(schemaEntry?.status).toBe("prepared"); + expect(promptEntry?.status).toBe("degraded"); + expect(promptEntry?.reasonCodes).toContain( + SEMANTIC_ASSET_REASON_CODES.degradedMissingSourceHash + ); + expect(builder.summarize(manifest).degradedEntryCount).toBe(1); + }); +}); diff --git a/apps/backend/test/unit/semantic-asset-preparer.service.spec.ts b/apps/backend/test/unit/semantic-asset-preparer.service.spec.ts new file mode 100644 index 0000000..6cf1fec --- /dev/null +++ b/apps/backend/test/unit/semantic-asset-preparer.service.spec.ts @@ -0,0 +1,201 @@ +import { RagDocumentFactory } from "../../src/modules/rag/ingestion/rag-document.factory"; +import { SemanticAssetFamilyChunkMapper } from "../../src/modules/knowledge/rag/preparation/semantic-asset-family-chunk.mapper"; +import { SemanticAssetManifestBuilder } from "../../src/modules/knowledge/rag/preparation/semantic-asset-manifest.builder"; +import { SemanticAssetPreparerService } from "../../src/modules/knowledge/rag/preparation/semantic-asset-preparer.service"; +import { SEMANTIC_ASSET_REASON_CODES } from "../../src/modules/knowledge/rag/preparation/semantic-asset-reason-codes"; + +const embeddingProfile = { + provider: "mock", + model: "mock-embedding", + dimensions: 8, + vectorVersion: "v1", + configSource: "settings" as const +}; + +describe("SemanticAssetPreparerService", () => { + const preparer = new SemanticAssetPreparerService( + new SemanticAssetManifestBuilder(), + new SemanticAssetFamilyChunkMapper() + ); + + it("prepares typed families into manifest entries and ingestion sources", () => { + const result = preparer.prepare({ + datasourceId: "ds-main", + workspaceId: "ws-main", + embeddingProfile, + sourceSnapshots: [ + { + family: "table_description", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.description" }, + sourceVersion: "schema-v1", + sourceHash: "hash-table-description", + tableName: "orders", + title: "Orders table", + summary: { description: "Paid order facts" } + }, + { + family: "full_schema", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.full_schema" }, + sourceVersion: "schema-v1", + sourceHash: "hash-full-schema", + tableName: "orders", + columns: [ + { name: "id", type: "uuid" }, + { name: "amount", type: "decimal", description: "Order amount" } + ] + }, + { + family: "column_batch", + sourceKind: "datasource_schema", + sourceRef: { type: "datasource_schema", ref: "orders.columns" }, + sourceVersion: "schema-v1", + sourceHash: "hash-column-batch", + tableName: "orders", + columns: [{ name: "status", type: "text" }] + }, + { + sourceKind: "glossary_term", + sourceRef: { type: "glossary_term", ref: "gmv" }, + sourceVersion: "glossary-v1", + sourceHash: "hash-business-term", + term: "GMV", + definition: "Gross merchandise value", + synonyms: ["sales"], + tableNames: ["orders"], + columnNames: ["amount"] + }, + { + sourceKind: "saved_prior_sql", + sourceRef: { type: "saved_prior_sql", ref: "prior-1" }, + sourceVersion: "prior-v1", + sourceHash: "hash-prior-sql", + question: "paid GMV", + sql: "SELECT SUM(amount) FROM orders WHERE status = 'paid'", + trusted: true, + verified: true, + viewStatus: "active", + tableNames: ["orders"], + columnNames: ["amount", "status"] + }, + { + family: "project_metadata", + sourceKind: "project_metadata", + sourceRef: { type: "project_metadata", ref: "workspace" }, + sourceVersion: "project-v1", + sourceHash: "hash-project", + projectName: "Commerce" + } + ] + }); + + expect(result.manifestSummary.entryCount).toBe(6); + expect(result.manifestSummary.preparedEntryCount).toBe(6); + expect(result.manifestSummary.familyCounts).toEqual( + expect.objectContaining({ + table_description: 1, + full_schema: 1, + column_batch: 1, + business_term: 1, + prior_question_sql: 1, + project_metadata: 1 + }) + ); + expect(result.ingestionSources).toHaveLength(6); + expect(result.ingestionSources.map((source) => source.sourceType)).toEqual( + expect.arrayContaining(["semantic_asset", "semantic_term", "sql_example"]) + ); + + const documentFactory = new RagDocumentFactory(); + const documentBuilds = result.ingestionSources.map((source) => + documentFactory.create(source) + ); + const metadata = documentBuilds.map((build) => + JSON.parse(build.chunks[0]?.metadata ?? "{}") + ); + expect(metadata).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + assetFamily: "table_description", + manifestFingerprint: result.manifest.fingerprint, + visibilityScope: "datasource" + }), + expect.objectContaining({ + assetFamily: "prior_question_sql", + trusted: true, + verified: true, + viewStatus: "active" + }) + ]) + ); + }); + + it("degrades only modeling relationship assets when payload or revision is missing", () => { + const result = preparer.prepare({ + datasourceId: "ds-main", + workspaceId: "ws-main", + embeddingProfile, + sourceSnapshots: [ + { + family: "relationship_binding", + sourceKind: "modeling_graph", + sourceRef: { type: "modeling_graph", ref: "relationships" }, + sourceVersion: "modeling-v1", + sourceHash: "hash-relationships" + }, + { + family: "metric", + sourceKind: "modeling_graph", + sourceRef: { type: "modeling_graph", ref: "metric.gmv" }, + sourceVersion: "modeling-v1", + sourceHash: "hash-metric", + modelingRevision: 7, + title: "GMV", + summary: { expression: "SUM(orders.amount)" } + } + ] + }); + + const relationship = result.manifest.entries.find( + (entry) => entry.family === "relationship_binding" + ); + const metric = result.manifest.entries.find((entry) => entry.family === "metric"); + + expect(relationship?.status).toBe("degraded"); + expect(relationship?.reasonCodes).toEqual( + expect.arrayContaining([ + SEMANTIC_ASSET_REASON_CODES.degradedMissingRelationshipPayload, + SEMANTIC_ASSET_REASON_CODES.degradedMissingModelingRevision + ]) + ); + expect(metric?.status).toBe("prepared"); + expect(result.ingestionSources).toHaveLength(1); + }); + + it("excludes non-promoted correction feedback from durable manifest entries", () => { + const result = preparer.prepare({ + datasourceId: "ds-main", + workspaceId: "ws-main", + embeddingProfile, + sourceSnapshots: [ + { + sourceKind: "correction_feedback", + sourceRef: { type: "correction_feedback", ref: "run-1" }, + sourceVersion: "run-1", + sourceHash: "hash-correction", + question: "fix SQL", + sql: "SELECT 1", + promoted: false + } + ] + }); + + expect(result.manifest.entries).toHaveLength(0); + expect(result.ingestionSources).toHaveLength(0); + expect(result.excludedSnapshots).toHaveLength(1); + expect(result.excludedSnapshots[0]?.reasonCodes).toContain( + SEMANTIC_ASSET_REASON_CODES.skippedCorrectionNotPromoted + ); + }); +}); diff --git a/apps/backend/test/unit/semantic-asset-readiness.service.spec.ts b/apps/backend/test/unit/semantic-asset-readiness.service.spec.ts new file mode 100644 index 0000000..920b78e --- /dev/null +++ b/apps/backend/test/unit/semantic-asset-readiness.service.spec.ts @@ -0,0 +1,143 @@ +import type { RagIndexVersionRecord } from "../../src/modules/rag/index/rag-index.repository"; +import type { SemanticAssetManifestSummary } from "../../src/modules/knowledge/rag/preparation/semantic-asset-manifest.types"; +import { SemanticAssetReadinessService } from "../../src/modules/knowledge/rag/preparation/semantic-asset-readiness.service"; + +function buildManifestSummary( + overrides: Partial = {} +): SemanticAssetManifestSummary { + return { + manifestVersion: "semantic-asset-manifest/v1", + fingerprint: "fingerprint-a", + datasourceId: "ds-main", + workspaceId: "ws-main", + embeddingProfile: { + provider: "mock", + model: "mock-embedding", + dimensions: 8, + vectorVersion: "v1", + configSource: "settings" + }, + sourceSnapshotSummary: { + triggers: ["schema"], + sourceVersion: "snapshot-v1" + }, + familyCounts: { + table_description: 1 + }, + reasonCodes: [], + entryCount: 1, + preparedEntryCount: 1, + skippedEntryCount: 0, + degradedEntryCount: 0, + ...overrides + }; +} + +function buildActiveVersion( + datasourceId: string, + sourceVersion: string +): RagIndexVersionRecord { + return { + id: `idx-${datasourceId}`, + datasourceId, + status: "active", + sourceVersion, + activatedAt: "2026-04-30T00:00:00.000Z", + createdAt: "2026-04-30T00:00:00.000Z", + updatedAt: "2026-04-30T00:00:00.000Z" + }; +} + +describe("SemanticAssetReadinessService", () => { + it("returns empty missing_active summary when no datasource context exists", async () => { + const getActiveVersion = jest.fn().mockResolvedValue(undefined); + const service = new SemanticAssetReadinessService({ + getActiveVersion + } as never); + + const summary = await service.snapshot(); + + expect(summary.status).toBe("missing_active"); + expect(summary.rebuildable).toBe(false); + expect(summary.familyCounts).toEqual({}); + expect(getActiveVersion).not.toHaveBeenCalled(); + }); + + it("marks readiness degraded and appends active_index_changed when active source drifts", async () => { + const activeVersions: Record = { + "ds-main": buildActiveVersion("ds-main", "fingerprint-b:source-v2") + }; + const getActiveVersion = jest + .fn() + .mockImplementation(async (datasourceId: string) => activeVersions[datasourceId]); + const service = new SemanticAssetReadinessService({ + getActiveVersion + } as never); + + const activation = service.recordActivation({ + datasourceId: " ds-main ", + workspaceId: "ws-main", + runId: "run-1", + sourceVersion: "fingerprint-a:source-v1", + indexVersionId: "idx-old", + manifestSummary: buildManifestSummary({ + reasonCodes: ["degraded_missing_source_hash"], + degradedEntryCount: 1 + }), + staleReasons: ["manual_flag", "manual_flag", " "] + }); + expect(activation.status).toBe("degraded"); + expect(activation.degradedFamilies).toContain("table_description"); + + const snapshot = await service.snapshot(); + + expect(snapshot.datasourceId).toBe("ds-main"); + expect(snapshot.activeIndexVersionId).toBe("idx-ds-main"); + expect(snapshot.activeSourceVersion).toBe("fingerprint-b:source-v2"); + expect(snapshot.status).toBe("degraded"); + expect(snapshot.staleReasons).toEqual(["active_index_changed", "manual_flag"]); + expect(snapshot.rebuildable).toBe(true); + }); + + it("returns missing_active with no_active_index when activation exists but no active version", async () => { + const getActiveVersion = jest.fn().mockResolvedValue(undefined); + const service = new SemanticAssetReadinessService({ + getActiveVersion + } as never); + + service.recordActivation({ + datasourceId: "ds-no-active", + sourceVersion: "fingerprint-c:source-v1", + indexVersionId: "idx-c", + manifestSummary: buildManifestSummary({ + datasourceId: "ds-no-active" + }) + }); + + const snapshot = await service.snapshot({ datasourceId: "ds-no-active" }); + + expect(snapshot.status).toBe("missing_active"); + expect(snapshot.staleReasons).toContain("no_active_index"); + expect(snapshot.rebuildable).toBe(true); + }); + + it("builds ready snapshot from active version even before any activation record exists", async () => { + const activeVersions: Record = { + "ds-fresh": buildActiveVersion("ds-fresh", "fingerprint-fresh:source-v1") + }; + const getActiveVersion = jest + .fn() + .mockImplementation(async (datasourceId: string) => activeVersions[datasourceId]); + const service = new SemanticAssetReadinessService({ + getActiveVersion + } as never); + + const snapshot = await service.snapshot({ datasourceId: "ds-fresh" }); + + expect(snapshot.status).toBe("ready"); + expect(snapshot.datasourceId).toBe("ds-fresh"); + expect(snapshot.activeIndexVersionId).toBe("idx-ds-fresh"); + expect(snapshot.activeManifestFingerprint).toBe("fingerprint-fresh"); + expect(snapshot.lastActivation?.indexVersionId).toBe("idx-ds-fresh"); + }); +}); diff --git a/apps/backend/test/unit/sql-generation.service.spec.ts b/apps/backend/test/unit/sql-generation.service.spec.ts index 26b9419..d3bf3fd 100644 --- a/apps/backend/test/unit/sql-generation.service.spec.ts +++ b/apps/backend/test/unit/sql-generation.service.spec.ts @@ -157,6 +157,47 @@ describe("SqlGenerationService semantic guardrails", () => { expect(providerRouter.generate).toHaveBeenCalledTimes(1); }); + it("forwards abortSignal into providerRouter.stream", async () => { + const providerRouter = { + generate: jest.fn(), + stream: jest.fn().mockResolvedValue({ + provider: "mock-provider", + model: "mock-model", + rawText: "```sql\nSELECT id, status FROM orders LIMIT 20;\n```" + }) + }; + const service = createService(providerRouter); + const abortController = new AbortController(); + + await service.stream( + "列出订单及其支付状态", + { + datasourceType: "sqlite", + semanticPlan: { + route: "answer", + standaloneQuestion: "列出订单及其支付状态", + selectedTables: ["orders", "payments"], + selectedColumns: ["orders.id", "orders.status", "payments.method"], + metrics: ["list"], + filters: ["route_kind:text_to_sql"], + evidenceRefs: ["schema-supplement:orders", "schema-supplement:payments"], + confidence: 0.62 + } + }, + { + abortSignal: abortController.signal + } + ); + + expect(providerRouter.stream).toHaveBeenCalledWith( + expect.any(Object), + expect.any(Object), + expect.objectContaining({ + abortSignal: abortController.signal + }) + ); + }); + it("uses a semantic shortcut for grouped count proportions before calling the stream provider", async () => { const providerRouter = { generate: jest.fn(), @@ -268,6 +309,150 @@ describe("SqlGenerationService semantic guardrails", () => { expect(providerRouter.generate).not.toHaveBeenCalled(); }); + it("uses an amount metric shortcut for recent sales instead of a count shortcut", async () => { + const providerRouter = { + generate: jest.fn(), + stream: jest.fn().mockRejectedValue( + new DomainError( + "LLM_REQUEST_FAILED", + "LLM 流式请求失败: The operation was aborted due to timeout", + 502 + ) + ) + }; + const service = createService(providerRouter); + + const draft = await service.stream("最近的销售额", { + datasourceType: "sqlite", + semanticPlan: { + route: "answer", + standaloneQuestion: "最近的销售额", + selectedTables: ["orders"], + selectedColumns: [ + "orders.id", + "orders.total_amount", + "orders.created_at", + "orders.paid_at" + ], + metrics: ["orders.total_amount"], + filters: ["route_kind:text_to_sql"], + evidenceRefs: ["schema-supplement:orders"], + confidence: 0.49 + }, + selectedContext: [ + { + chunk_id: "schema-supplement:orders", + content: "orders table schema", + metadata: { + datasourceId: "ds-1", + indexVersionId: "schema-supplement", + chunkId: "schema-supplement:orders", + domain: "schema", + tableNames: ["orders"], + columnNames: [ + "orders.id", + "orders.total_amount", + "orders.created_at", + "orders.paid_at" + ], + sourceMetadata: {} + } + } + ] + }); + + expect(draft.provider).toBe("semantic-shortcut"); + expect(draft.model).toBe("amount-metric-sum-v1"); + expect(draft.sql).toContain("SUM(orders.total_amount)"); + expect(draft.sql).toContain("FROM orders"); + expect(draft.sql).toContain("COALESCE(orders.paid_at, orders.created_at)"); + expect(draft.sql).not.toContain("COUNT(*)"); + expect(providerRouter.stream).not.toHaveBeenCalled(); + expect(providerRouter.generate).not.toHaveBeenCalled(); + }); + + it("keeps simple count shortcuts when count questions include amount columns in evidence", async () => { + const providerRouter = { + generate: jest.fn(), + stream: jest.fn().mockRejectedValue( + new DomainError( + "LLM_REQUEST_FAILED", + "LLM 流式请求失败: The operation was aborted due to timeout", + 502 + ) + ) + }; + const service = createService(providerRouter); + + const draft = await service.stream("一共有多少订单", { + datasourceType: "sqlite", + semanticPlan: { + route: "answer", + standaloneQuestion: "一共有多少订单", + selectedTables: ["orders"], + selectedColumns: ["orders.id", "orders.total_amount"], + metrics: ["count"], + filters: ["route_kind:text_to_sql"], + evidenceRefs: ["schema-supplement:orders"], + confidence: 0.62 + } + }); + + expect(draft.provider).toBe("semantic-shortcut"); + expect(draft.model).toBe("simple-count-v1"); + expect(draft.sql).toBe("SELECT COUNT(*) AS total_count FROM orders;"); + expect(providerRouter.stream).not.toHaveBeenCalled(); + expect(providerRouter.generate).not.toHaveBeenCalled(); + }); + + it("keeps simple count shortcuts when semantic metrics include amount columns from broad schema evidence", async () => { + const providerRouter = { + generate: jest.fn(), + stream: jest.fn().mockRejectedValue( + new DomainError( + "LLM_REQUEST_FAILED", + "LLM 流式请求失败: Invalid JSON response", + 502 + ) + ) + }; + const service = createService(providerRouter); + + const draft = await service.stream("一共有多少订单", { + datasourceType: "sqlite", + semanticPlan: { + route: "answer", + standaloneQuestion: "一共有多少订单", + selectedTables: ["orders"], + selectedColumns: [ + "orders.id", + "orders.subtotal_amount", + "orders.discount_amount", + "orders.shipping_amount", + "orders.tax_amount", + "orders.total_amount" + ], + metrics: [ + "count", + "orders.subtotal_amount", + "orders.discount_amount", + "orders.shipping_amount", + "orders.tax_amount", + "orders.total_amount" + ], + filters: ["route_kind:text_to_sql"], + evidenceRefs: ["schema-supplement:orders"], + confidence: 0.49 + } + }); + + expect(draft.provider).toBe("semantic-shortcut"); + expect(draft.model).toBe("simple-count-v1"); + expect(draft.sql).toBe("SELECT COUNT(*) AS total_count FROM orders;"); + expect(providerRouter.stream).not.toHaveBeenCalled(); + expect(providerRouter.generate).not.toHaveBeenCalled(); + }); + it("prioritizes structured semantic instructions when context pack is provided", async () => { const providerRouter = { generate: jest.fn().mockResolvedValue({ diff --git a/apps/backend/test/unit/sql-prompt.builder.spec.ts b/apps/backend/test/unit/sql-prompt.builder.spec.ts index ba5736a..64cdf1d 100644 --- a/apps/backend/test/unit/sql-prompt.builder.spec.ts +++ b/apps/backend/test/unit/sql-prompt.builder.spec.ts @@ -112,6 +112,32 @@ describe("SqlPromptBuilder", () => { ); }); + it("renders selected context asset provenance in prompt snippets", () => { + const builder = new SqlPromptBuilder(); + const prompt = builder.build("统计订单 GMV", "sqlite", [ + { + chunk_id: "chunk-orders-description", + content: "Orders table stores paid order facts.", + metadata: { + datasourceId: "ds-1", + indexVersionId: "idx-1", + chunkId: "chunk-orders-description", + domain: "semantic_asset", + assetFamily: "table_description", + manifestFingerprint: "semantic-assets-prompt-v1", + sourceVersion: "schema-v1", + tableNames: ["orders"], + columnNames: [], + sourceMetadata: {} + } + } + ]); + + expect(prompt.userPrompt).toContain( + "[semantic_asset | family=table_description | manifest=semantic-assets-prompt-v1 | sourceVersion=schema-v1]" + ); + }); + it("renders semantic-plan guardrails including coverage gaps and snapshot id", () => { const builder = new SqlPromptBuilder(); const prompt = builder.build("统计订单 GMV", "sqlite", undefined, { diff --git a/apps/backend/test/unit/text2sql-stream-event.mapper.spec.ts b/apps/backend/test/unit/text2sql-stream-event.mapper.spec.ts index 2f3367a..55c1637 100644 --- a/apps/backend/test/unit/text2sql-stream-event.mapper.spec.ts +++ b/apps/backend/test/unit/text2sql-stream-event.mapper.spec.ts @@ -1,3 +1,4 @@ +import { createChatStreamEventEnvelope } from "@text2sql/chat-stream-protocol"; import { Text2SqlStreamEventMapper } from "../../src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper"; describe("Text2SqlStreamEventMapper", () => { @@ -118,6 +119,11 @@ describe("Text2SqlStreamEventMapper", () => { input: { sql: "SELECT 1" } }); expect(called.type).toBe("tool-call"); + expect(called.data).toMatchObject({ + title: "调用 runReadOnlySql", + stage: "generation", + summary: "SELECT 1" + }); expect(called.traceToolCall?.status).toBe("called"); const result = mapper.mapLlmEvent({ @@ -127,6 +133,11 @@ describe("Text2SqlStreamEventMapper", () => { output: { rows: [] } }); expect(result.type).toBe("tool-result"); + expect(result.data).toMatchObject({ + title: "读取 runReadOnlySql 的结果", + stage: "generation", + summary: "返回 0 行" + }); expect(result.traceToolCall?.status).toBe("result"); const error = mapper.mapLlmEvent({ @@ -136,11 +147,16 @@ describe("Text2SqlStreamEventMapper", () => { message: "failed" }); expect(error.type).toBe("tool-error"); + expect(error.data).toMatchObject({ + title: "runReadOnlySql 调用失败", + stage: "generation", + summary: "failed" + }); expect(error.traceToolCall?.status).toBe("error"); }); - it("builds stream envelope with required fields", () => { - const envelope = mapper.createEnvelope({ + it("builds stream envelope with required fields via toolkit helper", () => { + const envelope = createChatStreamEventEnvelope({ type: "start", data: { requestId: "req-1" @@ -180,7 +196,7 @@ describe("Text2SqlStreamEventMapper", () => { runId: "run-v2-shell", lastSequence: 2 }); - const envelope = mapper.createEnvelope({ + const envelope = createChatStreamEventEnvelope({ type: "state", runId: "run-v2-shell", sessionId: "session-v2-shell", @@ -271,7 +287,7 @@ describe("Text2SqlStreamEventMapper", () => { runId: "run-v2-progress", lastSequence: 7 }); - const envelope = mapper.createEnvelope({ + const envelope = createChatStreamEventEnvelope({ type: "state", runId: "run-v2-progress", sessionId: "session-v2-progress", diff --git a/apps/backend/test/unit/text2sql-v2-validation.spec.ts b/apps/backend/test/unit/text2sql-v2-validation.spec.ts index 477941d..ad986d3 100644 --- a/apps/backend/test/unit/text2sql-v2-validation.spec.ts +++ b/apps/backend/test/unit/text2sql-v2-validation.spec.ts @@ -1,4 +1,5 @@ import { SqlValidationService } from "../../src/modules/conversation/adapters/sql-validation.service"; +import { DomainError } from "../../src/common/domain-error"; describe("text2sql v2 sql validation", () => { const createService = (overrides?: { @@ -8,6 +9,7 @@ describe("text2sql v2 sql validation", () => { dryRunReason?: string; dryPlanPass?: boolean; dryPlanReason?: string; + sqliteDryRunError?: DomainError; }) => new SqlValidationService( { @@ -28,6 +30,18 @@ describe("text2sql v2 sql validation", () => { missingTables: overrides?.dryPlanPass === false ? ["customers"] : [], reason: overrides?.dryPlanReason })) + } as never, + { + getDatasourceById: jest.fn().mockResolvedValue({ + id: "sqlite_main", + type: "sqlite" + }) + } as never, + { + dbPath: "/tmp/text2sql.db", + dryRun: overrides?.sqliteDryRunError + ? jest.fn().mockRejectedValue(overrides.sqliteDryRunError) + : jest.fn().mockResolvedValue(undefined) } as never ); @@ -374,6 +388,38 @@ describe("text2sql v2 sql validation", () => { expect(result.failure?.terminal).toBe(false); }); + it("fails sqlite dry-run missing columns as correctable validation errors", async () => { + const result = await createService({ + sqliteDryRunError: new DomainError( + "SQL_MISSING_COLUMN", + "missing column order_date", + 400, + { + originalMessage: "Error: in prepare, no such column: order_date" + } + ) + }).validate({ + sql: "SELECT SUM(total_amount) FROM orders WHERE order_date >= DATE('now', '-30 days')", + datasourceId: "sqlite_main", + datasourceType: "sqlite", + semanticPlan: plan({ selectedColumns: [] }) + }); + + expect(result.status).toBe("failed"); + expect(result.correctable).toBe(true); + expect(result.failure).toMatchObject({ + code: "SQL_MISSING_COLUMN", + message: "missing column order_date", + terminal: false, + correctable: true + }); + expect(result.checks.find((check) => check.check === "dry-run")).toMatchObject({ + status: "failed", + code: "SQL_MISSING_COLUMN", + message: "missing column order_date" + }); + }); + it("records dry-plan unsupported as explicit skipped warning", async () => { const result = await createService({ dryPlan: false }).validate({ sql: "SELECT COUNT(*) FROM orders", diff --git a/apps/backend/test/unit/text2sql-workflow-runner.spec.ts b/apps/backend/test/unit/text2sql-workflow-runner.spec.ts index c5afcbc..cc01ecf 100644 --- a/apps/backend/test/unit/text2sql-workflow-runner.spec.ts +++ b/apps/backend/test/unit/text2sql-workflow-runner.spec.ts @@ -71,7 +71,6 @@ describe("Text2SQLWorkflowRunner", () => { run: jest.fn().mockResolvedValue(undefined) }; const streamEventMapper = { - createEnvelope: jest.fn(), mapLlmEvent: jest.fn(), mapStepEvent: jest.fn() }; @@ -123,7 +122,6 @@ describe("Text2SQLWorkflowRunner", () => { { run: jest.fn().mockResolvedValue(undefined) } as never, { run: jest.fn().mockResolvedValue(undefined) } as never, { - createEnvelope: jest.fn(), mapLlmEvent: jest.fn(), mapStepEvent: jest.fn() } as never @@ -168,7 +166,6 @@ describe("Text2SQLWorkflowRunner", () => { { run: jest.fn().mockResolvedValue(undefined) } as never, { run: jest.fn().mockResolvedValue(undefined) } as never, { - createEnvelope: jest.fn(), mapLlmEvent: jest.fn(), mapStepEvent: jest.fn() } as never @@ -207,13 +204,6 @@ describe("Text2SQLWorkflowRunner", () => { }; const streamEventMapper = { - createEnvelope: jest.fn(({ type, data, runId, sessionId }) => ({ - type, - data, - runId, - sessionId, - at: "2026-04-26T00:00:02.000Z" - })), mapLlmEvent: jest.fn(() => ({ type: "text-delta", data: { @@ -350,13 +340,6 @@ describe("Text2SQLWorkflowRunner", () => { { run: jest.fn().mockResolvedValue(undefined) } as never, { run: jest.fn().mockResolvedValue(undefined) } as never, { - createEnvelope: jest.fn(({ type, data, runId, sessionId }) => ({ - type, - data, - runId, - sessionId, - at: "2026-04-26T00:00:02.000Z" - })), mapLlmEvent: jest.fn(), mapStepEvent: jest.fn(() => ({ data: { @@ -439,13 +422,6 @@ describe("Text2SQLWorkflowRunner", () => { run: jest.fn().mockResolvedValue(undefined) } as never, { - createEnvelope: jest.fn(({ type, data, runId, sessionId }) => ({ - type, - data, - runId, - sessionId, - at: "2026-04-26T00:00:02.000Z" - })), mapLlmEvent: jest.fn(), mapStepEvent: jest.fn() } as never @@ -467,4 +443,78 @@ describe("Text2SQLWorkflowRunner", () => { expect(events.map((item) => item.type)).toEqual(["start", "error"]); expect(events[1]?.code).toBe("GRAPH_FAILED"); }); + + it("passes abortSignal into the v2 stream stage", async () => { + const prepared = createPrepared(); + const runAgentGraphStage = { + runSync: jest.fn(), + runStream: jest.fn().mockResolvedValue(createRun()) + }; + const runner = new Text2SQLWorkflowRunner( + { shouldFallbackOnReject: jest.fn().mockReturnValue(false) } as never, + { run: jest.fn().mockResolvedValue(prepared) } as never, + runAgentGraphStage as never, + { run: jest.fn(async (run) => run) } as never, + { run: jest.fn().mockResolvedValue(undefined) } as never, + { run: jest.fn().mockResolvedValue(undefined) } as never, + { + mapLlmEvent: jest.fn(), + mapStepEvent: jest.fn() + } as never + ); + const abortController = new AbortController(); + + await runner.runStream({ + sessionId: "session-1", + message: "统计订单总数", + abortSignal: abortController.signal, + onEvent: jest.fn() + }); + + expect(runAgentGraphStage.runStream).toHaveBeenCalledWith( + prepared, + "/api/v1/sessions/:sessionId/messages/stream", + expect.objectContaining({ + abortSignal: abortController.signal + }) + ); + }); + + it("emits USER_CANCELLED and persists a failed run on cancellation", async () => { + const prepared = createPrepared(); + const runner = new Text2SQLWorkflowRunner( + { shouldFallbackOnReject: jest.fn().mockReturnValue(false) } as never, + { run: jest.fn().mockResolvedValue(prepared) } as never, + { + runSync: jest.fn(), + runStream: jest.fn().mockRejectedValue( + new DomainError("USER_CANCELLED", "用户已停止本轮生成。", 499) + ) + } as never, + { run: jest.fn(async (run) => run) } as never, + { run: jest.fn().mockResolvedValue(undefined) } as never, + { run: jest.fn().mockResolvedValue(undefined) } as never, + { + mapLlmEvent: jest.fn(), + mapStepEvent: jest.fn() + } as never + ); + + const events: Array<{ type: string; code?: string }> = []; + const run = await runner.runStream({ + sessionId: "session-1", + message: "统计订单总数", + onEvent: (event) => { + events.push({ + type: event.type, + code: (event.data as { code?: string }).code + }); + } + }); + + expect(run.status).toBe("failed"); + expect(run.error).toBe("用户已停止本轮生成。"); + expect(events.map((item) => item.type)).toEqual(["start", "error"]); + expect(events[1]?.code).toBe("USER_CANCELLED"); + }); }); diff --git a/apps/frontend/package.json b/apps/frontend/package.json index e544e79..c8cf5e0 100644 --- a/apps/frontend/package.json +++ b/apps/frontend/package.json @@ -3,19 +3,20 @@ "version": "0.1.0", "private": true, "scripts": { - "predev": "pnpm --filter @text2sql/shared-types build", + "predev": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "dev": "next dev -p 3001", - "prebuild": "pnpm --filter @text2sql/shared-types build", + "prebuild": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "build": "next build", "start": "next start -p 3001", - "pretest": "pnpm --filter @text2sql/shared-types build", + "pretest": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "test": "vitest run", - "prelint": "pnpm --filter @text2sql/shared-types build", + "prelint": "pnpm --filter @text2sql/shared-types build && pnpm --filter @text2sql/chat-stream-protocol build", "lint": "eslint . --max-warnings 0 && tsc -p tsconfig.json --noEmit" }, "dependencies": { "@assistant-ui/react": "^0.12.24", "@base-ui/react": "^1.3.0", + "@text2sql/chat-stream-protocol": "workspace:*", "@text2sql/shared-types": "workspace:*", "@xyflow/react": "^12.9.3", "class-variance-authority": "^0.7.1", diff --git a/apps/frontend/src/components/chat-panel.tsx b/apps/frontend/src/components/chat-panel.tsx index 1614c72..8b8ce3f 100644 --- a/apps/frontend/src/components/chat-panel.tsx +++ b/apps/frontend/src/components/chat-panel.tsx @@ -4,11 +4,8 @@ import { useEffect, useState } from "react"; import type { ChatSessionView, ChatMessage, - ChatStreamEvent, DeliveryContract, - ExecutionTraceStep, ModelCatalogItem, - ReasoningStage, Session, SqlRun } from "@text2sql/shared-types"; @@ -19,9 +16,11 @@ import { mergeRunThinkingSteps, normalizeDeliveryContract, normalizeRunForVisibility, + projectStreamEvent, toRunVisibilityStatusFromRunStatus, transitionRunVisibilityStatus, - type RunVisibilityStatus + type RunVisibilityStatus, + type RunVisibilityThinkingStep } from "@/components/chat/run-visibility-mapper"; import { ModelSelector } from "@/components/chat/model-selector"; import { SessionSidebar } from "@/components/chat/session-sidebar"; @@ -47,28 +46,7 @@ import { writeActiveDatasourceId } from "@/lib/datasource-session-context"; -interface ThinkingStateEventData { - node: string; - status: ExecutionTraceStep["status"]; - stepId?: string; - sequence?: number; - lifecycle?: ExecutionTraceStep["lifecycle"]; - detail: string; - stage?: ReasoningStage; - title?: string; - at?: string; - startedAt?: string; - endedAt?: string; - durationMs?: number; - inputSummary?: string; - outputSummary?: string; - errorSummary?: string; -} - -type ThinkingStep = ExecutionTraceStep & { - stage?: ReasoningStage; - title?: string; -}; +type ThinkingStep = RunVisibilityThinkingStep; const DATASOURCE_READONLY_STATUSES = new Set(["unavailable", "deleted"]); const SESSION_ERROR_CODES = new Set(["SESSION_NOT_FOUND", "VALIDATION_ERROR"]); @@ -117,106 +95,6 @@ function moveSessionToFront( return [target, ...next]; } -function toThinkingStep(event: ChatStreamEvent): ThinkingStep | null { - if (event.type === "tool-call") { - const payload = event.data as - | { toolName?: string; toolCallId?: string; input?: unknown } - | undefined; - const toolName = payload?.toolName ?? "tool"; - return { - node: `tool:${toolName}`, - status: "success", - stepId: `${event.runId}:tool:${payload?.toolCallId ?? event.at}`, - lifecycle: "running", - detail: summarizeToolPayload(payload?.input), - at: event.at, - startedAt: event.at, - stage: "generation", - title: `调用工具:${toolName}` - }; - } - if (event.type === "tool-result") { - const payload = event.data as - | { toolName?: string; toolCallId?: string; output?: unknown } - | undefined; - const toolName = payload?.toolName ?? "tool"; - return { - node: `tool:${toolName}`, - status: "success", - stepId: `${event.runId}:tool:${payload?.toolCallId ?? event.at}`, - lifecycle: "completed", - detail: summarizeToolPayload(payload?.output), - at: event.at, - endedAt: event.at, - stage: "generation", - title: `工具返回:${toolName}` - }; - } - if (event.type === "tool-error") { - const payload = event.data as - | { toolName?: string; toolCallId?: string; message?: string } - | undefined; - const toolName = payload?.toolName ?? "tool"; - return { - node: `tool:${toolName}`, - status: "failed", - stepId: `${event.runId}:tool:${payload?.toolCallId ?? event.at}`, - lifecycle: "failed", - detail: payload?.message ?? "工具调用失败", - errorSummary: payload?.message, - at: event.at, - endedAt: event.at, - stage: "generation", - title: `工具失败:${toolName}` - }; - } - if (event.type !== "state") { - return null; - } - const payload = event.data as ThinkingStateEventData; - return { - node: payload.node, - status: payload.status, - stepId: payload.stepId, - sequence: payload.sequence, - lifecycle: payload.lifecycle, - detail: payload.detail, - at: payload.at ?? event.at, - startedAt: payload.startedAt, - endedAt: payload.endedAt, - durationMs: payload.durationMs, - inputSummary: payload.inputSummary, - outputSummary: payload.outputSummary, - errorSummary: payload.errorSummary, - stage: payload.stage, - title: payload.title - }; -} - -function summarizeToolPayload(payload: unknown): string { - if (payload === undefined || payload === null) { - return ""; - } - if (typeof payload === "string") { - return payload.slice(0, 180); - } - if (typeof payload !== "object") { - return String(payload).slice(0, 180); - } - const record = payload as Record; - if (typeof record.sql === "string") { - return record.sql.slice(0, 180); - } - if (typeof record.rowCount === "number") { - return `返回 ${record.rowCount} 行`; - } - try { - return JSON.stringify(payload).slice(0, 180); - } catch { - return ""; - } -} - function appendThinkingStep( previous: Record, runId: string, @@ -255,6 +133,9 @@ function mergeSessionMessages( } export function ChatPanel() { + const [activeAbortController, setActiveAbortController] = useState( + null + ); const [writableSessions, setWritableSessions] = useState([]); const [readonlySessions, setReadonlySessions] = useState([]); const [datasourceId, setDatasourceId] = useState(""); @@ -349,6 +230,7 @@ export function ChatPanel() { setStreamTextStartedByRunId({}); setRunVisibilityByRunId({}); setActiveStreamRunId(null); + setActiveAbortController(null); setThinkingRequestPending(false); setSaveNotice(""); setThreadVersion((previous) => previous + 1); @@ -796,14 +678,28 @@ export function ChatPanel() { runVisibilityByRunId={runVisibilityByRunId} activeStreamRunId={activeStreamRunId} thinkingRequestPending={thinkingRequestPending} - debugEnabled={Boolean(activeSession?.debugEnabled)} + activeAbortController={activeAbortController} disabled={ sessionLoading || !sessionId || activeSession?.datasourceStatus === "unavailable" || activeSession?.datasourceStatus === "deleted" } + onStopStream={() => { + activeAbortController?.abort("user_stop"); + setActiveAbortController(null); + setThinkingRequestPending(false); + if (activeStreamRunId) { + setRunVisibilityByRunId((previous) => ({ + ...previous, + [activeStreamRunId]: + transitionRunVisibilityStatus(previous[activeStreamRunId], "cancelled") ?? + "cancelled" + })); + } + }} onRequestRun={ensureRunLoaded} + onAbortControllerChange={setActiveAbortController} onRunStart={() => { setActiveStreamRunId(null); setThinkingRequestPending(true); @@ -834,53 +730,36 @@ export function ChatPanel() { setThinkingRequestPending(true); return; } - if (event.type === "text-delta") { - const text = (event.data as { text?: unknown } | undefined)?.text; - if (typeof text === "string" && text.length > 0) { - setStreamTextStartedByRunId((previous) => ({ - ...previous, - [event.runId]: true - })); - } + const projection = projectStreamEvent(event); + if (projection.textStarted) { + setStreamTextStartedByRunId((previous) => ({ + ...previous, + [event.runId]: true + })); } - if (event.type === "finish") { - const finishData = event.data as - | { delivery?: unknown; status?: unknown } - | undefined; - const finishDelivery = normalizeDeliveryContract(finishData?.delivery); - if (finishDelivery) { + if (projection.deliveryPatch) { + const delivery = normalizeDeliveryContract(projection.deliveryPatch); + if (delivery) { setStreamDeliveryByRunId((previous) => ({ ...previous, - [event.runId]: finishDelivery - })); - } - const finishStatus = toRunVisibilityStatusFromRunStatus( - typeof finishData?.status === "string" - ? (finishData.status as SqlRun["status"]) - : "executionResult" - ); - if (finishStatus) { - setRunVisibilityByRunId((previous) => ({ - ...previous, - [event.runId]: - transitionRunVisibilityStatus( - previous[event.runId], - finishStatus - ) ?? finishStatus + [event.runId]: delivery })); } } - const step = toThinkingStep(event); - if (!step) { - if (event.type === "finish" || event.type === "error") { - if (event.type === "error") { - setRunVisibilityByRunId((previous) => ({ - ...previous, - [event.runId]: - transitionRunVisibilityStatus(previous[event.runId], "error") ?? - "error" - })); - } + if (projection.visibilityStatus) { + const visibilityStatus = projection.visibilityStatus; + setRunVisibilityByRunId((previous) => ({ + ...previous, + [event.runId]: + transitionRunVisibilityStatus( + previous[event.runId], + visibilityStatus + ) ?? visibilityStatus + })); + } + const thinkingStep = projection.thinkingStep; + if (!thinkingStep) { + if (projection.terminal) { setActiveStreamRunId((current) => current === event.runId ? null : current ); @@ -889,10 +768,11 @@ export function ChatPanel() { return; } setStreamThinkingByRunId((previous) => - appendThinkingStep(previous, event.runId, step) + appendThinkingStep(previous, event.runId, thinkingStep) ); }} onRunFinish={async (runId) => { + setActiveAbortController(null); setActiveStreamRunId((current) => (current === runId ? null : current)); setThinkingRequestPending(false); if (!sessionId) { @@ -904,7 +784,21 @@ export function ChatPanel() { } await refreshSessionBuckets(); }} + onRunCancelled={async (runId) => { + setActiveAbortController(null); + setActiveStreamRunId((current) => (current === runId ? null : current)); + setThinkingRequestPending(false); + if (runId) { + setRunVisibilityByRunId((previous) => ({ + ...previous, + [runId]: + transitionRunVisibilityStatus(previous[runId], "cancelled") ?? + "cancelled" + })); + } + }} onRunError={async (error) => { + setActiveAbortController(null); setActiveStreamRunId(null); setThinkingRequestPending(false); setSessionError(error.message || "消息发送失败,请稍后重试。"); diff --git a/apps/frontend/src/components/chat/assistant-composer.tsx b/apps/frontend/src/components/chat/assistant-composer.tsx index 9b0494d..cde86b5 100644 --- a/apps/frontend/src/components/chat/assistant-composer.tsx +++ b/apps/frontend/src/components/chat/assistant-composer.tsx @@ -2,44 +2,26 @@ import { useRef } from "react"; import { ComposerPrimitive } from "@assistant-ui/react"; -import { Play, Terminal } from "lucide-react"; -import { - ContextEnvelopePanel, - type ContextEnvelopeDraft -} from "@/components/chat/context-envelope-panel"; +import { Play, Square } from "lucide-react"; import { Button } from "@/components/ui/button"; import { cn } from "@/lib/utils"; interface AssistantComposerProps { disabled?: boolean; - onOpenDetail?: () => void; - contextEnvelopeDraft: ContextEnvelopeDraft; - clearContextEnvelopeAfterSend: boolean; - onContextEnvelopeDraftChange: (next: ContextEnvelopeDraft) => void; - onClearContextEnvelopeAfterSendChange: (next: boolean) => void; + showStopButton?: boolean; + onStop?: () => void; } export function AssistantComposer({ disabled = false, - onOpenDetail, - contextEnvelopeDraft, - clearContextEnvelopeAfterSend, - onContextEnvelopeDraftChange, - onClearContextEnvelopeAfterSendChange + showStopButton = false, + onStop }: AssistantComposerProps) { const imeSubmitPending = useRef(false); return (
- -

Shift + Enter 换行,Enter 发送

- {onOpenDetail ? ( + {showStopButton ? ( ) : null} -
+ +
@@ -45,51 +43,52 @@ interface AssistantMessageBubbleProps { run: SqlRun | null; streamDelivery?: DeliveryContract; runId?: string; - debugEnabled: boolean; thinkingSteps: Array; thinkingInProgress: boolean; + thinkingCancelled?: boolean; runLoading?: boolean; onRequestRun?: () => void; openSqlSignal?: number; - highlightSql?: boolean; } export function AssistantMessageBubble({ run, streamDelivery, runId, - debugEnabled, thinkingSteps, thinkingInProgress, + thinkingCancelled = false, runLoading = false, onRequestRun, - openSqlSignal = 0, - highlightSql = false + openSqlSignal = 0 }: AssistantMessageBubbleProps) { - const [panelSqlOpenSignal, setPanelSqlOpenSignal] = useState(0); - const resolvedSqlOpenSignal = openSqlSignal + panelSqlOpenSignal; const hasRunReference = Boolean(runId); const hasResultArtifact = Boolean(run?.delivery?.artifact ?? streamDelivery?.artifact); const showUnifiedResultShell = hasRunReference || hasResultArtifact || thinkingInProgress || thinkingSteps.length > 0; return ( - -
- + +
+
-
-
+
+
, - Empty: () => + Empty: () => ( + + + 正在思考... + + ) }} />
{showUnifiedResultShell ? (
@@ -98,6 +97,7 @@ export function AssistantMessageBubble({ run={run} streamSteps={thinkingSteps} inProgress={thinkingInProgress} + cancelled={thinkingCancelled} hasRunReference={hasRunReference} runLoading={runLoading} onRequestRun={onRequestRun} @@ -108,35 +108,11 @@ export function AssistantMessageBubble({ run={run} streamDelivery={streamDelivery} runId={runId} - openSqlSignal={resolvedSqlOpenSignal} - onRequestSqlDetails={() => { - setPanelSqlOpenSignal((previous) => previous + 1); - }} + openSqlSignal={openSqlSignal} />
) : null} - - {runId ? ( -

- 同 run 摘要可在 - {" "} - - 设置 / RAG 运行与记忆治理 - - {" "} - 查看。 -

- ) : null}
); diff --git a/apps/frontend/src/components/chat/assistant-runtime.ts b/apps/frontend/src/components/chat/assistant-runtime.ts index 2968893..d2c74df 100644 --- a/apps/frontend/src/components/chat/assistant-runtime.ts +++ b/apps/frontend/src/components/chat/assistant-runtime.ts @@ -8,12 +8,15 @@ import type { } from "@text2sql/shared-types"; import type { ChatModelAdapter, ThreadMessageLike } from "@assistant-ui/react"; import { useLocalRuntime } from "@assistant-ui/react"; +import { collectTextDelta } from "@text2sql/chat-stream-protocol"; import { streamMessageEvents } from "@/lib/api-client"; export interface AssistantRuntimeCallbacks { onStart?: () => void; + onAbortController?: (controller: AbortController | null) => void; onEvent?: (event: ChatStreamEvent) => void; onFinish?: (runId: string | undefined) => Promise | void; + onCancelled?: (runId: string | undefined) => Promise | void; onError?: (error: Error) => Promise | void; onFinally?: () => void; } @@ -56,6 +59,29 @@ function extractLatestUserText(messages: readonly ThreadMessageLike[]): string { return ""; } +function composeAbortSignal( + signal: AbortSignal | undefined, + localSignal: AbortSignal +): AbortSignal { + if (!signal) { + return localSignal; + } + if (signal.aborted) { + return signal; + } + + const controller = new AbortController(); + const abortFrom = (source: AbortSignal) => { + if (!controller.signal.aborted) { + controller.abort(source.reason); + } + }; + + signal.addEventListener("abort", () => abortFrom(signal), { once: true }); + localSignal.addEventListener("abort", () => abortFrom(localSignal), { once: true }); + return controller.signal; +} + function createChatModelAdapter( sessionId: string, callbacksRef: MutableRefObject, @@ -72,27 +98,36 @@ function createChatModelAdapter( const contextEnvelope = resolveContextEnvelopeRef.current?.(); callbacksRef.current?.onStart?.(); + const localAbortController = new AbortController(); + callbacksRef.current?.onAbortController?.(localAbortController); + const effectiveAbortSignal = composeAbortSignal( + abortSignal, + localAbortController.signal + ); let aggregatedText = ""; let runId: string | undefined; let streamError: Error | null = null; + let cancelledByUser = false; try { for await (const event of streamMessageEvents( sessionId, userText, - abortSignal, + effectiveAbortSignal, contextEnvelope )) { runId = event.runId || runId; callbacksRef.current?.onEvent?.(event); if (event.type === "text-delta") { - const delta = (event.data as { text?: string } | undefined)?.text ?? ""; - if (!delta) { + const nextText = collectTextDelta(aggregatedText, event, { + readableSectionBreak: true + }); + if (nextText === aggregatedText) { continue; } - aggregatedText += delta; + aggregatedText = nextText; yield { content: [ { @@ -110,13 +145,24 @@ function createChatModelAdapter( } if (event.type === "error") { - const message = - (event.data as { message?: string } | undefined)?.message ?? "流式响应失败"; + const payload = event.data as + | { code?: string; message?: string } + | undefined; + if (payload?.code === "USER_CANCELLED") { + cancelledByUser = true; + break; + } + const message = payload?.message ?? "流式响应失败"; streamError = new Error(message); break; } } + if (cancelledByUser || effectiveAbortSignal.aborted) { + await callbacksRef.current?.onCancelled?.(runId); + return; + } + if (streamError) { await callbacksRef.current?.onError?.(streamError); return; @@ -124,6 +170,10 @@ function createChatModelAdapter( await callbacksRef.current?.onFinish?.(runId); } catch (runtimeError) { + if (effectiveAbortSignal.aborted) { + await callbacksRef.current?.onCancelled?.(runId); + return; + } const normalizedError = runtimeError instanceof Error ? runtimeError @@ -131,6 +181,7 @@ function createChatModelAdapter( await callbacksRef.current?.onError?.(normalizedError); return; } finally { + callbacksRef.current?.onAbortController?.(null); callbacksRef.current?.onFinally?.(); } } diff --git a/apps/frontend/src/components/chat/assistant-thinking-panel.tsx b/apps/frontend/src/components/chat/assistant-thinking-panel.tsx index 027ff4d..00121a7 100644 --- a/apps/frontend/src/components/chat/assistant-thinking-panel.tsx +++ b/apps/frontend/src/components/chat/assistant-thinking-panel.tsx @@ -6,35 +6,31 @@ import type { ReasoningStage, SqlRun } from "@text2sql/shared-types"; -import { BrainCircuit, ChevronDown, ChevronUp, Loader2 } from "lucide-react"; +import { CheckCircle2, ChevronDown, ChevronUp, Loader2, Wrench, XCircle } from "lucide-react"; import { mergeRunThinkingSteps } from "@/components/chat/run-visibility-mapper"; -import { Badge } from "@/components/ui/badge"; import { Button } from "@/components/ui/button"; import { StateBlock } from "@/components/ui/state-block"; +import { cn } from "@/lib/utils"; export interface ThinkingStreamStep extends ExecutionTraceStep { stage?: ReasoningStage; title?: string; + streamKind?: "state" | "tool"; + toolName?: string; + toolCallId?: string; + toolStatus?: "called" | "result" | "error"; } interface AssistantThinkingPanelProps { run: SqlRun | null; streamSteps: ThinkingStreamStep[]; inProgress: boolean; + cancelled?: boolean; hasRunReference?: boolean; runLoading?: boolean; onRequestRun?: () => void; } -const stageLabels: Record = { - analysis: "问题分析", - generation: "SQL 生成", - validation: "安全校验", - execution: "查询执行", - response: "回答整理", - unknown: "处理中" -}; - const nodeTitleMap: Record = { clarify: "理解问题", "generate-sql": "生成 SQL", @@ -49,28 +45,22 @@ const nodeTitleMap: Record = { "build-semantic-query": "语义检索构建" }; -function resolveStepTitle(step: ThinkingStreamStep): string { - return step.title ?? nodeTitleMap[step.node] ?? step.node; +function isToolStep(step: ThinkingStreamStep): boolean { + return step.streamKind === "tool" || step.node.startsWith("tool:"); } -function statusVariant(step: ThinkingStreamStep): "default" | "destructive" | "secondary" { - if (step.lifecycle === "running") { - return "secondary"; - } - if (step.status === "failed") { - return "destructive"; - } - if (step.status === "skipped") { - return "secondary"; +function resolveStepTitle(step: ThinkingStreamStep): string { + if (isToolStep(step)) { + return step.toolName ?? step.node.replace(/^tool:/, ""); } - return "default"; + return step.title ?? nodeTitleMap[step.node] ?? step.node; } function statusLabel(step: ThinkingStreamStep): string { if (step.lifecycle === "running") { return "进行中"; } - if (step.status === "failed") { + if (step.lifecycle === "failed" || step.status === "failed") { return "失败"; } if (step.status === "skipped") { @@ -79,48 +69,39 @@ function statusLabel(step: ThinkingStreamStep): string { return "完成"; } -function formatTime(value?: string): string { - if (!value) { - return ""; +function resolveStepSummary(step: ThinkingStreamStep): string | undefined { + return step.detail ?? step.errorSummary ?? step.outputSummary ?? step.inputSummary; +} + +function StepIcon({ step }: { step: ThinkingStreamStep }) { + if (step.lifecycle === "running") { + return ; + } + if (step.lifecycle === "failed" || step.status === "failed") { + return ; } - try { - return new Date(value).toLocaleTimeString("zh-CN", { - hour12: false - }); - } catch { - return value; + if (isToolStep(step)) { + return ; } + return ; } export function AssistantThinkingPanel({ run, streamSteps, inProgress, + cancelled = false, hasRunReference = false, runLoading = false, onRequestRun }: AssistantThinkingPanelProps) { const panelContentId = useId(); - const [open, setOpen] = useState(false); + const [open, setOpen] = useState(inProgress); const [requested, setRequested] = useState(false); const steps = useMemo(() => { return mergeRunThinkingSteps(run?.trace.steps, streamSteps); }, [run, streamSteps]); - const latestStep = steps.at(-1); - const activeStep = useMemo(() => { - const runningStep = [...steps].reverse().find((step) => step.lifecycle === "running"); - return runningStep ?? (inProgress ? latestStep : undefined); - }, [inProgress, latestStep, steps]); - const completedCount = steps.filter( - (step) => - step.lifecycle === "completed" || - (!step.lifecycle && step.status === "success") - ).length; - const hasFailedStep = steps.some( - (step) => step.lifecycle === "failed" || step.status === "failed" - ); - const latestStageLabel = latestStep?.stage ? stageLabels[latestStep.stage] : undefined; useEffect(() => { if (!open || run) { @@ -128,6 +109,12 @@ export function AssistantThinkingPanel({ } }, [open, run]); + useEffect(() => { + if (inProgress) { + setOpen(true); + } + }, [inProgress]); + useEffect(() => { if ( open && @@ -156,116 +143,92 @@ export function AssistantThinkingPanel({ return null; } + const visibleSteps = open ? steps : steps.slice(-1); + return ( -
-
-
-

- - AI 思考过程 - {inProgress ? ( - - - 思考中 - - ) : null} -

- {steps.length > 0 ? ( -
- {inProgress && activeStep ? ( -

- 活跃步骤:{resolveStepTitle(activeStep)}({statusLabel(activeStep)}) -

- ) : null} -

- {latestStageLabel ? `当前阶段:${latestStageLabel} · ` : null} - 已记录 {steps.length} 步 - {inProgress ? `,已完成 ${completedCount}/${steps.length} 步` : null} - {hasFailedStep ? "(含失败步骤)" : null} -

-
- ) : hasRunReference ? ( -

- 可展开查看该轮结构化思考摘要 -

- ) : null} -
- +
+
+ {inProgress ? ( + + + 正在思考... + + ) : cancelled ? ( + 已停止生成 + ) : steps.length > 0 ? ( + 处理过程 + ) : ( + 可查看处理过程 + )} + + {steps.length > 0 || hasRunReference ? ( + + ) : null}
- {open ? ( -
- {steps.length === 0 && runLoading ? ( - 正在加载该轮思考轨迹... - ) : steps.length === 0 ? ( - - {inProgress ? "模型正在组织思路,请稍候..." : "暂未加载到该轮思考轨迹。"} - +
+ {steps.length === 0 && runLoading ? ( + 正在加载该轮处理轨迹... + ) : steps.length === 0 ? ( + inProgress ? ( +

等待模型返回下一段内容...

) : ( - steps.map((step, index) => ( -
暂未加载到该轮流程轨迹。 + ) + ) : ( + visibleSteps.map((step, index) => { + const failed = step.lifecycle === "failed" || step.status === "failed"; + const toolStep = isToolStep(step); + const summary = resolveStepSummary(step); + const line = `< | ${toolStep ? "function" : "大模型"} | ${resolveStepTitle(step)}(${statusLabel(step)})`; + return ( +
- -
-

- {resolveStepTitle(step)} -

-

- {formatTime(step.at || step.endedAt || step.startedAt)} -

-
-
- {step.stage ? ( - - {stageLabels[step.stage]} - - ) : null} - {step.durationMs !== undefined ? ( - {step.durationMs}ms - ) : null} - {statusLabel(step)} -
-
-
- {step.detail ?

Detail: {step.detail}

: null} - {step.inputSummary ?

Input: {step.inputSummary}

: null} - {step.outputSummary ?

Output: {step.outputSummary}

: null} - {step.errorSummary ?

Error: {step.errorSummary}

: null} -
-
- )) - )} -
- ) : null} +

+ + {line} +

+ {summary ? ( +

+ {summary} +

+ ) : null} +
+ ); + }) + )} +
); } diff --git a/apps/frontend/src/components/chat/assistant-thread.tsx b/apps/frontend/src/components/chat/assistant-thread.tsx index e617eec..e1c403f 100644 --- a/apps/frontend/src/components/chat/assistant-thread.tsx +++ b/apps/frontend/src/components/chat/assistant-thread.tsx @@ -1,6 +1,6 @@ "use client"; -import { useMemo, useState } from "react"; +import { useMemo } from "react"; import type { ChatMessage, ChatStreamEvent, @@ -14,10 +14,6 @@ import { } from "@assistant-ui/react"; import { ArrowDown, Loader2 } from "lucide-react"; import { AssistantComposer } from "@/components/chat/assistant-composer"; -import { - buildContextEnvelopeFromDraft, - createEmptyContextEnvelopeDraft -} from "@/components/chat/context-envelope-panel"; import { AssistantThinkingPanel, type ThinkingStreamStep @@ -48,11 +44,14 @@ interface AssistantThreadProps { runVisibilityByRunId: Record; activeStreamRunId: string | null; thinkingRequestPending: boolean; - debugEnabled: boolean; + activeAbortController: AbortController | null; disabled?: boolean; + onStopStream?: () => void; onRequestRun?: (runId: string) => Promise | void; + onAbortControllerChange?: (controller: AbortController | null) => void; onRunStart?: () => void; onRunFinish?: (runId: string | undefined) => Promise | void; + onRunCancelled?: (runId: string | undefined) => Promise | void; onRunError?: (error: Error) => Promise | void; onStreamEvent?: (event: ChatStreamEvent) => void; } @@ -86,21 +85,17 @@ export function AssistantThread({ runVisibilityByRunId, activeStreamRunId, thinkingRequestPending, - debugEnabled, + activeAbortController, disabled = false, + onStopStream, onRequestRun, + onAbortControllerChange, onRunStart, onRunFinish, + onRunCancelled, onRunError, onStreamEvent }: AssistantThreadProps) { - const [sqlOpenSignal, setSqlOpenSignal] = useState(0); - const [contextEnvelopeDraft, setContextEnvelopeDraft] = useState( - createEmptyContextEnvelopeDraft - ); - const [clearContextEnvelopeAfterSend, setClearContextEnvelopeAfterSend] = - useState(true); - const latestAssistantMessageId = useMemo(() => { return [...messages].reverse().find((message) => message.role === "assistant")?.id; }, [messages]); @@ -130,13 +125,17 @@ export function AssistantThread({ const callbacks = useMemo( () => ({ onStart: onRunStart, + onAbortController: onAbortControllerChange, onEvent: onStreamEvent, + onCancelled: onRunCancelled, onError: onRunError, onFinish: async (runId) => { await onRunFinish?.(runId); } }), [ + onAbortControllerChange, + onRunCancelled, onRunError, onRunFinish, onRunStart, @@ -144,25 +143,10 @@ export function AssistantThread({ ] ); - const resolveContextEnvelopeForSend = useMemo( - () => () => { - const envelope = buildContextEnvelopeFromDraft(contextEnvelopeDraft); - if (clearContextEnvelopeAfterSend) { - setContextEnvelopeDraft(createEmptyContextEnvelopeDraft()); - } - return envelope; - }, - [ - clearContextEnvelopeAfterSend, - contextEnvelopeDraft - ] - ); - const runtime = useChatAssistantRuntime({ sessionId, messages, - callbacks, - resolveContextEnvelope: resolveContextEnvelopeForSend + callbacks }); const activeStreamSteps = activeStreamRunId ? streamThinkingByRunId[activeStreamRunId] ?? [] @@ -172,7 +156,6 @@ export function AssistantThread({ : false; const showStandaloneThinking = Boolean(activeStreamRunId) && - activeStreamSteps.length > 0 && !activeStreamTextStarted; return ( @@ -212,6 +195,7 @@ export function AssistantThread({ }) : undefined; const thinkingInProgress = runVisibilityStatus === "loading"; + const thinkingCancelled = runVisibilityStatus === "cancelled"; const streamDelivery = resolvedRunId ? resolveVisibleDelivery({ runDelivery: run?.delivery, @@ -227,9 +211,9 @@ export function AssistantThread({ run={run} streamDelivery={streamDelivery} runId={resolvedRunId} - debugEnabled={debugEnabled} thinkingSteps={thinkingSteps} thinkingInProgress={thinkingInProgress} + thinkingCancelled={thinkingCancelled} runLoading={Boolean( resolvedRunId && runLoadingById[resolvedRunId] )} @@ -238,8 +222,6 @@ export function AssistantThread({ ? () => onRequestRun?.(resolvedRunId) : undefined } - openSqlSignal={isLatestAssistant ? sqlOpenSignal : 0} - highlightSql={isLatestAssistant} /> ); } @@ -248,7 +230,7 @@ export function AssistantThread({ {showStandaloneThinking && activeStreamRunId ? (
@@ -256,14 +238,15 @@ export function AssistantThread({ run={null} streamSteps={activeStreamSteps} inProgress + cancelled={false} hasRunReference />
) : null} {thinkingRequestPending && !activeStreamRunId ? ( -
- - 思考中... +
+ + 正在连接模型...
) : null}
@@ -271,15 +254,8 @@ export function AssistantThread({ { - setSqlOpenSignal((previous) => previous + 1); - }} - contextEnvelopeDraft={contextEnvelopeDraft} - clearContextEnvelopeAfterSend={clearContextEnvelopeAfterSend} - onContextEnvelopeDraftChange={setContextEnvelopeDraft} - onClearContextEnvelopeAfterSendChange={ - setClearContextEnvelopeAfterSend - } + showStopButton={Boolean(activeAbortController) && !disabled} + onStop={onStopStream} /> diff --git a/apps/frontend/src/components/chat/chatbi-result-panel.tsx b/apps/frontend/src/components/chat/chatbi-result-panel.tsx index db36901..edfe721 100644 --- a/apps/frontend/src/components/chat/chatbi-result-panel.tsx +++ b/apps/frontend/src/components/chat/chatbi-result-panel.tsx @@ -27,7 +27,6 @@ interface ChatBIResultPanelProps { streamDelivery?: DeliveryContract; runId?: string; openSqlSignal?: number; - onRequestSqlDetails?: () => void; } type JsonRecord = Record; @@ -350,8 +349,7 @@ export function ChatBIResultPanel({ run, streamDelivery, runId, - openSqlSignal = 0, - onRequestSqlDetails + openSqlSignal = 0 }: ChatBIResultPanelProps) { const delivery = run?.delivery ?? streamDelivery; const artifact = delivery?.artifact; @@ -399,10 +397,10 @@ export function ChatBIResultPanel({ return (
-
+

ChatBI Result

@@ -462,7 +460,7 @@ export function ChatBIResultPanel({

- View SQL 为次级证据入口,可快速跳转到运行详情查看执行链路、RAG 证据与调试信息。 + 用于生成当前回答的 SQL 证据。

{artifactView.sql || run?.sql ? (
@@ -471,11 +469,6 @@ export function ChatBIResultPanel({
               ) : (
                 当前结果未返回 SQL 文本。
               )}
-              {onRequestSqlDetails ? (
-                
-              ) : null}
             
diff --git a/apps/frontend/src/components/chat/chatbi-result-tabs.tsx b/apps/frontend/src/components/chat/chatbi-result-tabs.tsx index 301e12e..fc780ab 100644 --- a/apps/frontend/src/components/chat/chatbi-result-tabs.tsx +++ b/apps/frontend/src/components/chat/chatbi-result-tabs.tsx @@ -66,7 +66,7 @@ export function ChatBIResultTabs({ variant="line" aria-label="ChatBI result partitions" data-testid="chatbi-result-tablist" - className="flex h-auto w-full justify-start gap-1 overflow-x-auto rounded-md border border-[var(--border-default)] bg-[var(--surface-subtle)] px-1 py-1 [scrollbar-width:none]" + className="grid h-auto w-full grid-cols-3 gap-1 overflow-hidden rounded-md border border-[var(--border-default)] bg-[var(--surface-subtle)] px-1 py-1" > {TAB_DEFINITIONS.map((item) => { const Icon = item.icon; @@ -78,10 +78,10 @@ export function ChatBIResultTabs({ disabled={disabled} onKeyDown={buildKeyboardHandler(item.value, onValueChange)} aria-label={`Switch to ${item.label} partition`} - className="h-8 min-w-[96px] shrink-0 justify-start gap-1.5 rounded-md px-2 text-xs sm:text-sm" + className="h-8 w-full min-w-0 shrink-0 justify-center gap-1 px-1.5 text-[11px] sm:gap-1.5 sm:px-2 sm:text-sm" > - {item.label} + {item.label} ); })} diff --git a/apps/frontend/src/components/chat/run-visibility-mapper.ts b/apps/frontend/src/components/chat/run-visibility-mapper.ts index 1d1bb71..b37d80f 100644 --- a/apps/frontend/src/components/chat/run-visibility-mapper.ts +++ b/apps/frontend/src/components/chat/run-visibility-mapper.ts @@ -3,17 +3,29 @@ import type { DeliveryContract, DeliveryEvidenceLayer, ExecutionTraceStep, - ReasoningStage, RunStatus, SqlRun } from "@text2sql/shared-types"; +import { + toRunVisibilityStatusFromRunStatus, + transitionRunVisibilityStatus +} from "@text2sql/chat-stream-protocol"; +export { + projectStreamEvent, + toRunVisibilityStatusFromRunStatus, + toThinkingStepFromStreamEvent, + transitionRunVisibilityStatus +} from "@text2sql/chat-stream-protocol"; +export type { + RunVisibilityStatus, + RunVisibilityThinkingStep, + StreamEventProjection +} from "@text2sql/chat-stream-protocol"; -export type RunVisibilityStatus = "loading" | "success" | "error" | "empty"; - -export type RunVisibilityThinkingStep = ExecutionTraceStep & { - stage?: ReasoningStage; - title?: string; -}; +import type { + RunVisibilityStatus, + RunVisibilityThinkingStep +} from "@text2sql/chat-stream-protocol"; type JsonRecord = Record; type SavedPriorSqlLayer = NonNullable; @@ -895,34 +907,6 @@ export function mergeRunThinkingSteps( }); } -export function toRunVisibilityStatusFromRunStatus( - status: RunStatus | undefined -): RunVisibilityStatus | undefined { - if (!status) { - return undefined; - } - if (status === "failed" || status === "rejected") { - return "error"; - } - if (status === "clarification") { - return "empty"; - } - return "success"; -} - -export function transitionRunVisibilityStatus( - previous: RunVisibilityStatus | undefined, - next: RunVisibilityStatus | undefined -): RunVisibilityStatus | undefined { - if (!next) { - return previous; - } - if ((previous === "success" || previous === "error" || previous === "empty") && next === "loading") { - return previous; - } - return next; -} - export function resolveRunVisibilityStatus(input: { runStatus?: RunStatus; streamStatus?: RunVisibilityStatus; diff --git a/apps/frontend/src/lib/api-client.ts b/apps/frontend/src/lib/api-client.ts index fd1686a..445c913 100644 --- a/apps/frontend/src/lib/api-client.ts +++ b/apps/frontend/src/lib/api-client.ts @@ -1,11 +1,11 @@ import type { AgentRunResponse, ApiResponse, - ContextEnvelope, - DeliveryEvidenceLayer, ChatStreamEvent, ChatSessionView, + ContextEnvelope, Datasource, + DeliveryEvidenceLayer, PreviewDatasourceTablesRequest, PreviewDatasourceTablesResponse, PromptTemplateTraceEvidenceCompat, @@ -17,6 +17,7 @@ import type { UpsertDatasourceWorkflowRequest, UpsertDatasourceWorkflowResponse } from "@text2sql/shared-types"; +import { readSseStream } from "@text2sql/chat-stream-protocol"; const API_BASE_OVERRIDE = process.env.NEXT_PUBLIC_API_BASE_URL?.trim(); const API_BASE = API_BASE_OVERRIDE ? API_BASE_OVERRIDE.replace(/\/+$/, "") : ""; @@ -900,44 +901,8 @@ export async function* streamMessageEvents( if (!response.ok) { throw new Error(`流式请求失败(HTTP ${response.status})`); } - const reader = response.body?.getReader(); - if (!reader) { - throw new Error("流式响应体不可读。"); - } - const decoder = new TextDecoder(); - let buffer = ""; - while (true) { - const { done, value } = await reader.read(); - if (done) { - break; - } - buffer += decoder.decode(value, { stream: true }); - const blocks = buffer.split("\n\n"); - buffer = blocks.pop() ?? ""; - - for (const block of blocks) { - const lines = block - .split("\n") - .map((line) => line.trim()) - .filter(Boolean); - if (lines.length === 0) { - continue; - } - const eventLine = lines.find((line) => line.startsWith("event:")); - const dataLines = lines.filter((line) => line.startsWith("data:")); - if (!eventLine || dataLines.length === 0) { - continue; - } - const eventType = eventLine.replace(/^event:\s*/, ""); - const payload = dataLines - .map((line) => line.replace(/^data:\s*/, "")) - .join("\n"); - const event = JSON.parse(payload) as ChatStreamEvent; - yield event; - if (eventType === "error") { - return; - } - } + for await (const event of readSseStream(response.body)) { + yield event; } } diff --git a/apps/frontend/tests/e2e/chat-demo-flow.spec.tsx b/apps/frontend/tests/e2e/chat-demo-flow.spec.tsx index c36ee71..e665976 100644 --- a/apps/frontend/tests/e2e/chat-demo-flow.spec.tsx +++ b/apps/frontend/tests/e2e/chat-demo-flow.spec.tsx @@ -385,14 +385,14 @@ describe("chat demo flow", () => { const initialGetMessagesCalls = mockGetMessages.mock.calls.length; await user.type(screen.getByLabelText("聊天输入"), "统计订单支付方式"); await user.click(screen.getByRole("button", { name: "发送" })); - expect(await screen.findByText(/思考中/)).toBeInTheDocument(); + expect((await screen.findAllByText(/正在连接模型|正在思考/)).length).toBeGreaterThan(0); releaseFirstEvent?.(); await waitFor(() => { expect(mockStreamMessageEvents).toHaveBeenCalled(); }); await waitFor(() => { - expect(screen.queryByText(/思考中/)).not.toBeInTheDocument(); + expect(screen.queryAllByText(/正在连接模型|正在思考/)).toHaveLength(0); }); await waitFor(() => { expect(mockGetMessages.mock.calls.length).toBeGreaterThan( @@ -408,10 +408,10 @@ describe("chat demo flow", () => { "finish" ]); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); - expect(screen.getByText("知识检索")).toBeInTheDocument(); - expect(screen.getByText("意图规划")).toBeInTheDocument(); - expect(screen.getByText("语义检索构建")).toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: /处理过程/ })); + expect(screen.getByText(/知识检索/)).toBeInTheDocument(); + expect(screen.getByText(/意图规划/)).toBeInTheDocument(); + expect(screen.getByText(/语义检索构建/)).toBeInTheDocument(); expect(screen.getByTestId("assistant-result-shell")).toHaveAttribute("data-run-id", "run-1"); const resultPanel = await screen.findByTestId("chatbi-result-panel"); @@ -425,16 +425,16 @@ describe("chat demo flow", () => { expect(resultQueries.getByTestId("chatbi-bar-chart")).toBeInTheDocument(); await user.click(resultQueries.getByRole("tab", { name: /view sql/i })); - await user.click(resultQueries.getByRole("button", { name: "打开运行详情" })); - expect(screen.getByText("运行详情")).toBeInTheDocument(); - expect(screen.getByText("运行 ID:run-1")).toBeInTheDocument(); - expect(screen.getByText("degrade_reason:retrieval_timeout")).toBeInTheDocument(); - expect(screen.getByText("semantic_registry_degraded")).toBeInTheDocument(); - expect(screen.getByText("context:user-explicit")).toBeInTheDocument(); - expect(screen.getByText("context:system-inferred")).toBeInTheDocument(); + expect(resultQueries.getByText(/SELECT payment_method, COUNT\(\*\) AS cnt FROM orders GROUP BY payment_method/i)).toBeInTheDocument(); + expect(screen.queryByText("运行详情")).not.toBeInTheDocument(); + expect(screen.queryByText("运行 ID:run-1")).not.toBeInTheDocument(); + expect(screen.queryByText("degrade_reason:retrieval_timeout")).not.toBeInTheDocument(); + expect(screen.queryByText("semantic_registry_degraded")).not.toBeInTheDocument(); + expect(screen.queryByText("context:user-explicit")).not.toBeInTheDocument(); + expect(screen.queryByText("context:system-inferred")).not.toBeInTheDocument(); expect( - screen.getByRole("link", { name: "设置 / RAG 运行与记忆治理" }) - ).toHaveAttribute("href", "/settings?tab=rag&runId=run-1"); + screen.queryByRole("link", { name: "设置 / RAG 运行与记忆治理" }) + ).not.toBeInTheDocument(); expect( screen.getAllByText("SELECT payment_method, COUNT(*) AS cnt FROM orders GROUP BY payment_method").length ).toBeGreaterThan(0); diff --git a/apps/frontend/tests/e2e/chat-mobile-smoke.spec.tsx b/apps/frontend/tests/e2e/chat-mobile-smoke.spec.tsx index f7bb2e8..96f1c59 100644 --- a/apps/frontend/tests/e2e/chat-mobile-smoke.spec.tsx +++ b/apps/frontend/tests/e2e/chat-mobile-smoke.spec.tsx @@ -179,66 +179,43 @@ describe("chat mobile smoke", () => { }); it( - "renders mobile-usable chat controls and progressive delivery toggles", + "renders a mobile-stable result panel with keyboard-accessible tabs", async () => { - const user = userEvent.setup(); - render(); - await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); + const user = userEvent.setup(); + render(); + await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); - expect(screen.getByLabelText("聊天输入")).toBeEnabled(); - expect(screen.getByRole("button", { name: "结果详情" })).toBeInTheDocument(); - expect(screen.getByRole("button", { name: "会话" })).toBeInTheDocument(); + expect(screen.getByLabelText("聊天输入")).toBeEnabled(); + expect(screen.getByRole("button", { name: "会话" })).toBeInTheDocument(); - await user.click(screen.getByRole("button", { name: "结果详情" })); - const answerTrigger = await screen.findByRole("button", { - name: "切换 Answer 区块" - }); - const evidenceTrigger = screen.getByRole("button", { - name: "切换 Evidence 区块" - }); - expect(answerTrigger).toBeInTheDocument(); - expect(evidenceTrigger).toBeInTheDocument(); - expect(screen.getByRole("button", { name: "切换 Artifact 区块" })).toBeInTheDocument(); - - expect(evidenceTrigger).toHaveAttribute("aria-expanded", "false"); - expect(evidenceTrigger).toHaveAttribute("aria-controls"); - evidenceTrigger.focus(); - await user.keyboard("{Enter}"); - expect(evidenceTrigger).toHaveAttribute("aria-expanded", "true"); - evidenceTrigger.focus(); - await user.keyboard(" "); - await waitFor(() => { - expect(evidenceTrigger).toHaveAttribute("aria-expanded", "false"); - }); - - await user.type(screen.getByLabelText("聊天输入"), "移动端发送链路回归"); - await user.click(screen.getByRole("button", { name: "发送" })); - await waitFor(() => { - expect(mockStreamMessageEvents).toHaveBeenCalled(); - }); - expect(screen.getByTestId("assistant-result-shell")).toHaveAttribute("data-run-id", "run-1"); - - const answerTab = screen.getByRole("tab", { name: /answer/i }); - const chartTab = screen.getByRole("tab", { name: /chart/i }); - const sqlTab = screen.getByRole("tab", { name: /view sql/i }); - const tabList = screen.getByTestId("chatbi-result-tablist"); - expect(answerTab).toHaveAttribute("aria-selected", "true"); - expect(chartTab).toHaveAttribute("aria-controls"); - expect(sqlTab).toHaveAttribute("aria-controls"); - expect(tabList).toHaveClass("overflow-x-auto"); + await user.type(screen.getByLabelText("聊天输入"), "移动端发送链路回归"); + await user.click(screen.getByRole("button", { name: "发送" })); + await waitFor(() => { + expect(mockStreamMessageEvents).toHaveBeenCalled(); + }); + expect(screen.getByTestId("assistant-result-shell")).toHaveAttribute("data-run-id", "run-1"); - chartTab.focus(); - await user.keyboard("{Enter}"); - expect(chartTab).toHaveAttribute("aria-selected", "true"); + const answerTab = screen.getByRole("tab", { name: /answer/i }); + const chartTab = screen.getByRole("tab", { name: /chart/i }); + const sqlTab = screen.getByRole("tab", { name: /view sql/i }); + const tabList = screen.getByTestId("chatbi-result-tablist"); + expect(answerTab).toHaveAttribute("aria-selected", "true"); + expect(chartTab).toHaveAttribute("aria-controls"); + expect(sqlTab).toHaveAttribute("aria-controls"); + expect(tabList).toHaveClass("grid-cols-3"); + expect(tabList).toHaveClass("overflow-hidden"); - sqlTab.focus(); - await user.keyboard("[Space]"); - expect(sqlTab).toHaveAttribute("aria-selected", "true"); - expect(screen.getByRole("button", { name: "打开运行详情" })).toBeInTheDocument(); + chartTab.focus(); + await user.keyboard("{Enter}"); + expect(chartTab).toHaveAttribute("aria-selected", "true"); - expect( - screen.getByRole("link", { name: "设置 / RAG 运行与记忆治理" }) - ).toHaveAttribute("href", "/settings?tab=rag&runId=run-1"); + sqlTab.focus(); + await user.keyboard("[Space]"); + expect(sqlTab).toHaveAttribute("aria-selected", "true"); + expect(screen.getByText("用于生成当前回答的 SQL 证据。")).toBeInTheDocument(); + expect( + screen.getByText(/SELECT payment_method, COUNT\(\*\) AS cnt FROM orders GROUP BY payment_method/i) + ).toBeInTheDocument(); }, 15000 ); diff --git a/apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx b/apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx index 7c8b037..7db92ad 100644 --- a/apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx +++ b/apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx @@ -5,7 +5,7 @@ import { AssistantThinkingPanel } from "@/components/chat/assistant-thinking-pan import { createMockRun } from "./fixtures"; describe("AssistantThinkingPanel", () => { - it("shows active step and progress in collapsed header during in-progress run", () => { + it("shows active step and progress in an expanded timeline during in-progress run", () => { render( { /> ); - expect(screen.getByRole("button", { name: "展开思考过程" })).toHaveAttribute( + expect(screen.getByRole("button", { name: "收起处理过程" })).toHaveAttribute( "aria-expanded", - "false" + "true" ); - expect(screen.getByText("活跃步骤:生成 SQL(进行中)")).toBeInTheDocument(); - expect(screen.getByText(/已完成 1\/2 步/)).toBeInTheDocument(); + expect(screen.getByText("< | 大模型 | 生成 SQL(进行中)")).toBeInTheDocument(); + expect(screen.getByText("< | 大模型 | 理解问题(完成)")).toBeInTheDocument(); }); it("supports keyboard toggle with synced aria-expanded state", async () => { @@ -62,14 +62,14 @@ describe("AssistantThinkingPanel", () => { /> ); - const trigger = screen.getByRole("button", { name: "展开思考过程" }); + const trigger = screen.getByRole("button", { name: "展开处理过程" }); expect(trigger).toHaveAttribute("aria-expanded", "false"); expect(trigger).toHaveAttribute("aria-controls"); trigger.focus(); await user.keyboard("{Enter}"); expect(trigger).toHaveAttribute("aria-expanded", "true"); - expect(screen.getAllByText("生成 SQL").length).toBeGreaterThan(0); + expect(screen.getAllByText(/生成 SQL/).length).toBeGreaterThan(0); await user.click(trigger); expect(trigger).toHaveAttribute("aria-expanded", "false"); @@ -88,7 +88,7 @@ describe("AssistantThinkingPanel", () => { /> ); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); + await user.click(screen.getByRole("button", { name: "展开处理过程" })); await waitFor(() => { expect(onRequestRun).toHaveBeenCalledTimes(1); }); @@ -114,7 +114,7 @@ describe("AssistantThinkingPanel", () => { /> ); - const trigger = screen.getByRole("button", { name: "展开思考过程" }); + const trigger = screen.getByRole("button", { name: "展开处理过程" }); await user.click(trigger); expect(trigger).toHaveAttribute("aria-expanded", "true"); @@ -159,9 +159,9 @@ describe("AssistantThinkingPanel", () => { /> ); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); - expect(screen.getByText("失败")).toBeInTheDocument(); - expect(screen.getByText("Error: sql contains forbidden keyword")).toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: "展开处理过程" })); + expect(screen.getByText("< | 大模型 | 安全校验(失败)")).toBeInTheDocument(); + expect(screen.getByText("sql contains forbidden keyword")).toBeInTheDocument(); }); it("keeps completed steps visible after finish and does not regress to loading placeholder", async () => { @@ -184,8 +184,8 @@ describe("AssistantThinkingPanel", () => { /> ); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); - expect(screen.getByText("生成 SQL")).toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: "展开处理过程" })); + expect(screen.getByText(/生成 SQL/)).toBeInTheDocument(); rerender( { /> ); - expect(screen.getByText("生成 SQL")).toBeInTheDocument(); - expect(screen.queryByText("正在加载该轮思考轨迹...")).not.toBeInTheDocument(); + expect(screen.getByText(/生成 SQL/)).toBeInTheDocument(); + expect(screen.queryByText("正在加载该轮处理轨迹...")).not.toBeInTheDocument(); + }); + + it("shows cancelled state copy when user stops generation", () => { + render( + + ); + + expect(screen.getByText("已停止生成")).toBeInTheDocument(); }); }); diff --git a/apps/frontend/tests/unit/chat-panel.spec.tsx b/apps/frontend/tests/unit/chat-panel.spec.tsx index 03b85fe..6d28139 100644 --- a/apps/frontend/tests/unit/chat-panel.spec.tsx +++ b/apps/frontend/tests/unit/chat-panel.spec.tsx @@ -162,14 +162,15 @@ describe("ChatPanel", () => { expect(screen.getByRole("button", { name: "发送" })).toBeDisabled(); }); - it("keeps advanced context panel collapsed by default", async () => { + it("does not render advanced context entry on chat page", async () => { render(); await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); - const advancedContextButton = screen.getByRole("button", { - name: "展开高级上下文" - }); - expect(advancedContextButton).toHaveAttribute("aria-expanded", "false"); + expect( + screen.queryByRole("button", { + name: "展开高级上下文" + }) + ).not.toBeInTheDocument(); expect(screen.queryByLabelText("指标口径")).not.toBeInTheDocument(); }); @@ -187,21 +188,16 @@ describe("ChatPanel", () => { }); expect(screen.queryByText("发送成功,已收到后端响应。")).not.toBeInTheDocument(); - expect(screen.getByText("AI 思考过程")).toBeInTheDocument(); expect(screen.getByTestId("assistant-result-shell")).toBeInTheDocument(); expect(screen.getByTestId("assistant-result-shell-steps")).toBeInTheDocument(); expect(screen.getByTestId("assistant-result-shell-answer")).toBeInTheDocument(); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); - expect(screen.getByText("生成 SQL")).toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: /处理过程/ })); + expect(screen.getByText(/大模型 \| 生成 SQL/)).toBeInTheDocument(); expect(screen.getByText("已为你生成 SQL,并展示结果。")).toBeInTheDocument(); - - await user.click(screen.getByRole("button", { name: "展开 SQL 详情" })); - expect( - screen.getByText("SELECT payment_method, COUNT(*) AS cnt FROM orders GROUP BY payment_method") - ).toBeInTheDocument(); + expect(screen.queryByRole("button", { name: "打开运行详情" })).not.toBeInTheDocument(); expect( - screen.getByRole("link", { name: "设置 / RAG 运行与记忆治理" }) - ).toHaveAttribute("href", "/settings?tab=rag&runId=run-1"); + screen.queryByRole("link", { name: "设置 / RAG 运行与记忆治理" }) + ).not.toBeInTheDocument(); }); it("shows live thinking steps before the model emits text", async () => { @@ -257,11 +253,66 @@ describe("ChatPanel", () => { const liveThinking = await screen.findByTestId("assistant-live-thinking"); expect(liveThinking).toHaveAttribute("data-run-id", "run-thinking"); - expect(screen.getByText("活跃步骤:生成 SQL(进行中)")).toBeInTheDocument(); + expect(screen.getByText("< | 大模型 | 生成 SQL(进行中)")).toBeInTheDocument(); releaseStream?.(); }); + it("shows stop button and keeps partial content visible after user stops a stream", async () => { + let capturedSignal: AbortSignal | undefined; + mockStreamMessageEvents.mockImplementationOnce(async function* ( + _sessionId, + _message, + abortSignal + ) { + capturedSignal = abortSignal; + yield { + type: "start", + runId: "run-stop", + sessionId: "session-1", + at: "2026-04-10T00:00:00.000Z", + data: { + requestId: null + } + }; + yield { + type: "text-delta", + runId: "run-stop", + sessionId: "session-1", + at: "2026-04-10T00:00:01.000Z", + data: { + text: "SELECT payment_method" + } + }; + await new Promise((resolve, reject) => { + abortSignal?.addEventListener( + "abort", + () => { + reject(new DOMException("The operation was aborted.", "AbortError")); + }, + { once: true } + ); + }); + }); + + const user = userEvent.setup(); + render(); + + await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); + await user.type(screen.getByLabelText("聊天输入"), "近30天支付方式分布"); + await user.click(screen.getByRole("button", { name: "发送" })); + + const stopButton = await screen.findByRole("button", { name: "停止生成" }); + expect(stopButton).toBeEnabled(); + expect(capturedSignal).toBeDefined(); + + await user.click(stopButton); + + await waitFor(() => { + expect(capturedSignal?.aborted).toBe(true); + }); + }); + it("submits by pressing Enter in chat input", async () => { const user = userEvent.setup(); render(); @@ -334,7 +385,7 @@ describe("ChatPanel", () => { await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); await screen.findByText("已为你生成 SQL,并展示结果。"); expect(screen.getByTestId("assistant-result-shell")).toHaveAttribute("data-run-id", "run-1"); - await user.click(await screen.findByRole("button", { name: "展开思考过程" })); + await user.click(await screen.findByRole("button", { name: /处理过程/ })); await waitFor(() => { expect(mockGetRun).toHaveBeenCalledWith("run-1"); @@ -357,16 +408,16 @@ describe("ChatPanel", () => { const resultShell = screen.getByTestId("assistant-result-shell"); expect(resultShell).toHaveAttribute("data-run-id", "run-1"); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); + await user.click(screen.getByRole("button", { name: /处理过程/ })); await waitFor(() => { expect(mockGetRun).toHaveBeenCalledWith("run-1"); }); expect(screen.getByText("已为你生成 SQL,并展示结果。")).toBeInTheDocument(); - expect(screen.getByText("暂未加载到该轮思考轨迹。")).toBeInTheDocument(); + expect(screen.getByText("暂未加载到该轮流程轨迹。")).toBeInTheDocument(); }); - it("keeps run link bindings isolated per assistant message", async () => { + it("keeps run bindings isolated per assistant message", async () => { const run2 = createMockRun({ runId: "run-2", question: "近7天支付方式分布", @@ -421,17 +472,13 @@ describe("ChatPanel", () => { await screen.findByText("第一轮回答"); await screen.findByText("第二轮回答"); - const runLinks = screen.getAllByRole("link", { - name: "设置 / RAG 运行与记忆治理" - }); - expect(runLinks).toHaveLength(2); - expect(runLinks[0]).toHaveAttribute("href", "/settings?tab=rag&runId=run-1"); - expect(runLinks[1]).toHaveAttribute("href", "/settings?tab=rag&runId=run-2"); - const resultShells = screen.getAllByTestId("assistant-result-shell"); expect(resultShells).toHaveLength(2); expect(resultShells[0]).toHaveAttribute("data-run-id", "run-1"); expect(resultShells[1]).toHaveAttribute("data-run-id", "run-2"); + expect( + screen.queryByRole("link", { name: "设置 / RAG 运行与记忆治理" }) + ).not.toBeInTheDocument(); }); it("shows initialization failure when session creation fails", async () => { @@ -478,7 +525,7 @@ describe("ChatPanel", () => { await user.type(screen.getByLabelText("聊天输入"), "近30天支付方式分布"); await user.click(screen.getByRole("button", { name: "发送" })); - expect(await screen.findByText(/思考中/)).toBeInTheDocument(); + expect((await screen.findAllByText(/正在连接模型|正在思考/)).length).toBeGreaterThan(0); if (releaseFirstEvent) { releaseFirstEvent(); } @@ -554,29 +601,12 @@ describe("ChatPanel", () => { expect(latestCall?.[3]).toBeUndefined(); }); - it("injects advanced context envelope into stream request and clears it after send by default", async () => { + it("sends stream request without advanced context envelope", async () => { const user = userEvent.setup(); render(); await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); - await user.click( - screen.getByRole("button", { - name: "展开高级上下文" - }) - ); - - await user.type( - screen.getByLabelText("指标口径"), - "按支付成功口径统计订单" - ); - await user.type(screen.getByLabelText("开始日期"), "2026-03-01"); - await user.type(screen.getByLabelText("结束日期"), "2026-03-31"); - await user.type( - screen.getByLabelText("实体映射"), - "华北大区=region_north" - ); - await user.type(screen.getByLabelText("聊天输入"), "近30天支付方式分布"); await user.click(screen.getByRole("button", { name: "发送" })); @@ -585,29 +615,10 @@ describe("ChatPanel", () => { }); const latestCall = mockStreamMessageEvents.mock.calls.at(-1); - expect(latestCall?.[3]).toEqual({ - metricDefinition: "按支付成功口径统计订单", - timeRange: { - from: "2026-03-01", - to: "2026-03-31" - }, - entityMappings: [ - { - entity: "华北大区", - mappedTo: "region_north" - } - ] - }); - - await user.click( - screen.getByRole("button", { - name: "展开高级上下文" - }) - ); - expect(screen.getByLabelText("指标口径")).toHaveValue(""); + expect(latestCall?.[3]).toBeUndefined(); }); - it("keeps run detail entry visible while run backfill is still loading", async () => { + it("shows processing timeline backfill state while historical run data is still loading", async () => { const user = userEvent.setup(); let resolveRun: (() => void) | undefined; const runBackfillGate = new Promise((resolve) => { @@ -632,19 +643,18 @@ describe("ChatPanel", () => { render(); await screen.findByText(/Datasource: sqlite_main · Session: session-1/i); - await user.type(screen.getByLabelText("聊天输入"), "近30天支付方式分布"); - await user.click(screen.getByRole("button", { name: "发送" })); - await screen.findByText("已为你生成 SQL,并展示结果。"); - await user.click(screen.getByRole("button", { name: "展开 SQL 详情" })); - expect(screen.getByText("运行详情回填中,请稍候...")).toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: /处理过程/ })); + await waitFor(() => { + expect(screen.getByText("正在加载该轮处理轨迹...")).toBeInTheDocument(); + }); if (resolveRun) { resolveRun(); } }, 15000); - it("shows phase-b retrieval stages and run detail summary signals for the same run", async () => { + it("shows phase-b retrieval stages without surfacing debug detail panels for the same run", async () => { const user = userEvent.setup(); const phaseBRun = createMockRun({ trace: { @@ -799,18 +809,15 @@ describe("ChatPanel", () => { expect(mockStreamMessageEvents).toHaveBeenCalled(); }); - await user.click(screen.getByRole("button", { name: "展开思考过程" })); - expect(screen.getByText("知识检索")).toBeInTheDocument(); - expect(screen.getByText("意图规划")).toBeInTheDocument(); - expect(screen.getByText("语义检索构建")).toBeInTheDocument(); - - await user.click(screen.getByRole("button", { name: "展开 SQL 详情" })); - expect(screen.getByText("运行详情")).toBeInTheDocument(); - expect(screen.getByText("degrade_reason:retrieval_timeout")).toBeInTheDocument(); - expect(screen.getByText("semantic_registry_degraded")).toBeInTheDocument(); - expect(screen.getByText("运行 ID:run-1")).toBeInTheDocument(); + await user.click(screen.getByRole("button", { name: /处理过程/ })); + expect(screen.getByText(/知识检索/)).toBeInTheDocument(); + expect(screen.getByText(/意图规划/)).toBeInTheDocument(); + expect(screen.getByText(/语义检索构建/)).toBeInTheDocument(); + expect(screen.queryByText("degrade_reason:retrieval_timeout")).not.toBeInTheDocument(); + expect(screen.queryByText("semantic_registry_degraded")).not.toBeInTheDocument(); + expect(screen.queryByText("运行详情")).not.toBeInTheDocument(); expect( - screen.getByRole("link", { name: "设置 / RAG 运行与记忆治理" }) - ).toHaveAttribute("href", "/settings?tab=rag&runId=run-1"); + screen.queryByRole("link", { name: "设置 / RAG 运行与记忆治理" }) + ).not.toBeInTheDocument(); }); }); diff --git a/apps/frontend/tests/unit/chat-sql-preview.integration.spec.tsx b/apps/frontend/tests/unit/chat-sql-preview.integration.spec.tsx index a8e87ac..00b52ca 100644 --- a/apps/frontend/tests/unit/chat-sql-preview.integration.spec.tsx +++ b/apps/frontend/tests/unit/chat-sql-preview.integration.spec.tsx @@ -137,7 +137,22 @@ describe("chat to sql preview integration", () => { messages: createMockMessages(), latestRun: createMockRun({ sql: "SELECT * FROM orders LIMIT 20", - explanation: "返回最近 20 条订单。" + explanation: "返回最近 20 条订单。", + delivery: { + answer: { + text: "返回最近 20 条订单。", + status: "executionResult", + provider: "mock" + }, + artifact: { + sql: "SELECT * FROM orders LIMIT 20", + rowCount: 20, + hasError: false, + summary: { + text: "返回最近 20 条订单。" + } + } + } }) }); mockGetRun.mockResolvedValue(createMockRun()); @@ -148,7 +163,7 @@ describe("chat to sql preview integration", () => { vi.clearAllMocks(); }); - it("updates sql preview after successful message submission", async () => { + it("updates sql evidence tab after successful message submission", async () => { const user = userEvent.setup(); render(); @@ -160,7 +175,7 @@ describe("chat to sql preview integration", () => { expect(mockStreamMessageEvents).toHaveBeenCalled(); }); - await user.click(screen.getByRole("button", { name: "展开 SQL 详情" })); + await user.click(screen.getByRole("tab", { name: /view sql/i })); expect(screen.getByText("SELECT * FROM orders LIMIT 20")).toBeInTheDocument(); }); }); diff --git a/apps/frontend/tests/unit/chatbi-result-panel.spec.tsx b/apps/frontend/tests/unit/chatbi-result-panel.spec.tsx index 44c2664..f72ba03 100644 --- a/apps/frontend/tests/unit/chatbi-result-panel.spec.tsx +++ b/apps/frontend/tests/unit/chatbi-result-panel.spec.tsx @@ -1,13 +1,12 @@ import { render, screen } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { describe, expect, it, vi } from "vitest"; +import { describe, expect, it } from "vitest"; import { ChatBIResultPanel } from "@/components/chat/chatbi-result-panel"; import { createMockRun } from "./fixtures"; describe("ChatBIResultPanel", () => { - it("supports Answer/View SQL/Chart partition switching and keeps table evidence in Answer", async () => { + it("supports Answer/View SQL/Chart partition switching and keeps SQL user-facing", async () => { const user = userEvent.setup(); - const onRequestSqlDetails = vi.fn(); const run = createMockRun({ delivery: { @@ -61,7 +60,6 @@ describe("ChatBIResultPanel", () => { run={run} streamDelivery={undefined} runId={run.runId} - onRequestSqlDetails={onRequestSqlDetails} /> ); @@ -81,10 +79,13 @@ describe("ChatBIResultPanel", () => { await user.click(sqlTab); expect(sqlTab).toHaveAttribute("aria-selected", "true"); - expect(screen.getByText(/View SQL 为次级证据入口/i)).toBeInTheDocument(); - - await user.click(screen.getByRole("button", { name: "打开运行详情" })); - expect(onRequestSqlDetails).toHaveBeenCalledTimes(1); + expect(screen.getByText("用于生成当前回答的 SQL 证据。")).toBeInTheDocument(); + expect( + screen.getByText(/select month, revenue from revenue_monthly/i) + ).toBeInTheDocument(); + expect( + screen.queryByRole("button", { name: "打开运行详情" }) + ).not.toBeInTheDocument(); }); it("renders metric chart for metric artifacts", async () => { @@ -218,6 +219,7 @@ describe("ChatBIResultPanel", () => { const sqlTab = screen.getByRole("tab", { name: /view sql/i }); expect(sqlTab).toHaveAttribute("aria-selected", "true"); + expect(screen.getByText("用于生成当前回答的 SQL 证据。")).toBeInTheDocument(); expect(screen.getByText(/select \* from metrics_daily/i)).toBeInTheDocument(); }); }); diff --git a/apps/frontend/tests/unit/chatbi-result-tabs.spec.tsx b/apps/frontend/tests/unit/chatbi-result-tabs.spec.tsx index a2b6d4c..5c0286c 100644 --- a/apps/frontend/tests/unit/chatbi-result-tabs.spec.tsx +++ b/apps/frontend/tests/unit/chatbi-result-tabs.spec.tsx @@ -28,10 +28,14 @@ describe("ChatBIResultTabs", () => { const answerTab = screen.getByRole("tab", { name: /answer/i }); const sqlTab = screen.getByRole("tab", { name: /view sql/i }); const chartTab = screen.getByRole("tab", { name: /chart/i }); + const tabList = screen.getByTestId("chatbi-result-tablist"); expect(answerTab).toHaveAttribute("aria-selected", "true"); expect(sqlTab).toHaveAttribute("aria-selected", "false"); expect(chartTab).toHaveAttribute("aria-selected", "false"); + expect(tabList).toHaveClass("grid-cols-3"); + expect(tabList).toHaveClass("overflow-hidden"); + expect(tabList).not.toHaveClass("overflow-x-auto"); await user.click(chartTab); expect(chartTab).toHaveAttribute("aria-selected", "true"); diff --git a/apps/frontend/tests/unit/run-visibility-mapper.spec.ts b/apps/frontend/tests/unit/run-visibility-mapper.spec.ts index 6fb9283..634b170 100644 --- a/apps/frontend/tests/unit/run-visibility-mapper.spec.ts +++ b/apps/frontend/tests/unit/run-visibility-mapper.spec.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest"; import { mergeRunThinkingSteps, normalizeDeliveryContract, + projectStreamEvent, resolveVisibleDelivery, transitionRunVisibilityStatus } from "@/components/chat/run-visibility-mapper"; @@ -281,9 +282,27 @@ describe("run-visibility-mapper", () => { expect(transitionRunVisibilityStatus("success", "loading")).toBe("success"); expect(transitionRunVisibilityStatus("error", "loading")).toBe("error"); expect(transitionRunVisibilityStatus("empty", "loading")).toBe("empty"); + expect(transitionRunVisibilityStatus("cancelled", "loading")).toBe("cancelled"); expect(transitionRunVisibilityStatus(undefined, "loading")).toBe("loading"); }); + it("projects USER_CANCELLED stream errors to cancelled visibility", () => { + const projection = projectStreamEvent({ + type: "error", + runId: "run-1", + sessionId: "session-1", + at: "2026-04-10T00:00:00.000Z", + data: { + code: "USER_CANCELLED", + message: "用户已停止本轮生成。", + details: null + } + }); + + expect(projection.terminal).toBe(true); + expect(projection.visibilityStatus).toBe("cancelled"); + }); + it("merges sync and stream thinking steps by stepId and preserves stream stage metadata", () => { const merged = mergeRunThinkingSteps( [ diff --git a/apps/frontend/vitest.config.ts b/apps/frontend/vitest.config.ts index 5a6ff69..66a090f 100644 --- a/apps/frontend/vitest.config.ts +++ b/apps/frontend/vitest.config.ts @@ -4,7 +4,13 @@ import { fileURLToPath } from "node:url"; export default defineConfig({ resolve: { alias: { - "@": fileURLToPath(new URL("./src", import.meta.url)) + "@": fileURLToPath(new URL("./src", import.meta.url)), + "@text2sql/chat-stream-protocol": fileURLToPath( + new URL("../../packages/chat-stream-protocol/dist/index.js", import.meta.url) + ), + "@text2sql/chat-stream-protocol/client": fileURLToPath( + new URL("../../packages/chat-stream-protocol/dist/client.js", import.meta.url) + ) } }, test: { diff --git a/assets/readme-platform-overview.png b/assets/readme-platform-overview.png new file mode 100644 index 0000000..7943b66 Binary files /dev/null and b/assets/readme-platform-overview.png differ diff --git a/assets/screenshots/01-modeling-erd-workbench.png b/assets/screenshots/01-modeling-erd-workbench.png new file mode 100644 index 0000000..7f5cc5d Binary files /dev/null and b/assets/screenshots/01-modeling-erd-workbench.png differ diff --git a/assets/screenshots/02-datasource-create-wizard.png b/assets/screenshots/02-datasource-create-wizard.png new file mode 100644 index 0000000..4e8a03b Binary files /dev/null and b/assets/screenshots/02-datasource-create-wizard.png differ diff --git a/assets/screenshots/03-chat-answer-result.png b/assets/screenshots/03-chat-answer-result.png new file mode 100644 index 0000000..2e6284f Binary files /dev/null and b/assets/screenshots/03-chat-answer-result.png differ diff --git a/assets/screenshots/04-chat-sql-evidence.png b/assets/screenshots/04-chat-sql-evidence.png new file mode 100644 index 0000000..73a5b3e Binary files /dev/null and b/assets/screenshots/04-chat-sql-evidence.png differ diff --git a/assets/screenshots/05-chat-chart-result.png b/assets/screenshots/05-chat-chart-result.png new file mode 100644 index 0000000..56e8a77 Binary files /dev/null and b/assets/screenshots/05-chat-chart-result.png differ diff --git "a/docs/leran/A2A\345\215\217\350\256\256.md" "b/docs/leran/A2A\345\215\217\350\256\256.md" new file mode 100644 index 0000000..34a52cf --- /dev/null +++ "b/docs/leran/A2A\345\215\217\350\256\256.md" @@ -0,0 +1,798 @@ +# Text2SQL 项目前后端 A2A 流式协议说明 + +本文基于当前仓库实现,解释聊天主链路里这套“前端 Agent UI <-> 后端 Agent Runtime”协议到底如何工作。 + +这里的 “A2A” 是本项目内部叫法,重点不是多 Agent 联邦通信,而是: + +- 前端如何把用户问题送入后端运行时 +- 后端如何把运行中的结构化事件持续推回前端 +- 前端如何把这些事件投影成聊天气泡、thinking panel、tool 过程和最终 delivery + +底层传输不是 WebSocket,而是: + +`HTTP POST + SSE(text/event-stream)` + +## 1. 一句话先记住 + +本项目的主链路不是“发消息,然后等一整段回答返回”,而是: + +`message/contextEnvelope 入站 -> backend 创建 run -> SSE 持续推送 ChatStreamEvent -> frontend 分别消费 text/state/tool/finish/error -> 回读持久化 messages/run 对齐最终事实` + +这是一套“结构化事件流协议”,不是“纯文本流协议”。 + +## 2. 读代码顺序 + +如果你是第一次读这条链路,建议按这个顺序看。 + +协议合同与工具: + +- `packages/shared-types/src/api.ts` +- `packages/chat-stream-protocol/src/types.ts` +- `packages/chat-stream-protocol/src/server.ts` +- `packages/chat-stream-protocol/src/client.ts` +- `packages/chat-stream-protocol/src/ui.ts` + +后端入口: + +- `apps/backend/src/modules/conversation/chat/chat.controller.ts` +- `apps/backend/src/modules/conversation/chat/chat.service.ts` +- `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` +- `apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts` +- `apps/backend/src/modules/llm/llm-gateway.interface.ts` + +前端入口: + +- `apps/frontend/src/lib/api-client.ts` +- `apps/frontend/src/components/chat/assistant-runtime.ts` +- `apps/frontend/src/components/chat-panel.tsx` +- `apps/frontend/src/components/chat/assistant-thread.tsx` +- `apps/frontend/src/components/chat/run-visibility-mapper.ts` + +## 3. 责任分层 + +当前实现把协议责任拆成了三层。 + +### 3.1 `packages/shared-types` + +这里是协议的单一事实源,定义: + +- `ChatStreamEventType` +- `ChatStreamEventData` +- `ChatStreamEvent` +- `DeliveryContract` +- `AgentRunResponse` + +也就是说,事件 envelope 和 finish 里的 delivery 合同都以这里为准。 + +### 3.2 `packages/chat-stream-protocol` + +这里是协议工具层,不再让前后端各自维护一套 parser / writer / terminal 判定逻辑。 + +它负责: + +- 服务端写 SSE:`serializeSseEvent`、`writeSseEvent`、`writeErrorEvent` +- 客户端读 SSE:`readSseStream` +- envelope 校验:`validateChatStreamEnvelope` +- terminal 判定:`isTerminalEvent` +- 文本增量拼接:`collectTextDelta` +- UI 投影:`projectStreamEvent` + +### 3.3 backend / frontend 业务代码 + +业务层只消费协议,不再重新定义协议。 + +- backend 负责“什么时候 emit 什么事件” +- frontend 负责“收到事件后如何更新 UI 状态” + +## 4. 总体时序 + +```mermaid +flowchart LR + A["Frontend input"] --> B["POST /api/v1/sessions/:sessionId/messages/stream"] + B --> C["ChatController"] + C --> D["Text2SQLWorkflowRunner.runStream"] + D --> E["RunV2LangGraphStage"] + E --> F["emit start / state / tool-* / text-delta"] + F --> G["SSE response"] + G --> H["api-client.readSseStream"] + H --> I["assistant-runtime"] + I --> J["ChatPanel projectStreamEvent"] + J --> K["Assistant bubble / Thinking panel / Delivery preview"] + D --> L["EnrichDeliveryStage + PersistRunStage"] + L --> M["GET messages / GET run"] + M --> K +``` + +## 5. 请求如何进入后端 + +### 5.1 前端真正发请求的地方 + +流式请求从这里发起: + +- `apps/frontend/src/lib/api-client.ts` +- `streamMessageEvents(sessionId, message, abortSignal, contextEnvelope)` + +真实请求目标: + +```text +/api/v1/sessions/:sessionId/messages/stream +``` + +请求方式仍然是普通 `fetch POST`,不是 `EventSource GET`。 + +### 5.4 用户停止生成如何传播 + +前端 stop 不是只停止浏览器读流,而是沿着同一条运行链路把取消信号向后传递: + +- `apps/frontend/src/components/chat/assistant-composer.tsx` 提供停止按钮 +- `apps/frontend/src/components/chat/assistant-runtime.ts` 为当前流维护 `AbortController` +- `apps/frontend/src/lib/api-client.ts` 把 `abortSignal` 传给 `fetch` +- `apps/backend/src/modules/conversation/chat/chat.controller.ts` 监听 request / response close,并创建 run-scoped `AbortController` +- `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` 把 `abortSignal` 继续传给 LangGraph runtime +- `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts` 在阶段边界做 cooperative abort check +- `apps/backend/src/modules/conversation/nodes/generate-sql.node.ts` / `agent/sql/sql-generation.service.ts` / `modules/llm/llm-gateway.service.ts` 最终把 signal 传到 provider stream + +第一版取消语义不新增公开 stream event type,而是继续使用既有 terminal `error` 事件,并约定: + +```ts +{ + type: "error", + data: { + code: "USER_CANCELLED", + message: "用户已停止本轮生成。" + } +} +``` + +前端把它投影为“已停止生成”,保留已经产生的文本和 thinking/tool 过程,而不是把它显示成普通 provider failure。 + +### 5.2 请求体 + +当前请求体合同来自 `SendMessageRequest`: + +```ts +{ + message: string, + contextEnvelope?: { + metricDefinition?, + timeRange?, + entityMappings?, + mustIncludeTables?, + mustExcludeTables?, + pinnedTables?, + pinnedColumns?, + businessConstraints? + } +} +``` + +真正最核心的用户输入字段仍然是: + +```ts +message: "用户自然语言问题" +``` + +### 5.3 额外请求头 + +前端还会附带这些 header: + +- `x-user-role` +- `x-user-id` +- `x-workspace-id`(可选) + +这些不是 SSE 协议本身的一部分,但它们会影响后端的 actor / workspace 上下文。 + +## 6. 为什么这是 SSE,不是 WebSocket + +后端流式接口在 `chat.controller.ts` 里直接设置: + +```ts +Content-Type: text/event-stream; charset=utf-8 +Cache-Control: no-cache, no-transform +Connection: keep-alive +``` + +然后通过 `writeSseEvent` 把事件一条条写入响应。 + +标准 framing 形式是: + +```text +event: +data: + +``` + +也就是: + +```ts +`event: ${eventType}\ndata: ${JSON.stringify(event)}\n\n` +``` + +注意两点: + +1. 这条路的 HTTP 状态码主要表示“传输是否建立成功”,不表示业务是否成功。 +2. 业务失败通常通过流内 `error` 事件返回,而不是依赖最终 HTTP 4xx/5xx。 + +## 7. 协议 envelope 长什么样 + +所有流事件统一都是这个壳子: + +```ts +interface ChatStreamEvent { + type: ChatStreamEventType; + runId: string; + sessionId: string; + at: string; + data: ChatStreamEventData; +} +``` + +这 5 个字段是协议最关键的骨架: + +- `type` +- `runId` +- `sessionId` +- `at` +- `data` + +前端解析时会校验: + +- payload 必须是对象 +- `type` 必须是受支持的事件类型 +- `runId` / `sessionId` / `at` 必须是非空字符串 +- SSE `event:` 行上的名字必须和 JSON 里的 `type` 一致 + +这也是 `packages/chat-stream-protocol` 存在的意义之一:避免 writer / parser 不一致。 + +## 8. 当前事件 taxonomy + +协议事件类型固定为: + +```ts +type ChatStreamEventType = + | "start" + | "text-delta" + | "tool-call" + | "tool-result" + | "tool-error" + | "state" + | "finish" + | "error"; +``` + +协议工具层定义的 terminal 事件只有两类: + +- `finish` +- `error` + +其中“用户主动停止”在当前实现中仍属于 `error` terminal,只是 `code` 固定为 `USER_CANCELLED`。 + +### 8.1 `start` + +表示本次 run 已经建立,常用于: + +- 给 UI 创建一个新的活动 run +- 把 run 标成 loading +- 初始化 thinking 容器 + +`data` 结构: + +```ts +{ requestId: string | null } +``` + +### 8.2 `text-delta` + +表示回答文本的增量输出。 + +`data` 结构: + +```ts +{ text: string } +``` + +前端不会把每条 delta 当作独立消息,而是用 `collectTextDelta` 持续拼接成当前 assistant 文本。 + +### 8.3 `tool-call` + +表示模型触发了一次工具调用。 + +`data` 结构核心字段: + +```ts +{ + toolName: string; + toolCallId: string; + input?: unknown; + title?: string; + stage?: ReasoningStage; + summary?: string; +} +``` + +### 8.4 `tool-result` + +表示工具返回成功结果。 + +核心字段与 `tool-call` 类似,只是把 `input` 换成 `output`。 + +### 8.5 `tool-error` + +表示工具调用失败。 + +核心字段: + +```ts +{ + toolName: string; + toolCallId: string; + message: string; + title?: string; + stage?: ReasoningStage; + summary?: string; +} +``` + +### 8.6 `state` + +这是当前实现里最容易被忽视、但也最重要的一类事件。 + +它不是最终回答文本,而是 runtime 的过程状态。 + +`data` 结构核心字段: + +```ts +{ + node: string; + status: "success" | "failed" | "skipped"; + stepId?: string; + sequence?: number; + lifecycle?: "running" | "completed" | "failed" | "skipped"; + detail: string; + stage?: ReasoningStage; + title?: string; + at?: string; + startedAt?: string; + endedAt?: string; + durationMs?: number; + inputSummary?: string; + outputSummary?: string; + errorSummary?: string; + v2?: { + stageArtifact?: Text2SqlV2StageArtifact; + runtimePlan?: { ... }; + planLedger?: SemanticPlanLedgerSummaryV1; + }; +} +``` + +重点是:`state` 已经不是“随便打一条日志”,而是当前 v2 runtime 对外暴露的安全摘要层。 + +### 8.7 `finish` + +表示本次流式 run 正常走到了结尾,并带上最终结果摘要。 + +`data` 结构: + +```ts +{ + status: RunStatus; + rowCount: number; + delivery?: DeliveryContract; +} +``` + +这里的 `delivery` 很关键,它和同步接口、持久化 run 里的 `delivery` 是同一类合同。 + +### 8.8 `error` + +表示流内失败信息。 + +`data` 结构: + +```ts +{ + code?: string; + message: string; + details?: Record | null; +} +``` + +它通常用于: + +- 运行时失败 +- 只读安全拒绝 +- 其他已经无法继续正常流式交互的情况 + +## 9. backend 端如何产生这些事件 + +主 orchestration 入口在: + +- `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` + +当前流式主路径可以概括成: + +1. `PrepareRunStage` 准备 session、runId、question、actor、上下文 +2. emit `start` +3. 调 `RunV2LangGraphStage.runStream(...)` +4. 把 LLM 侧流式事件映射成 `text-delta / tool-*` +5. 把步骤事件映射成 `state` +6. 汇总 `trace.toolCalls` +7. `EnrichDeliveryStage` 生成最终 delivery +8. emit `finish` +9. `PersistRunStage` 持久化 +10. `PostRunHooksStage` 做后置流程 + +## 10. LLM 事件和协议事件之间的关系 + +LLM gateway 对外的原始流式事件更底层: + +```ts +type LlmGatewayStreamEvent = + | { type: "text-delta"; text: string } + | { type: "tool-call"; toolName: string; toolCallId: string; input?: unknown } + | { type: "tool-result"; toolName: string; toolCallId: string; output?: unknown } + | { type: "tool-error"; toolName: string; toolCallId: string; message: string }; +``` + +这些事件会在: + +- `apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts` + +里被映射成统一的 `ChatStreamEvent.data`。 + +映射后做了两件事: + +1. 统一前端消费面,不暴露 provider 细节 +2. 同步补齐 `trace.toolCalls`,方便持久化与 replay + +## 11. `state` 事件为什么是这条链路的核心 + +如果只盯 `text-delta`,你看到的只是“助手说了什么”。 + +如果盯 `state`,你才能看到: + +- 当前处在哪个 runtime node +- 当前步骤的 lifecycle +- 每一步的 detail / inputSummary / outputSummary / errorSummary +- v2 阶段产物摘要 `v2.stageArtifact` +- 运行时计划摘要 `v2.runtimePlan` +- plan ledger 摘要 `v2.planLedger` + +因此当前实现实际上是双轨输出: + +- `text-delta`:用户能读的回答文本 +- `state` + `tool-*`:UI 和调试层能读的过程状态 + +## 12. frontend 如何读这条流 + +### 12.1 协议解析 + +前端不再手写 SSE parser,而是: + +- `apps/frontend/src/lib/api-client.ts` +- `streamMessageEvents(...)` +- `readSseStream(response.body)` + +也就是说,前端解析逻辑和 backend 写出逻辑已经通过 `packages/chat-stream-protocol` 对齐。 + +### 12.2 assistant runtime 如何消费 + +`apps/frontend/src/components/chat/assistant-runtime.ts` 是 assistant-ui 对接层。 + +它会: + +1. 从线程消息里提取最新一条 user message +2. 调 `streamMessageEvents(...)` +3. 收到 `text-delta` 就用 `collectTextDelta(...)` 拼接文本 +4. 把每条事件透传给上层回调 +5. 把当前临时 assistant 文本以 `yield` 形式交给 assistant-ui +6. 在消息 metadata 里挂上 `runId` + +所以 assistant 气泡看到的是“累计文本”,不是单次 delta。 + +## 13. frontend 如何把事件翻译成页面状态 + +真正的页面状态管理在: + +- `apps/frontend/src/components/chat-panel.tsx` + +这里不会直接 everywhere 手写 `switch(event.type)` 更新 UI 细节,而是先通过: + +- `projectStreamEvent(event)` + +把协议事件投影成 UI 更容易消费的结构: + +```ts +{ + thinkingStep, + textStarted, + terminal, + visibilityStatus?, + deliveryPatch? +} +``` + +然后 `ChatPanel` 再维护这些状态: + +- `streamThinkingByRunId` +- `streamDeliveryByRunId` +- `streamTextStartedByRunId` +- `runVisibilityByRunId` +- `activeStreamRunId` + +### 13.1 `start` 到来时 + +前端会: + +- 把 run 设为 loading +- 设置 `activeStreamRunId` +- 初始化该 run 的 thinking steps +- 清掉上一次残留的 stream delivery / textStarted + +### 13.2 `text-delta` 到来时 + +前端会: + +- 标记该 run 已经开始输出文本 +- assistant-ui 继续把聚合后的文本显示在气泡里 + +### 13.3 `state` / `tool-*` 到来时 + +前端会把它们转换成 thinking steps,并追加到 thinking panel。 + +### 13.4 `finish` 到来时 + +前端会: + +- 记录 `deliveryPatch` +- 更新 run 的可见状态 +- 结束当前活动 run + +### 13.5 `error` 到来时 + +前端会: + +- 把流视为终止 +- 进入错误态 +- 走错误回收逻辑 + +## 14. 当前实现里一个很重要的细节 + +`readSseStream(...)` 默认会把 `error` 当成 terminal 事件。 + +这意味着: + +- 一旦前端读到了 `error` +- generator 会停止继续消费后续 SSE block + +而 backend 的当前实现里,某些失败路径可能会先 emit `error`,随后仍然 emit `finish` 并继续持久化 run。 + +所以不要把“前端是否读到 `finish`”理解成“后端是否真的完成了最终持久化”。 + +当前设计的真正保障是: + +- 正常结束:`onRunFinish` 回读持久化数据 +- 错误结束:`onRunError` 也会回读 session messages + +也就是说,错误场景下前端依赖“回读持久化事实”,而不是依赖继续消费 `error` 之后的流事件。 + +## 15. 为什么 finish 之后还要回读持久化 run + +这是这套协议最关键的设计决策之一。 + +流式事件只负责: + +- 实时体验 +- 中间状态 +- 临时 delivery patch + +真正的最终事实来源仍然是持久化 run / messages。 + +当前前端在 `onRunFinish` 里会做三件事: + +1. `loadMessages(sessionId, datasourceId)` +2. `ensureRunLoaded(runId)` +3. `refreshSessionBuckets()` + +目的很明确: + +- 把页面展示和数据库事实对齐 +- 拿到完整的持久化 `run.trace` / `run.delivery` +- 支持后续 replay、save-as-view、审计 + +## 16. assistant thread 是怎么把“流状态”和“持久化状态”拼在一起的 + +在: + +- `apps/frontend/src/components/chat/assistant-thread.tsx` + +里,最终展示并不是只看 stream,也不是只看 persisted run,而是做了 merge: + +- `run.trace.steps` 和 `streamThinkingByRunId` 会合并 +- `run.delivery` 和 `streamDeliveryByRunId` 会做可见层选择 +- `runVisibilityByRunId` 和 `activeStreamRunId` 会决定当前是否还在 loading + +这就是为什么当前 UI 能同时做到: + +- 流式展示实时 thinking +- finish 后切回持久化事实 +- 历史消息重新打开时仍能复现 run 详情 + +## 17. 一个最小完整时序 + +下面是一轮典型交互的抽象顺序。 + +### 17.1 前端发送 + +```ts +POST /api/v1/sessions/s1/messages/stream +{ + "message": "上周哪个门店的退货率最高?", + "contextEnvelope": { + "mustIncludeTables": ["orders", "refunds"] + } +} +``` + +### 17.2 backend 建立 run + +```json +{ + "type": "start", + "runId": "run_1", + "sessionId": "s1", + "at": "2026-04-30T10:00:00.000Z", + "data": { "requestId": "req_1" } +} +``` + +### 17.3 runtime 推进 + +```json +{ + "type": "state", + "runId": "run_1", + "sessionId": "s1", + "at": "2026-04-30T10:00:01.000Z", + "data": { + "node": "semantic-plan", + "status": "success", + "detail": "已生成语义计划", + "lifecycle": "completed" + } +} +``` + +### 17.4 工具调用 + +```json +{ + "type": "tool-call", + "runId": "run_1", + "sessionId": "s1", + "at": "2026-04-30T10:00:02.000Z", + "data": { + "toolName": "runReadOnlySql", + "toolCallId": "tool_1", + "summary": "select ..." + } +} +``` + +### 17.5 文本增量 + +```json +{ + "type": "text-delta", + "runId": "run_1", + "sessionId": "s1", + "at": "2026-04-30T10:00:03.000Z", + "data": { + "text": "根据上周数据,退货率最高的是华东一区门店。" + } +} +``` + +### 17.6 最终结果 + +```json +{ + "type": "finish", + "runId": "run_1", + "sessionId": "s1", + "at": "2026-04-30T10:00:05.000Z", + "data": { + "status": "executionResult", + "rowCount": 10, + "delivery": { + "...": "..." + } + } +} +``` + +### 17.7 前端回读 + +```text +GET /api/v1/sessions/:sessionId/messages +GET /api/v1/runs/:runId +``` + +## 18. 排查问题时最值得盯的字段 + +如果你是为了排障,优先看这些点。 + +请求侧: + +- `message` +- `contextEnvelope` +- `x-user-role` +- `x-user-id` +- `x-workspace-id` + +事件 envelope: + +- `type` +- `runId` +- `sessionId` +- `at` + +文本流: + +- `event.type === "text-delta"` +- `event.data.text` + +过程流: + +- `event.type === "state"` +- `event.data.node` +- `event.data.lifecycle` +- `event.data.detail` +- `event.data.v2.stageArtifact` +- `event.data.v2.runtimePlan` +- `event.data.v2.planLedger` + +工具流: + +- `event.type === "tool-call"` +- `event.type === "tool-result"` +- `event.type === "tool-error"` +- `toolName` +- `toolCallId` + +最终结果: + +- `event.type === "finish"` +- `event.data.status` +- `event.data.rowCount` +- `event.data.delivery` + +错误: + +- `event.type === "error"` +- `event.data.code` +- `event.data.message` + +持久化对齐: + +- `run.trace.steps` +- `run.trace.toolCalls` +- `run.delivery` +- `delivery.evidence.runId` + +## 19. 最后总结 + +当前项目里的 A2A 流式协议,可以用下面这句话概括: + +`frontend 用 POST 把 message/contextEnvelope 送进 session run,backend 用统一 ChatStreamEvent envelope 通过 SSE 推送 start/state/tool/text/finish/error,frontend 再通过 protocol toolkit 把这些事件投影成 assistant 文本、thinking steps、delivery patch,并在结束后回读持久化 run 对齐最终事实。` + +如果只记最小心智模型,可以记成: + +```text +message in +-> run start +-> state/tool/text 流式返回 +-> finish 或 error +-> reload persisted messages/run +``` diff --git a/docs/plans/2026-04-30-001-feat-chat-stream-protocol-toolkit-plan.md b/docs/plans/2026-04-30-001-feat-chat-stream-protocol-toolkit-plan.md new file mode 100644 index 0000000..ad9a4ae --- /dev/null +++ b/docs/plans/2026-04-30-001-feat-chat-stream-protocol-toolkit-plan.md @@ -0,0 +1,604 @@ +--- +title: feat: Chat Stream Protocol Toolkit and Cancellable Stream Runtime +type: feat +status: completed +date: 2026-04-30 +updated: 2026-05-01 +origin: docs/brainstorms/2026-04-30-chat-stream-protocol-toolkit-requirements.md +--- + +# feat: Chat Stream Protocol Toolkit and Cancellable Stream Runtime + +## Overview + +`packages/chat-stream-protocol` already exists and now owns the reusable SSE protocol layer: envelope creation, SSE parsing/framing, terminal guards, text aggregation, framework-light UI projections, and test fixtures. This refresh extends the original package-extraction plan into the remaining runtime work: make chat streams cancellable end-to-end, keep UI thinking/tool progress grounded in stream events, and document the `claude-code`-inspired event/message layering without copying its transport or full transcript system. + +The implementation should preserve the external stream contract: `POST /api/v1/sessions/:sessionId/messages/stream` still emits `ChatStreamEvent` envelopes with `type/runId/sessionId/at/data`, and the canonical event taxonomy remains `start/text-delta/tool-call/tool-result/tool-error/state/finish/error` unless a later plan explicitly introduces a new terminal type. + +## Problem Frame + +The initial duplication problem is mostly addressed: backend stream tests and gateway smoke already consume `@text2sql/chat-stream-protocol`, and frontend/backend code already import helpers such as `readSseStream`, `writeSseEvent`, `createChatStreamEventEnvelope`, `collectTextDelta`, and `projectStreamEvent`. + +The remaining gap is runtime control and contributor understanding. The frontend can pass an `AbortSignal` into `fetch`, but backend `chat.controller -> StreamMessageUsecase -> Text2SQLWorkflowRunner -> RunV2LangGraphStage -> Text2SqlV2LangGraphRunnerService -> GenerateSqlNode -> SqlGenerationService -> LlmGatewayService` does not yet carry a user cancellation signal to the LLM provider or tool execution path. This means a user-visible stop action can abort the browser reader while backend work may continue consuming model/tool resources. The UI also needs a clear stopped/cancelled state that preserves already streamed text and thinking steps. + +The plan uses `claude-code` only as an architectural reference for two ideas: partial runtime events should be projected into user-visible messages rather than flattened into final text, and interrupt/cancel should be a first-class control signal. It does not migrate this project to WebSocket/ACP or copy `claude-code`'s full message taxonomy. + +## Requirements Trace + +- R1-R4. Keep `packages/chat-stream-protocol` as the internal protocol toolkit while `packages/shared-types` remains the canonical `ChatStreamEvent` schema owner. +- R5-R10. Preserve and harden server/client helpers for envelope creation, SSE framing, readable-stream parsing, terminal checks, malformed/partial/multi-block behavior, and text delta collection. +- R11-R16. Keep UI adapters framework-light and use stream events as the source of truth for thinking/tool/process visibility. +- R17-R24. Keep test helpers and first migration points aligned with existing backend e2e/integration, frontend runtime, and gateway smoke consumers. +- R25-R30. Add user interrupt/cancellation semantics from frontend stop action through backend route, workflow, LangGraph runner, SQL generation, and LLM gateway/provider calls. +- R31-R34. Add learning documentation that explains transport framing, protocol envelope, UI projection, cancellation, and why thinking UI is public workflow/tool state rather than model private chain-of-thought. +- R35-R40. Capture the `claude-code` event/message layering as a reference model while keeping the first implementation scoped to existing Text2SQL stream events and real consumers. +- R41-R43. Keep toolkit exports small, update stream standards/docs, and remove non-portable absolute links from learning docs. + +## Scope Boundaries + +- Do not change the public stream route, HTTP method, or envelope fields. +- Do not introduce a `cancelled` event type in this plan. Use existing `error` terminal semantics with a stable `USER_CANCELLED` code unless implementation proves the existing taxonomy cannot express the state safely. +- Do not move Text2SQL stage catalog, LangGraph state, delivery business logic, toast/session orchestration, or React state machines into `packages/chat-stream-protocol`. +- Do not expose provider role taxonomy directly to UI. `UserMessage` carrying `tool_result` remains a provider-protocol teaching point, not a frontend event contract. +- Do not implement `SystemMessage`, `AttachmentMessage`, `TombstoneMessage`, or full `ToolUseSummaryMessage` transcript semantics unless a concrete Text2SQL stream consumer exists. +- Do not require WebSocket, EventSource-only GET streams, ACP, or `claude-code` transport architecture. +- Do not make database/Prisma schema changes. + +### Deferred to Separate Tasks + +- First-class `cancelled` terminal event: separate compatibility plan if product wants cancellation distinct from existing `error` terminals. +- General Agent transcript system: separate plan if future work needs system/attachment/tombstone/replay message semantics beyond Text2SQL stream projection. +- Tool-use summary persistence: separate plan if summary needs to become persisted run metadata rather than UI-only projection. + +## Context & Research + +### Relevant Code and Patterns + +- Protocol toolkit: `packages/chat-stream-protocol/src/types.ts`, `packages/chat-stream-protocol/src/server.ts`, `packages/chat-stream-protocol/src/client.ts`, `packages/chat-stream-protocol/src/ui.ts`, `packages/chat-stream-protocol/src/test.ts`, `packages/chat-stream-protocol/src/protocol.spec.ts` +- Shared schema owner: `packages/shared-types/src/api.ts`, `packages/shared-types/src/api.contract.spec.ts` +- Backend stream entry: `apps/backend/src/modules/conversation/chat/chat.controller.ts` +- Backend stream application path: `apps/backend/src/modules/conversation/chat/chat.service.ts`, `apps/backend/src/modules/conversation/chat/application/stream-message.usecase.ts`, `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` +- V2 runtime path: `apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts`, `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph-runner.service.ts`, `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts`, `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts` +- SQL/LLM stream path: `apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts`, `apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts`, `apps/backend/src/modules/llm/provider-router.service.ts`, `apps/backend/src/modules/llm/llm-gateway.interface.ts`, `apps/backend/src/modules/llm/llm-gateway.service.ts` +- Frontend stream path: `apps/frontend/src/lib/api-client.ts`, `apps/frontend/src/components/chat/assistant-runtime.ts`, `apps/frontend/src/components/chat/assistant-thread.tsx`, `apps/frontend/src/components/chat/assistant-composer.tsx`, `apps/frontend/src/components/chat-panel.tsx` +- Frontend projection/display: `apps/frontend/src/components/chat/run-visibility-mapper.ts`, `apps/frontend/src/components/chat/assistant-thinking-panel.tsx` +- Tests and smoke: `apps/backend/test/unit/llm-gateway.service.spec.ts`, `apps/backend/test/unit/text2sql-workflow-runner.spec.ts`, `apps/backend/test/e2e/chat-stream-api.spec.ts`, `apps/backend/test/integration/delivery-contract.spec.ts`, `apps/frontend/tests/unit/chat-panel.spec.tsx`, `apps/frontend/tests/unit/run-visibility-mapper.spec.ts`, `tests/smoke/nginx-dev-gateway-smoke.mjs` +- Standards: `docs/standards/llm-stream-tool-migration-spec.md`, `docs/standards/frontend-react-shadcn-spec.md` + +### Institutional Learnings + +- `docs/solutions/workflow-issues/frontend-vitest-hang-reactflow-oninit-mock-loop-2026-04-23.md`: frontend tests that wire callbacks to state setters can hang if mocks repeatedly trigger lifecycle callbacks. Apply this when testing stop/abort UI: prefer explicit user actions and one-shot async gates over effects that loop. +- `docs/solutions/workflow-issues/ce-compound-full-mode-workflow-context-pass-through-2026-04-23.md`: preserve evidence boundaries in docs. Apply this to learning docs by clearly separating verified local paths from `claude-code` analogy. + +### External References + +- No external research was needed. The work extends existing local patterns, and the relevant third-party surface (`ai` stream/generate `abortSignal`) is already used in `apps/backend/src/modules/llm/llm-gateway.service.ts`. + +## Key Technical Decisions + +- Keep `ChatStreamEvent` in `packages/shared-types`: `packages/chat-stream-protocol` remains helper owner, not schema owner. +- Reuse existing `error` terminal for user cancellation in the first implementation: emit or persist `code: "USER_CANCELLED"` where possible, avoiding taxonomy churn. +- Treat cancellation as a control plane, not just reader cleanup: the same abort reason must travel through route close listeners, workflow inputs, LangGraph stream options, SQL generation options, and LLM gateway options. +- Preserve partial output on cancel: the frontend should keep already streamed answer text and thinking steps, stop loading, and show a readable stopped/cancelled state. +- Keep provider-specific message roles behind backend assembly: `tool_result` as user-role provider input should be documented as an internal/provider constraint, not surfaced to the UI contract. +- Keep tool summary and tombstone as guarded future concepts: they can shape projection design, but they should not enter the protocol until there is a concrete stream fallback, replay correction, or collapsed-summary consumer. + +## Open Questions + +### Resolved During Planning + +- **Should cancellation add a new event type now?** No. Use existing `error` terminal with `USER_CANCELLED` first; defer `cancelled` event taxonomy to a separate compatibility plan. +- **Should `claude-code` message classes be copied?** No. Use its layering as a reference and map only real Text2SQL stream consumers. +- **Should external SSE/parser libraries be introduced?** No planning requirement. Existing toolkit helpers are already in place and tested. + +### Deferred to Implementation + +- Exact helper names for request-close abort controllers and signal composition can be chosen during implementation, provided the chain is testable. +- Whether `assistant-ui` exposes a built-in cancel control in the current version should be verified during implementation. If unavailable, use a local stop control wired to the runtime abort mechanism. +- Persisted cancellation shape should use the existing `SqlRun`/trace fields where possible; exact metadata placement depends on current persistence model constraints. +- Tool execution cancellation depth may vary by tool. The plan requires propagation and cooperative checks where the current tool boundary allows it; hard-killing non-cooperative operations is a separate concern. + +## Output Structure + +The package already follows this shape. New work should preserve it and add only narrowly scoped helpers/tests if needed. + +```text +packages/chat-stream-protocol/ + src/ + index.ts + types.ts + server.ts + client.ts + ui.ts + test.ts + protocol.spec.ts +``` + +## High-Level Technical Design + +> *This illustrates the intended approach and is directional guidance for review, not implementation specification. The implementing agent should treat it as context, not code to reproduce.* + +```mermaid +sequenceDiagram + participant UI as Frontend UI + participant Runtime as assistant-runtime/api-client + participant Controller as ChatController + participant Workflow as Text2SQLWorkflowRunner + participant Graph as LangGraph runtime + participant SQL as SqlGenerationService + participant LLM as LlmGatewayService + + UI->>Runtime: submit message + Runtime->>Controller: POST stream with AbortSignal + Controller->>Workflow: runStream(..., abortSignal) + Workflow->>Graph: runStream(..., abortSignal) + Graph->>SQL: generate SQL stream(..., abortSignal) + SQL->>LLM: streamText(..., merged timeout/user signal) + LLM-->>UI: text-delta/tool/state via SSE + UI->>Runtime: stop generation + Runtime->>Controller: abort fetch/readable stream + Controller->>Workflow: request close aborts run signal + Workflow->>Graph: cooperative cancellation + Graph->>SQL: abort-aware stage exits + SQL->>LLM: provider abort signal triggers + Controller-->>UI: error USER_CANCELLED if still writable +``` + +```mermaid +flowchart LR + Provider["Provider partial events
text/tool deltas"] --> Stream["ChatStreamEvent
project-level protocol"] + Stream --> RuntimeText["assistant message text
text-delta aggregation"] + Stream --> Thinking["thinking/tool steps
state/tool-* projection"] + Stream --> Terminal["finish/error terminal
visibility + delivery"] + Provider -. "provider role/tool_result rules" .-> BackendOnly["backend prompt assembly only"] +``` + +## Implementation Units + +- [x] **Unit 1: Protocol toolkit core** + +**Goal:** Keep the existing toolkit package as the protocol owner for helper code. + +**Requirements:** R1-R10, R17-R20, R41 + +**Dependencies:** None + +**Files:** +- Existing: `packages/chat-stream-protocol/package.json` +- Existing: `packages/chat-stream-protocol/src/types.ts` +- Existing: `packages/chat-stream-protocol/src/server.ts` +- Existing: `packages/chat-stream-protocol/src/client.ts` +- Existing: `packages/chat-stream-protocol/src/test.ts` +- Existing: `packages/chat-stream-protocol/src/protocol.spec.ts` + +**Approach:** +- Preserve the existing `types / server / client / ui / test` export grouping. +- Keep `packages/shared-types` as the only schema owner. +- Add only missing helpers discovered by later units; avoid widening the package into generic frontend/backend utilities. + +**Patterns to follow:** +- `packages/chat-stream-protocol/src/index.ts` +- `packages/chat-stream-protocol/src/protocol.spec.ts` + +**Test scenarios:** +- Happy path: round-trip serialized `start`, `text-delta`, and `finish` events through parser helpers. +- Edge case: partial chunks, empty/comment blocks, multi-line data blocks, and trailing incomplete buffers behave with stable parse results. +- Error path: event-name/payload-type mismatch and malformed JSON are reported by toolkit helpers. + +**Verification:** +- Toolkit tests pass and package exports remain small and grouped. + +- [x] **Unit 2: Backend SSE writer and event envelope adoption** + +**Goal:** Keep backend stream route and workflow event creation behind toolkit helpers. + +**Requirements:** R5-R7, R20-R24 + +**Dependencies:** Unit 1 + +**Files:** +- Existing/modify as needed: `apps/backend/src/modules/conversation/chat/chat.controller.ts` +- Existing/modify as needed: `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` +- Existing/modify as needed: `apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts` +- Test: `apps/backend/test/unit/text2sql-workflow-runner.spec.ts` +- Test: `apps/backend/test/e2e/chat-stream-api.spec.ts` + +**Approach:** +- Keep `Text2SqlStreamEventMapper` responsible for business mapping only. +- Keep `writeSseEvent`/`writeErrorEvent` as the transport boundary. +- Preserve current stream headers, route, status, and callback shape. + +**Patterns to follow:** +- `packages/chat-stream-protocol/src/server.ts` +- `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` + +**Test scenarios:** +- Happy path: `start -> state/text/tool -> finish` events share one run id and session id. +- Error path: route or workflow failure emits `error` with structured data. +- Integration: backend e2e stream contract still sees `text/event-stream` and valid envelopes. + +**Verification:** +- Backend stream tests no longer depend on duplicated writer/parser rules. + +- [x] **Unit 3: Frontend stream reader and UI projection adoption** + +**Goal:** Keep frontend stream reading and generic event projection behind toolkit helpers. + +**Requirements:** R8-R16, R20-R24 + +**Dependencies:** Unit 1 + +**Files:** +- Existing/modify as needed: `apps/frontend/src/lib/api-client.ts` +- Existing/modify as needed: `apps/frontend/src/components/chat/assistant-runtime.ts` +- Existing/modify as needed: `apps/frontend/src/components/chat/run-visibility-mapper.ts` +- Existing/modify as needed: `apps/frontend/src/components/chat-panel.tsx` +- Test: `apps/frontend/tests/unit/chat-panel.spec.tsx` +- Test: `apps/frontend/tests/unit/run-visibility-mapper.spec.ts` +- Test: `apps/frontend/tests/unit/chat-rag-sync-stream-consistency.spec.tsx` + +**Approach:** +- Keep `readSseStream`, `collectTextDelta`, and `projectStreamEvent` as the shared protocol-to-runtime seam. +- Keep delivery normalization and React state updates in frontend app code. +- Preserve terminal monotonicity and same-runId semantics. + +**Patterns to follow:** +- `packages/chat-stream-protocol/src/client.ts` +- `packages/chat-stream-protocol/src/ui.ts` +- `apps/frontend/src/components/chat-panel.tsx` + +**Test scenarios:** +- Happy path: `text-delta` aggregates into assistant message content with `metadata.custom.runId`. +- Edge case: pending-before-first-event shows connecting/thinking state. +- Regression: duplicate/out-of-order terminal and loading events do not regress visibility state. + +**Verification:** +- Frontend stream tests pass without page-level parser duplication. + +- [x] **Unit 4: Shared test helper and smoke alignment** + +**Goal:** Keep backend tests and gateway smoke aligned with toolkit parsing/envelope assertions. + +**Requirements:** R17-R19, R21 + +**Dependencies:** Units 1-3 + +**Files:** +- Existing/modify as needed: `apps/backend/test/e2e/chat-stream-api.spec.ts` +- Existing/modify as needed: `apps/backend/test/integration/delivery-contract.spec.ts` +- Existing/modify as needed: `tests/smoke/nginx-dev-gateway-smoke.mjs` +- Test: `packages/chat-stream-protocol/src/protocol.spec.ts` + +**Approach:** +- Use toolkit parser/assertion helpers in backend tests and smoke where runtime format allows. +- Keep `packages/shared-types` independent from toolkit to avoid schema-owner cycles. + +**Test scenarios:** +- Happy path: e2e stream tests parse with toolkit and assert full envelope consistency. +- Error path: smoke first-event parser reports stream-specific invalid block errors. + +**Verification:** +- No active duplicate parser remains in backend e2e/integration tests. + +- [ ] **Unit 5: Abort signal primitives and LLM gateway cancellation support** + +**Goal:** Add reusable backend cancellation primitives and make LLM stream/generate paths accept user abort signals while preserving runtime timeout behavior. + +**Requirements:** R25-R30 + +**Dependencies:** Units 1-4 + +**Files:** +- Modify: `apps/backend/src/modules/llm/llm-gateway.interface.ts` +- Modify: `apps/backend/src/modules/llm/llm-gateway.service.ts` +- Modify: `apps/backend/src/modules/llm/provider-router.service.ts` +- Modify: `apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts` +- Test: `apps/backend/test/unit/llm-gateway.service.spec.ts` +- Test: `apps/backend/test/unit/sql-generation.service.spec.ts` + +**Approach:** +- Extend `LlmGateway.stream` options with `abortSignal?: AbortSignal`. +- Extend `LlmGateway.generate` only if stream fallback or sync generation needs user cancellation; otherwise keep sync unchanged and document why. +- Compose timeout and user abort into a single signal before calling `streamText`. +- Distinguish timeout abort from user cancellation. User cancellation should not trigger the existing recoverable timeout fallback to non-stream generation. +- In mock mode, check abort state between emitted lines so tests can prove cooperative cancellation. + +**Patterns to follow:** +- Existing `AbortSignal.timeout(runtime.streamTimeoutMs ?? runtime.timeoutMs)` usage in `apps/backend/src/modules/llm/llm-gateway.service.ts` +- Existing stream timeout fallback tests in `apps/backend/test/unit/llm-gateway.service.spec.ts` + +**Test scenarios:** +- Happy path: when no user signal is provided, stream still uses configured `streamTimeoutMs`. +- Happy path: when a non-aborted user signal is provided, `streamText` receives a signal that aborts when either timeout or user signal aborts. +- Error path: user-aborted stream throws or maps to a cancellation-domain error/code and does not call `generateText` fallback. +- Error path: timeout abort still follows existing recoverable behavior where appropriate. +- Edge case: already-aborted user signal fails fast before consuming provider stream. +- Integration: `SqlGenerationService.stream` forwards `abortSignal` through provider router to gateway. + +**Verification:** +- LLM gateway tests prove timeout and user-cancel semantics are distinguishable. + +- [ ] **Unit 6: Backend request-close propagation through workflow and LangGraph** + +**Goal:** Carry cancellation from HTTP request/response close through stream usecase, workflow runner, LangGraph runtime state, and SQL generation stage. + +**Requirements:** R25-R30 + +**Dependencies:** Unit 5 + +**Files:** +- Modify: `apps/backend/src/modules/conversation/chat/chat.controller.ts` +- Modify: `apps/backend/src/modules/conversation/chat/chat.service.ts` +- Modify: `apps/backend/src/modules/conversation/chat/application/stream-message.usecase.ts` +- Modify: `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` +- Modify: `apps/backend/src/modules/conversation/runtime/stages/run-v2-langgraph.stage.ts` +- Modify: `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph-runner.service.ts` +- Modify: `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts` +- Modify: `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts` +- Modify: `apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts` +- Test: `apps/backend/test/unit/text2sql-workflow-runner.spec.ts` +- Test: `apps/backend/test/unit/langgraph-runtime.spec.ts` +- Test: `apps/backend/test/e2e/chat-stream-api.spec.ts` + +**Approach:** +- In `chat.controller.ts`, create a run-scoped abort controller for the stream request. +- Listen to request/response close or abort events, clean listeners in `finally`, and avoid writing extra bytes after `res.writableEnded`. +- Add `abortSignal?: AbortSignal` to stream input types and pass it through every layer. +- Store `abortSignal` in LangGraph stream options/state so `generate-sql` can pass it into `GenerateSqlNode.run`. +- Add cooperative abort checks before starting expensive stages and before/after LLM/tool streaming where local boundaries allow. +- If the response remains writable, emit an `error` terminal with `code: "USER_CANCELLED"`; if not writable, still return/persist a cancellation outcome. + +**Patterns to follow:** +- Current stream route in `apps/backend/src/modules/conversation/chat/chat.controller.ts` +- Current V2 stream options in `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.state.ts` +- Current `onLlmEvent` propagation from `text2sql-v2-langgraph.graph.ts` to `GenerateSqlNode` + +**Test scenarios:** +- Happy path: non-cancelled stream still emits `start`, intermediate events, and `finish`. +- Error path: abort before first LLM event produces/persists failed or cancelled run outcome without continuing to provider stream. +- Error path: abort during `text-delta` streaming stops further LLM events and does not emit `finish`. +- Error path: abort after `tool-call` but before `tool-result` records a clear cancellation outcome and does not leave UI/run in loading. +- Edge case: response already closed prevents writing cancellation terminal but still cleans listeners and persistence path. +- Integration: `runV2LangGraphStage.runStream` receives `abortSignal` from workflow runner. + +**Verification:** +- Unit tests prove the abort signal reaches the LangGraph/generate-sql boundary and cancellation does not run normal finish flow. + +- [ ] **Unit 7: Cancellation outcome persistence and terminal semantics** + +**Goal:** Ensure cancelled runs are visible, explainable, and not stuck in loading across stream UI, run view, and persisted trace. + +**Requirements:** R26-R30 + +**Dependencies:** Units 5-6 + +**Files:** +- Modify: `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` +- Modify: `apps/backend/src/modules/conversation/text2sql/stages/persist-run.stage.ts` if persistence mapping requires adjustment +- Modify: `packages/chat-stream-protocol/src/ui.ts` +- Modify: `apps/frontend/src/components/chat/run-visibility-mapper.ts` +- Test: `apps/backend/test/unit/text2sql-workflow-runner.spec.ts` +- Test: `apps/backend/test/e2e/chat-stream-api.spec.ts` +- Test: `apps/frontend/tests/unit/run-visibility-mapper.spec.ts` + +**Approach:** +- Normalize user cancellation to a stable code, recommended `USER_CANCELLED`. +- Persist a run with `status: "failed"` unless the current domain model already has a safer non-failed cancellation status. Do not invent a new `RunStatus` in this plan. +- Set trace stream status to `failed` and include cancellation-specific error details so run read/replay can explain what happened. +- Update `projectStreamEvent` or frontend mapper to treat `USER_CANCELLED` as terminal and user-stopped, not as an unhandled provider failure. +- Preserve partial text and thinking steps already emitted. + +**Patterns to follow:** +- Existing graph-failure persistence path in `Text2SQLWorkflowRunner.runStream` +- Existing terminal status monotonicity helpers in `packages/chat-stream-protocol/src/ui.ts` + +**Test scenarios:** +- Happy path: normal failed provider errors still show error visibility. +- Error path: `error` event with `code: "USER_CANCELLED"` is terminal and clears active loading state. +- Error path: persisted cancelled run has a clear error/cancellation message and `trace.streamStatus` is not `in_progress`. +- Edge case: if cancellation happens before a run id is visible to frontend, pending state is cleared and user sees a stopped/cancelled explanation. +- Regression: terminal error/cancelled state cannot be overwritten by a later loading patch. + +**Verification:** +- Cancelled streams never leave `activeStreamRunId`, `thinkingRequestPending`, or persisted run status in an indefinite loading state. + +- [ ] **Unit 8: Frontend stop generation UX** + +**Goal:** Provide a clear user stop action that aborts the current stream, preserves partial output, and renders a stopped/cancelled state. + +**Requirements:** R25-R26, R38-R39 + +**Dependencies:** Units 5-7 + +**Files:** +- Modify: `apps/frontend/src/components/chat/assistant-runtime.ts` +- Modify: `apps/frontend/src/components/chat/assistant-thread.tsx` +- Modify: `apps/frontend/src/components/chat/assistant-composer.tsx` +- Modify: `apps/frontend/src/components/chat/assistant-thinking-panel.tsx` +- Modify: `apps/frontend/src/components/chat-panel.tsx` +- Test: `apps/frontend/tests/unit/chat-panel.spec.tsx` +- Test: `apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx` +- Test: `apps/frontend/tests/e2e/chat-demo-flow.spec.tsx` +- Test: `apps/frontend/tests/e2e/chat-mobile-smoke.spec.tsx` + +**Approach:** +- Prefer assistant-ui's built-in abort/cancel mechanism if available in the installed version. If not, introduce a local active-stream abort controller bridge around the existing runtime callbacks. +- Add a stop control using existing shadcn/button conventions and lucide iconography, not a raw uncontrolled HTML control. +- Stop action should immediately clear spinner-only states and mark the active run as stopped/cancelled while preserving streamed text and thinking steps. +- Keep send behavior, IME behavior, mobile layout, and keyboard accessibility intact. +- Ensure stopping one stream does not poison the next stream's `AbortSignal`. + +**Patterns to follow:** +- `apps/frontend/src/components/chat/assistant-composer.tsx` +- `apps/frontend/src/components/chat-panel.tsx` +- Existing frontend shadcn component usage under `apps/frontend/src/components/ui` + +**Test scenarios:** +- Happy path: while a stream is active, a stop button is visible and enabled. +- Happy path: clicking stop aborts the generator/fetch signal and clears connecting/thinking spinner state. +- Happy path: partial `text-delta` content remains visible after stopping. +- Happy path: partial thinking steps remain visible and the panel no longer says work is still running. +- Edge case: stopping before the first `start` event clears pending state. +- Edge case: stopping after `tool-call` but before `tool-result` shows a stopped/cancelled state without pretending the tool completed. +- Regression: after stopping one stream, sending a new message uses a fresh non-aborted signal. +- Accessibility: stop control has an accessible name and is keyboard reachable at 375px width. + +**Verification:** +- Frontend unit/e2e tests cover stop-before-first-event, stop-after-text, and stop-during-tool-progress states. + +- [ ] **Unit 9: Message-layering guardrails and optional summary projection** + +**Goal:** Encode the `claude-code` reference model as guardrails for Text2SQL projection without expanding the public protocol prematurely. + +**Requirements:** R35-R40 + +**Dependencies:** Units 1, 3, 8 + +**Files:** +- Modify: `packages/chat-stream-protocol/src/ui.ts` +- Modify: `apps/frontend/src/components/chat/assistant-thinking-panel.tsx` +- Modify: `apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts` only if existing summary fields need normalization +- Test: `packages/chat-stream-protocol/src/protocol.spec.ts` +- Test: `apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx` +- Test: `apps/backend/test/unit/text2sql-stream-event.mapper.spec.ts` + +**Approach:** +- Keep stream event and display message layers conceptually separate in helper names and docs. +- Continue mapping `tool-call`, `tool-result`, and `tool-error` to thinking/tool lifecycle steps. +- If current event data already carries `summary`, use it to support collapsed/compact tool display; do not add `ToolUseSummaryMessage` as a new protocol event. +- Do not add tombstone/invalidated semantics unless implementation finds a real stream fallback or replay correction consumer. +- Ensure provider `tool_result` as user-role input is mentioned only in backend/provider assembly docs, not frontend runtime code. + +**Patterns to follow:** +- `packages/chat-stream-protocol/src/ui.ts` +- `apps/backend/src/modules/conversation/text2sql/stream/text2sql-stream-event.mapper.ts` + +**Test scenarios:** +- Happy path: text and tool events can interleave without tool steps overwriting assistant text. +- Happy path: tool event `summary` appears in thinking panel detail or compact view where available. +- Edge case: events without summary still render a safe synthesized detail. +- Regression: no new public event type is required for summaries. +- Guardrail: no tombstone/collapsed transcript event appears in `CHAT_STREAM_EVENT_TYPES`. + +**Verification:** +- Event/message layering is visible in tests and docs without changing the stream taxonomy. + +- [ ] **Unit 10: Learning docs, standards, and contributor guide updates** + +**Goal:** Make stream architecture understandable and keep repo standards aligned with the implemented toolkit/cancellation path. + +**Requirements:** R31-R34, R36-R43 + +**Dependencies:** Units 5-9 + +**Files:** +- Modify: `docs/leran/A2A协议.md` +- Modify: `docs/standards/llm-stream-tool-migration-spec.md` +- Modify: `README.md` +- Modify: `AGENTS.md` if scripts, gates, or MUST-level standards change +- Test: `scripts/check-docs-rag-understanding.mjs` only if RAG understanding docs are touched +- Test: `tests/smoke/docs-rag-understanding-contract-smoke.mjs` only if RAG understanding docs are touched + +**Approach:** +- Explain SSE in three layers: transport framing, protocol envelope, and UI projection. +- Add the shortest local path walkthrough: `ChatController -> writeSseEvent -> readSseStream -> streamMessageEvents -> assistant-runtime/chat-panel -> projectStreamEvent`. +- Add cancellation walkthrough: frontend stop/abort, backend request close, workflow/LangGraph propagation, LLM gateway signal composition, persisted outcome. +- Add a `claude-code` comparison section that maps event/message concepts to this project and states what is intentionally not copied. +- Replace any machine-local absolute links in `docs/leran/A2A协议.md` with repo-relative paths. +- Update `docs/standards/llm-stream-tool-migration-spec.md` with toolkit ownership and cancellation expectations. + +**Patterns to follow:** +- `docs/standards/llm-stream-tool-migration-spec.md` +- Existing requirement doc section `Code-Level Walkthrough` in `docs/brainstorms/2026-04-30-chat-stream-protocol-toolkit-requirements.md` + +**Test scenarios:** +- Test expectation: none for pure docs, unless implementation touches docs with existing contract-check scripts. +- Documentation review: manually verify every file path is repo-relative and the docs do not claim private model chain-of-thought is displayed. + +**Verification:** +- A new contributor can identify protocol owner, stream flow, stop flow, and message-layering boundaries without reading unrelated controller/page code first. + +- [ ] **Unit 11: Final gate and regression verification plan** + +**Goal:** Close the work with targeted unit/integration/e2e coverage and required stream/Text2SQL gates. + +**Requirements:** All requirements, especially R13, R18-R21, R25-R30 + +**Dependencies:** Units 5-10 + +**Files:** +- Test: `packages/chat-stream-protocol/src/protocol.spec.ts` +- Test: `apps/backend/test/unit/llm-gateway.service.spec.ts` +- Test: `apps/backend/test/unit/text2sql-workflow-runner.spec.ts` +- Test: `apps/backend/test/unit/langgraph-runtime.spec.ts` +- Test: `apps/backend/test/e2e/chat-stream-api.spec.ts` +- Test: `apps/backend/test/integration/delivery-contract.spec.ts` +- Test: `apps/frontend/tests/unit/chat-panel.spec.tsx` +- Test: `apps/frontend/tests/unit/run-visibility-mapper.spec.ts` +- Test: `apps/frontend/tests/unit/assistant-thinking-panel.spec.tsx` +- Test: `apps/frontend/tests/e2e/chat-demo-flow.spec.tsx` +- Test: `tests/smoke/nginx-dev-gateway-smoke.mjs` + +**Approach:** +- Treat cancellation as cross-layer behavior. Unit tests should prove signal propagation and terminal state; integration/e2e tests should prove stream contract remains stable. +- Keep package-level tests focused on protocol mechanics, not Text2SQL business semantics. +- Keep backend/focused gates aligned with AGENTS stream and Text2SQL v2 instructions. + +**Test scenarios:** +- Package: SSE parse/framing and UI projection tests pass. +- Backend unit: LLM user abort is not misclassified as provider timeout. +- Backend unit: workflow persists cancellation outcome and skips normal finish on cancellation. +- Backend e2e: stream contract still emits valid envelopes and terminal events for normal/rejected/error flows. +- Frontend unit: stop action clears active loading state and preserves partial output. +- Frontend e2e: chat send/stream/thinking/stop path works at desktop and mobile widths. +- Smoke: unified gateway first stream event remains valid through `http://localhost:3000`. +- Static/quality: no legacy duplicate parser or unsupported event taxonomy drift. + +**Verification:** +- Implementation completion should show these verification outcomes: + - protocol toolkit lint and protocol tests pass + - backend lint, unit/integration tests, e2e stream coverage, and build pass + - frontend lint, unit/e2e stream UI coverage, and build pass + - gateway smoke confirms first stream event remains valid through the unified local entry when the dev stack is running + - Text2SQL v2 eval gate, focused coverage gate, and no-legacy-compat gate pass for the affected stream/runtime surfaces + +## System-Wide Impact + +- **Interaction graph:** Stop/cancel now affects UI runtime, fetch/readable stream, Nest controller request lifecycle, workflow runner, LangGraph stream state, SQL generation, provider router, and LLM gateway. +- **Error propagation:** User cancellation should be distinguishable from provider timeout, provider malformed response, tool failure, and business rejection. It should clear UI loading and persist an explainable terminal outcome. +- **State lifecycle risks:** Cancellation can happen before `start`, after `start`, during token streaming, during tool execution, after tool call but before result, or during enrichment/persistence. Each state must avoid indefinite loading. +- **API surface parity:** Public stream event taxonomy remains unchanged. Sync endpoint behavior should not regress, and stream/sync delivery evidence parity remains governed by existing tests. +- **Integration coverage:** Unit tests alone are insufficient; frontend stop UI and backend persisted outcome both need cross-layer coverage. +- **Unchanged invariants:** `ChatStreamEvent` schema owner remains `packages/shared-types`; protocol helper owner remains `packages/chat-stream-protocol`; business stage semantics remain under conversation/Text2SQL modules. + +## Risks & Dependencies + +| Risk | Mitigation | +|------|------------| +| User abort is mistaken for provider timeout and triggers non-stream fallback | Add explicit abort reason/code checks in LLM gateway tests and skip fallback for user cancellation | +| Response is already closed before cancellation terminal can be written | Persist cancellation outcome and clear backend listeners even when no SSE terminal can be sent | +| Tool execution cannot be forcibly interrupted | Propagate signal and add cooperative checks; document non-cooperative tool limits as implementation notes | +| Stop UI fights assistant-ui internal runtime state | Prefer built-in assistant-ui abort controls if present; otherwise isolate local stop bridge and test new send after stop | +| Adding message-layer concepts expands protocol too far | Keep `CHAT_STREAM_EVENT_TYPES` unchanged and cover guardrails in tests/docs | +| Existing ignored docs/plans are not visible in git diff | Mention ignored doc behavior in handoff; implementation can still use local plan artifact | + +## Documentation / Operational Notes + +- This plan updates an ignored `docs/plans` artifact in the current repo policy. The file is still useful locally, but staging it requires deliberate repo policy action. +- Implementation changes touch code files, so after code modification sessions run `graphify update .` per AGENTS guidance. +- No Prisma migration is involved. +- After frontend stop UI changes, verify at 375px width and keyboard access. +- Gateway smoke needs a running local stack through `http://localhost:3000`. + +## Sources & References + +- **Origin document:** `docs/brainstorms/2026-04-30-chat-stream-protocol-toolkit-requirements.md` +- Protocol toolkit: `packages/chat-stream-protocol/src/index.ts` +- Shared stream schema: `packages/shared-types/src/api.ts` +- Backend stream route: `apps/backend/src/modules/conversation/chat/chat.controller.ts` +- Backend workflow: `apps/backend/src/modules/conversation/application/workflow/text2sql-workflow-runner.service.ts` +- LangGraph runtime: `apps/backend/src/modules/conversation/runtime/langgraph/text2sql-v2-langgraph.graph.ts` +- LLM gateway: `apps/backend/src/modules/llm/llm-gateway.service.ts` +- Frontend stream runtime: `apps/frontend/src/components/chat/assistant-runtime.ts` +- Frontend chat panel: `apps/frontend/src/components/chat-panel.tsx` +- Stream standard: `docs/standards/llm-stream-tool-migration-spec.md` diff --git a/docs/standards/llm-stream-tool-migration-spec.md b/docs/standards/llm-stream-tool-migration-spec.md index 097f7b3..ef2bdd0 100644 --- a/docs/standards/llm-stream-tool-migration-spec.md +++ b/docs/standards/llm-stream-tool-migration-spec.md @@ -47,6 +47,10 @@ ### Stream Contract (`/messages/stream`) - Response content type: `text/event-stream` +- Ownership: + - `packages/shared-types` owns `ChatStreamEvent` schema and event taxonomy. + - `packages/chat-stream-protocol` owns reusable envelope / framing / parsing / terminal helper implementation. + - backend / frontend / smoke compatibility glue must not become a second canonical protocol rule source. - Event types: - `start` - `text-delta` @@ -64,6 +68,9 @@ - `at` - `data` - `data` is always a structured object, not raw string. +- SSE writer / parser helper changes should land in `packages/chat-stream-protocol` first, then be adopted by app/test consumers. +- 前端 stop / fetch abort 必须贯通到 backend request-close、workflow runner、LangGraph runtime、SQL generation 与 LLM gateway/provider 调用;不允许只停止浏览器 reader 而让后端继续消耗 provider / tool 资源。 +- 在单独的兼容性计划批准前,用户停止生成继续复用既有 terminal `error` 事件,并固定返回 `code: "USER_CANCELLED"`;不得直接新增公开 `cancelled` event type。 - `finish` 事件中的 `data.delivery.evidence.promptTemplate?`、`modelingRevision?`、`effectiveContextSummary?`、`conflictHint?` 必须与同步接口字段语义一致(不再要求历史 snake_case / alias hydration)。 - strict-completion/runtime-intelligence 场景下,`finish` 事件中的 `data.delivery.evidence.v2` 也必须保持 `contextPackSummary`、`metadataAnswer`、`correctionGrounding`、`runtimePlan`、`artifactRefs`、`smartDefaults` 与 `run.trace.v2` 的同语义镜像;`state` 事件只能暴露 runtime plan 的安全摘要,不得暴露 raw graph state/checkpoint。Full Mermaid stages 必须在 stream `state` 中产生安全的 `running` 与 terminal lifecycle(completed/skipped/failed/clarification)摘要。 diff --git a/graphify-out/GRAPH_REPORT.md b/graphify-out/GRAPH_REPORT.md index 435f43e..d834be2 100644 --- a/graphify-out/GRAPH_REPORT.md +++ b/graphify-out/GRAPH_REPORT.md @@ -1,12 +1,12 @@ -# Graph Report - /Users/lienli/Documents/GitHub/text2sql (2026-04-29) +# Graph Report - /Users/lienli/Documents/GitHub/text2sql (2026-05-01) ## Corpus Check -- 781 files · ~2,266,621 words +- 802 files · ~2,357,057 words - Verdict: corpus is large enough that graph structure adds value. ## Summary -- 4314 nodes · 8504 edges · 538 communities detected -- Extraction: 79% EXTRACTED · 21% INFERRED · 0% AMBIGUOUS · INFERRED: 1809 edges (avg confidence: 0.8) +- 4480 nodes · 8904 edges · 546 communities detected +- Extraction: 78% EXTRACTED · 22% INFERRED · 0% AMBIGUOUS · INFERRED: 1917 edges (avg confidence: 0.8) - Token cost: 0 input · 0 output ## Community Hubs (Navigation) @@ -548,36 +548,44 @@ - [[_COMMUNITY_Community 535|Community 535]] - [[_COMMUNITY_Community 536|Community 536]] - [[_COMMUNITY_Community 537|Community 537]] +- [[_COMMUNITY_Community 538|Community 538]] +- [[_COMMUNITY_Community 539|Community 539]] +- [[_COMMUNITY_Community 540|Community 540]] +- [[_COMMUNITY_Community 541|Community 541]] +- [[_COMMUNITY_Community 542|Community 542]] +- [[_COMMUNITY_Community 543|Community 543]] +- [[_COMMUNITY_Community 544|Community 544]] +- [[_COMMUNITY_Community 545|Community 545]] ## God Nodes (most connected - your core abstractions) 1. `ok()` - 90 edges -2. `WorkspaceModelingService` - 79 edges -3. `RagRetrievalService` - 78 edges +2. `RagRetrievalService` - 87 edges +3. `WorkspaceModelingService` - 79 edges 4. `DeliveryContractMapper` - 61 edges 5. `AppConfigService` - 59 edges 6. `GlossaryService` - 59 edges 7. `isRecord()` - 55 edges 8. `Text2SqlV2ArtifactBuilder` - 53 edges 9. `toAdminApiError()` - 51 edges -10. `DatasourceService` - 45 edges +10. `SqlGenerationService` - 48 edges ## Surprising Connections (you probably didn't know these) - `loadPreviewForDetails()` --calls--> `getWorkspaceModelingPreview()` [INFERRED] /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/settings/modeling/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/lib/admin-api-client.ts -- `load()` --calls--> `setError()` [INFERRED] - /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/glossary/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/components/settings/workspace-management-panel.tsx - `load()` --calls--> `listGlossaryTerms()` [INFERRED] /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/glossary/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/lib/admin-api-client.ts - `syncWorkspaceContext()` --calls--> `writeActiveWorkspaceId()` [INFERRED] /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/settings/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/lib/datasource-session-context.ts - `resolveWorkspaceId()` --calls--> `readActiveWorkspaceId()` [INFERRED] /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/settings/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/lib/datasource-session-context.ts +- `refreshModels()` --calls--> `setError()` [INFERRED] + /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/settings/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/components/settings/workspace-management-panel.tsx ## Communities ### Community 0 - "Community 0" -Cohesion: 0.02 -Nodes (41): extractLatestUserText(), mapToThreadMessages(), ChartBiGroundingGuard, ChartBiResultProfiler, buildContextEnvelopeFromDraft(), parseEntityMappings(), splitByDelimiters(), trimOrUndefined() (+33 more) +Cohesion: 0.01 +Nodes (53): extractLatestUserText(), mapToThreadMessages(), ChartBiGroundingGuard, ChartBiResultProfiler, buildContextEnvelopeFromDraft(), parseEntityMappings(), splitByDelimiters(), trimOrUndefined() (+45 more) ### Community 1 - "Community 1" Cohesion: 0.02 @@ -585,75 +593,75 @@ Nodes (19): ChatController, ChatPolicyGuardService, ChatRepository, ChatRunPersi ### Community 2 - "Community 2" Cohesion: 0.03 -Nodes (145): addWorkspaceDatasourceBindings(), addWorkspaceMembers(), AdminApiError, commitModelingSetup(), composeApiUrl(), createGlossaryAnchor(), createGlossaryTerm(), createPromptTemplate() (+137 more) +Nodes (147): addWorkspaceDatasourceBindings(), addWorkspaceMembers(), AdminApiError, commitModelingSetup(), composeApiUrl(), createGlossaryAnchor(), createGlossaryTerm(), createPromptTemplate() (+139 more) ### Community 3 - "Community 3" Cohesion: 0.02 -Nodes (22): BuildRagIndexJob, asActor(), asActor(), LlmConfigRepository, toModelHealthStatus(), toProviderCode(), toProviderSyncStatus(), createRunId() (+14 more) +Nodes (41): AppConfigService, DatasourceRegistryService, countTracedRuns(), main(), parseArgs(), computeLangsmithCoverage(), bootstrap(), buildGraph() (+33 more) ### Community 4 - "Community 4" -Cohesion: 0.02 -Nodes (35): AppConfigService, DatasourceRegistryService, countTracedRuns(), main(), parseArgs(), computeLangsmithCoverage(), bootstrap(), buildGraph() (+27 more) +Cohesion: 0.03 +Nodes (19): ok(), ChartBiValidator, DatasourceController, EvalController, GlossaryController, MemoryController, createDeliverySandboxPolicy(), isSandboxFilesystemWriteAllowed() (+11 more) ### Community 5 - "Community 5" Cohesion: 0.03 -Nodes (14): ok(), ChartBiValidator, DatasourceController, EvalController, GlossaryController, MemoryController, SettingsController, UserController (+6 more) +Nodes (13): BuildRagIndexJob, asActor(), asActor(), GlossaryService, RagDocumentRepository, RagIndexBuilderService, RagIndexRepository, withActor() (+5 more) ### Community 6 - "Community 6" -Cohesion: 0.04 -Nodes (9): ModelingSchemaChangeRepository, toScopeKey(), SaveViewFromRunUsecase, normalizeName(), SchemaChangeDetectorService, normalizeName(), WorkspaceModelingSchemaChangeDetectorService, normalizeTableName() (+1 more) +Cohesion: 0.03 +Nodes (84): BuildSemanticQueryNode, buildConversationKnowledgeAllowlistSet(), collectFiles(), compactLine(), createLineStarts(), extractImportReferences(), findRepoRoot(), indexToLineColumn() (+76 more) ### Community 7 - "Community 7" Cohesion: 0.03 -Nodes (10): DataBootstrapService, PlatformDataBootstrapModule, FakePgClient, DatasourceRepository, DatasourceService, DatasourceWorkflowService, MysqlExecutorService, PostgresExecutorService (+2 more) +Nodes (15): DataBootstrapService, PlatformDataBootstrapModule, FakePgClient, DatasourceRepository, DatasourceService, DatasourceWorkflowService, MysqlExecutorService, PostgresExecutorService (+7 more) ### Community 8 - "Community 8" -Cohesion: 0.03 -Nodes (74): BuildSemanticQueryNode, buildConversationKnowledgeAllowlistSet(), collectFiles(), compactLine(), createLineStarts(), extractImportReferences(), findRepoRoot(), indexToLineColumn() (+66 more) +Cohesion: 0.04 +Nodes (13): setupWorkspaceScopedSession(), AuditLogRepository, ModelingGraphValidator, nonEmpty(), asNonEmptyString(), toBindingKey(), toRuleKey(), toTablePermissionKey() (+5 more) ### Community 9 - "Community 9" Cohesion: 0.03 -Nodes (22): BuildPhysicalPlanNode, ChartBiIntentParser, parseSseEvents(), parseSseEvents(), FileDatasourceExecutorService, GateMetricsService, parsePayload(), PlannerCacheService (+14 more) +Nodes (9): LlmConfigRepository, toModelHealthStatus(), toProviderCode(), toProviderSyncStatus(), PromptTemplateService, ProviderCatalogService, RagTaskConfigRepository, RagTaskConfigService (+1 more) ### Community 10 - "Community 10" -Cohesion: 0.04 -Nodes (9): shouldRetryCorrection(), AnswerNode, CorrectSqlNode, FormatAnswerNode, SavedPriorSqlService, SemanticAssetReindexService, SqlCorrectionService, Text2SQLWorkflowRunner (+1 more) +Cohesion: 0.03 +Nodes (27): composeAbortSignals(), createUserCancelledError(), isUserCancelledError(), throwIfAborted(), shouldRetryCorrection(), AnswerNode, CorrectSqlNode, FormatAnswerNode (+19 more) ### Community 11 - "Community 11" -Cohesion: 0.04 -Nodes (15): EmbeddingRouterService, GeminiAdapter, resolveGeminiUrl(), isTimeoutAbortError(), LlmGatewayService, toErrorMessage(), LlmModelFactory, resolveProviderBaseUrl() (+7 more) +Cohesion: 0.03 +Nodes (86): handleKeyDown(), isRecord(), nextFieldId(), normalizeFunctionGroupHints(), resolveCalculatedFieldSaveError(), save(), toForm(), upsertField() (+78 more) ### Community 12 - "Community 12" Cohesion: 0.04 -Nodes (64): handleKeyDown(), isRecord(), nextFieldId(), normalizeFunctionGroupHints(), resolveCalculatedFieldSaveError(), save(), toForm(), upsertField() (+56 more) +Nodes (9): ModelingSchemaChangeRepository, toScopeKey(), normalizeName(), SchemaChangeDetectorService, WorkspaceDatasourceController, normalizeName(), WorkspaceModelingSchemaChangeDetectorService, normalizeTableName() (+1 more) ### Community 13 - "Community 13" -Cohesion: 0.05 -Nodes (5): GenerateSqlNode, RagCacheKeyFactory, SqlGenerationService, SqlOutputExtractor, SqlPromptBuilder +Cohesion: 0.04 +Nodes (101): ApiClientRequestError, composeApiUrl(), createDatasource(), createIdempotencyKey(), createSession(), DatasourceApiError, deleteSession(), getMessages() (+93 more) ### Community 14 - "Community 14" -Cohesion: 0.06 -Nodes (7): LaneTimeoutError, RagRetrievalService, appendUnique(), fuseWithRrf(), laneOrder(), normalizeRankConstant(), stableSortLaneHits() +Cohesion: 0.03 +Nodes (18): ChartBiIntentParser, GateMetricsService, parsePayload(), RagAuditReplayService, formatDateTime(), mergeWorkspaceRoles(), parseHeaderSegments(), parseWorkspaceRolesJson() (+10 more) ### Community 15 - "Community 15" -Cohesion: 0.05 -Nodes (5): ModelRerankerAdapter, ProviderRouterService, RagBudgetPolicy, RagReplayRepository, RagRerankService +Cohesion: 0.04 +Nodes (14): BuildPhysicalPlanNode, getChunkProfileConfig(), PlannerCacheService, RagCacheKeyFactory, RagChunkingService, sha256(), RagDocumentFactory, sha256() (+6 more) ### Community 16 - "Community 16" Cohesion: 0.05 -Nodes (8): GraphAccelerationCircuitBreaker, GraphService, IntakeNode, RelationshipPublishGateFacade, RowFilterRewriteService, SafetyCheckNode, SqlSafetyGuard, SqlTableAccessGuardService +Nodes (7): LaneTimeoutError, RagRetrievalService, appendUnique(), fuseWithRrf(), laneOrder(), normalizeRankConstant(), stableSortLaneHits() ### Community 17 - "Community 17" -Cohesion: 0.08 -Nodes (8): setupWorkspaceScopedSession(), asNonEmptyString(), toBindingKey(), toRuleKey(), toTablePermissionKey(), toTablePermissionSetKey(), WorkspaceDatasourcePolicyRepository, WorkspaceDatasourceService +Cohesion: 0.05 +Nodes (4): GenerateSqlNode, SqlGenerationService, SqlOutputExtractor, SqlPromptBuilder ### Community 18 - "Community 18" -Cohesion: 0.06 -Nodes (62): createIdempotencyKey(), resolveDatasourceWorkflowApiError(), applySessionView(), dedupeSessions(), extractErrorCode(), init(), loadMessages(), mergeSessionMessages() (+54 more) +Cohesion: 0.04 +Nodes (6): ChatPostRunHooksService, MemoryPromotionPolicy, MemoryPromotionService, RagBudgetPolicy, RagReplayRepository, RagRerankService ### Community 19 - "Community 19" -Cohesion: 0.07 -Nodes (7): AuditLogRepository, ModelingGraphRepository, toScopeKey(), ModelingGraphValidator, nonEmpty(), normalizeId(), WorkspaceRelationshipService +Cohesion: 0.04 +Nodes (9): GraphAccelerationCircuitBreaker, GraphService, IntakeNode, QueryExecutorRouterService, RelationshipPublishGateFacade, RowFilterRewriteService, SafetyCheckNode, SqlSafetyGuard (+1 more) ### Community 20 - "Community 20" Cohesion: 0.08 @@ -661,2086 +669,2118 @@ Nodes (7): normalizeValue(), WorkspaceAdminGuard, toMembershipKey(), toWorkspace ### Community 21 - "Community 21" Cohesion: 0.08 -Nodes (10): toPlatformUserStatus(), UserRepository, UserService, onConfirmBatchDelete(), onConfirmDeleteOne(), onConfirmResetPassword(), onSubmitEditor(), onToggleStatus() (+2 more) +Nodes (2): Text2SqlV2ArtifactBuilder, Text2SqlV2LangGraphResultMapper ### Community 22 - "Community 22" -Cohesion: 0.06 -Nodes (7): getChunkProfileConfig(), RagChunkingService, sha256(), RagDocumentFactory, sha256(), RagEventConsumerService, SemanticRegistryService +Cohesion: 0.08 +Nodes (10): toPlatformUserStatus(), UserRepository, UserService, onConfirmBatchDelete(), onConfirmDeleteOne(), onConfirmResetPassword(), onSubmitEditor(), onToggleStatus() (+2 more) ### Community 23 - "Community 23" -Cohesion: 0.1 -Nodes (1): GlossaryService - -### Community 24 - "Community 24" Cohesion: 0.06 Nodes (5): HealthController, RagIngestionMetricsService, RagQualityController, RagQualityService, SemanticSpineShadowService +### Community 24 - "Community 24" +Cohesion: 0.08 +Nodes (3): SemanticAssetReadinessService, SemanticAssetReindexService, SemanticRegistryService + ### Community 25 - "Community 25" -Cohesion: 0.12 -Nodes (1): Text2SqlV2ArtifactBuilder +Cohesion: 0.08 +Nodes (19): BuildIntentPlanNode, buildRuleReasonCodes(), ClarificationFusionPolicy, mapRuleDecisionToEvidence(), unique(), ClarifyNode, buildDecision(), buildFallbackSlotFillingDecision() (+11 more) ### Community 26 - "Community 26" -Cohesion: 0.06 -Nodes (7): ChartBiSpecCompiler, LangsmithTraceService, createText2SqlV2LangGraph(), Text2SqlV2LangGraphResultMapper, Text2SqlV2LangGraphRunnerService, createText2SqlV2LangGraphInitialState(), mergeRuntimePlanReducer() +Cohesion: 0.08 +Nodes (4): AssembleContextNode, ChartBiArtifactService, ChatDeliveryEnrichmentService, SemanticPlanNode ### Community 27 - "Community 27" Cohesion: 0.08 -Nodes (19): BuildIntentPlanNode, buildRuleReasonCodes(), ClarificationFusionPolicy, mapRuleDecisionToEvidence(), unique(), ClarifyNode, buildDecision(), buildFallbackSlotFillingDecision() (+11 more) +Nodes (3): ModelRerankerAdapter, ProviderRouterService, RagTaskHealthProbeService ### Community 28 - "Community 28" Cohesion: 0.08 -Nodes (4): AssembleContextNode, ChartBiArtifactService, ChatDeliveryEnrichmentService, SemanticPlanNode +Nodes (19): resolveText2SqlReasoningStage(), resolveText2SqlTitle(), resolveText2SqlV2StageCatalogEntry(), Text2SqlStreamEventMapper, assertStateNotAborted(), computeDurationMs(), createNodeUpdate(), createRuntimePlanUpdate() (+11 more) ### Community 29 - "Community 29" -Cohesion: 0.1 -Nodes (40): ApiClientRequestError, composeApiUrl(), createDatasource(), createSession(), DatasourceApiError, deleteSession(), getMessages(), getRun() (+32 more) +Cohesion: 0.14 +Nodes (38): appendThinkingStep(), ensureRunLoaded(), isRecord(), isTerminalStep(), mergeRunThinkingSteps(), mergeThinkingStep(), normalizeArtifact(), normalizeChartArtifact() (+30 more) ### Community 30 - "Community 30" Cohesion: 0.08 -Nodes (18): resolveText2SqlReasoningStage(), resolveText2SqlTitle(), resolveText2SqlV2StageCatalogEntry(), Text2SqlStreamEventMapper, computeDurationMs(), createNodeUpdate(), createRuntimePlanUpdate(), createStageArtifact() (+10 more) +Nodes (6): ChartBiSpecCompiler, LangsmithTraceService, createText2SqlV2LangGraph(), Text2SqlV2LangGraphRunnerService, createText2SqlV2LangGraphInitialState(), mergeRuntimePlanReducer() ### Community 31 - "Community 31" -Cohesion: 0.16 -Nodes (35): appendThinkingStep(), ensureRunLoaded(), isRecord(), isTerminalStep(), mergeRunThinkingSteps(), mergeThinkingStep(), normalizeArtifact(), normalizeChartArtifact() (+27 more) +Cohesion: 0.13 +Nodes (1): SemanticSpineRepository ### Community 32 - "Community 32" Cohesion: 0.13 -Nodes (1): SemanticSpineRepository +Nodes (21): applyCollectorDefaults(), collectClarificationBalanceGate(), detectMetadataBypass(), detectStrictSemanticPath(), ensureFixture(), evaluatePostClarifySemanticPass(), extractSemanticPlanStatus(), findTraceStep() (+13 more) ### Community 33 - "Community 33" -Cohesion: 0.13 -Nodes (2): QueryExecutorRouterService, SqlValidationService +Cohesion: 0.14 +Nodes (1): SqlValidationService ### Community 34 - "Community 34" Cohesion: 0.13 -Nodes (21): applyCollectorDefaults(), collectClarificationBalanceGate(), detectMetadataBypass(), detectStrictSemanticPath(), ensureFixture(), evaluatePostClarifySemanticPass(), extractSemanticPlanStatus(), findTraceStep() (+13 more) +Nodes (1): SemanticContextPackService ### Community 35 - "Community 35" -Cohesion: 0.09 -Nodes (3): ChatPostRunHooksService, MemoryPromotionPolicy, MemoryPromotionService +Cohesion: 0.15 +Nodes (1): SemanticPlanService ### Community 36 - "Community 36" -Cohesion: 0.12 -Nodes (22): buildOperationNotice(), closeDialog(), closeDrawer(), createIdempotencyKey(), formatDate(), handleDelete(), handleToggleTerm(), load() (+14 more) +Cohesion: 0.23 +Nodes (2): ModelingGraphRepository, toScopeKey() ### Community 37 - "Community 37" -Cohesion: 0.13 -Nodes (1): SemanticContextPackService - -### Community 38 - "Community 38" Cohesion: 0.15 -Nodes (1): SemanticPlanService +Nodes (16): appendReadableDelta(), assertValidChatStreamEnvelope(), collectTextDelta(), isChatStreamEventType(), isRecord(), isTerminalEvent(), isTerminalEventType(), parseSseBlock() (+8 more) -### Community 39 - "Community 39" +### Community 38 - "Community 38" Cohesion: 0.13 Nodes (7): formatCell(), normalizeNullableText(), normalizeTableKey(), openEditDialog(), resolveModelRelationships(), resolveRelationshipCounterpartTable(), submitMetadataUpdate() -### Community 40 - "Community 40" +### Community 39 - "Community 39" Cohesion: 0.22 Nodes (2): ClarificationSemanticEvaluatorService, SemanticEvaluatorTimeoutError -### Community 41 - "Community 41" +### Community 40 - "Community 40" Cohesion: 0.39 Nodes (14): isRecord(), normalizeArtifact(), normalizeChart(), normalizeChartMappings(), normalizeDisplayType(), normalizeFallback(), normalizeSummary(), normalizeTable() (+6 more) -### Community 42 - "Community 42" +### Community 41 - "Community 41" Cohesion: 0.26 Nodes (10): createIdempotencyKey(), isConflictError(), isRecord(), normalizeTableNames(), readNumberCandidate(), resolveConflictPolicyVersion(), resolveConflictSummary(), setsEqual() (+2 more) +### Community 42 - "Community 42" +Cohesion: 0.29 +Nodes (1): FileDatasourceExecutorService + ### Community 43 - "Community 43" -Cohesion: 0.25 -Nodes (6): createDeliverySandboxPolicy(), isSandboxFilesystemWriteAllowed(), isSandboxNetworkAllowed(), isSandboxProcessSpawnAllowed(), normalizeString(), SandboxRuntimeService +Cohesion: 0.27 +Nodes (4): GraphAccelerationAdapter, GraphAccelerationError, GraphAccelerationTimeoutError, GraphAccelerationUnsupportedOperatorError ### Community 44 - "Community 44" -Cohesion: 0.26 -Nodes (1): SemanticSpineShadowGateService +Cohesion: 0.36 +Nodes (1): SkillRegistryService ### Community 45 - "Community 45" -Cohesion: 0.27 -Nodes (4): GraphAccelerationAdapter, GraphAccelerationError, GraphAccelerationTimeoutError, GraphAccelerationUnsupportedOperatorError +Cohesion: 0.28 +Nodes (3): resetToLatest(), resolveConfigVersion(), syncFromConfig() ### Community 46 - "Community 46" -Cohesion: 0.24 -Nodes (6): handleOpenChange(), includesFailClosedHint(), normalizeAlerts(), resolveAutoExpandedSection(), resolveFailClosedAlerts(), sectionSeverityScore() +Cohesion: 0.42 +Nodes (1): ResolveSavedPriorSqlNode ### Community 47 - "Community 47" -Cohesion: 0.28 -Nodes (3): resetToLatest(), resolveConfigVersion(), syncFromConfig() +Cohesion: 0.25 +Nodes (0): ### Community 48 - "Community 48" -Cohesion: 0.42 -Nodes (1): ResolveSavedPriorSqlNode +Cohesion: 0.32 +Nodes (3): resetToLatest(), resolveConfigVersion(), syncFromConfig() ### Community 49 - "Community 49" Cohesion: 0.25 -Nodes (0): +Nodes (0): ### Community 50 - "Community 50" -Cohesion: 0.32 -Nodes (3): resetToLatest(), resolveConfigVersion(), syncFromConfig() +Cohesion: 0.29 +Nodes (3): InMemorySemanticSpineRepositoryDouble, SemanticSpineContractError, validateSemanticObject() ### Community 51 - "Community 51" -Cohesion: 0.25 -Nodes (0): +Cohesion: 0.54 +Nodes (2): IngestionSourceAdapter, uniqueNonEmpty() ### Community 52 - "Community 52" Cohesion: 0.29 -Nodes (3): InMemorySemanticSpineRepositoryDouble, SemanticSpineContractError, validateSemanticObject() +Nodes (0): ### Community 53 - "Community 53" Cohesion: 0.29 -Nodes (0): +Nodes (0): ### Community 54 - "Community 54" -Cohesion: 0.29 -Nodes (0): - -### Community 55 - "Community 55" Cohesion: 0.33 Nodes (2): providerStatusLabel(), providerStatusVariant() +### Community 55 - "Community 55" +Cohesion: 0.29 +Nodes (1): KnowledgeChatSupportFacade + ### Community 56 - "Community 56" -Cohesion: 0.38 -Nodes (4): createModelingFlowFieldHandleId(), createModelingFlowRelationshipHandleId(), ModelingFlowNode(), normalizeHandleToken() +Cohesion: 0.33 +Nodes (0): ### Community 57 - "Community 57" -Cohesion: 0.48 -Nodes (6): computeElkLayout(), ElkLayoutTimeoutError, sortEdges(), sortNodes(), toElapsedMs(), withTimeout() +Cohesion: 0.33 +Nodes (0): ### Community 58 - "Community 58" -Cohesion: 0.57 -Nodes (2): IngestionSourceAdapter, uniqueNonEmpty() +Cohesion: 0.33 +Nodes (0): ### Community 59 - "Community 59" -Cohesion: 0.29 -Nodes (1): KnowledgeChatSupportFacade +Cohesion: 0.4 +Nodes (3): buildReadySnapshot(), buildSnakeCaseSnapshot(), SemanticSpineRepositoryStub ### Community 60 - "Community 60" Cohesion: 0.33 -Nodes (0): +Nodes (1): PolicyEvaluatorService ### Community 61 - "Community 61" Cohesion: 0.33 -Nodes (0): +Nodes (4): RelationshipBridgeDto, RelationshipBridgeEndpointDto, RelationshipGraphEdgeDto, ReplaceWorkspaceRelationshipGraphDto ### Community 62 - "Community 62" -Cohesion: 0.33 -Nodes (0): +Cohesion: 0.4 +Nodes (1): ResizeObserver ### Community 63 - "Community 63" Cohesion: 0.4 -Nodes (3): buildReadySnapshot(), buildSnakeCaseSnapshot(), SemanticSpineRepositoryStub +Nodes (0): ### Community 64 - "Community 64" -Cohesion: 0.33 -Nodes (1): PolicyEvaluatorService +Cohesion: 0.4 +Nodes (0): ### Community 65 - "Community 65" -Cohesion: 0.33 -Nodes (4): RelationshipBridgeDto, RelationshipBridgeEndpointDto, RelationshipGraphEdgeDto, ReplaceWorkspaceRelationshipGraphDto +Cohesion: 0.4 +Nodes (0): ### Community 66 - "Community 66" -Cohesion: 0.4 -Nodes (1): ResizeObserver +Cohesion: 0.5 +Nodes (2): CarouselNext(), useCarousel() ### Community 67 - "Community 67" -Cohesion: 0.4 -Nodes (0): +Cohesion: 0.6 +Nodes (4): nextEdgeId(), RelationshipEdgeEditorDialog(), toEdge(), toForm() ### Community 68 - "Community 68" Cohesion: 0.4 -Nodes (0): +Nodes (0): ### Community 69 - "Community 69" Cohesion: 0.4 -Nodes (0): +Nodes (0): ### Community 70 - "Community 70" Cohesion: 0.5 -Nodes (2): CarouselNext(), useCarousel() +Nodes (2): isToolStep(), resolveStepTitle() ### Community 71 - "Community 71" -Cohesion: 0.6 -Nodes (4): nextEdgeId(), RelationshipEdgeEditorDialog(), toEdge(), toForm() +Cohesion: 0.4 +Nodes (0): ### Community 72 - "Community 72" Cohesion: 0.4 -Nodes (0): +Nodes (1): FakePgClient ### Community 73 - "Community 73" -Cohesion: 0.4 -Nodes (0): +Cohesion: 0.8 +Nodes (4): normalizeConfidence(), normalizeEndpoint(), normalizeRelationshipGraphEdge(), normalizeRequired() ### Community 74 - "Community 74" -Cohesion: 0.6 -Nodes (3): resolveStepTitle(), statusLabel(), statusVariant() +Cohesion: 0.4 +Nodes (4): UpsertDatasourceWorkflowDto, WorkflowDatasourcePayloadDto, WorkflowWorkspaceCreateDto, WorkflowWorkspacePayloadDto ### Community 75 - "Community 75" Cohesion: 0.4 -Nodes (0): +Nodes (1): RunV2LangGraphStage ### Community 76 - "Community 76" Cohesion: 0.4 -Nodes (1): FakePgClient +Nodes (1): RunV2StateMachineStage ### Community 77 - "Community 77" -Cohesion: 0.8 -Nodes (4): normalizeConfidence(), normalizeEndpoint(), normalizeRelationshipGraphEdge(), normalizeRequired() +Cohesion: 0.5 +Nodes (0): ### Community 78 - "Community 78" -Cohesion: 0.4 -Nodes (4): UpsertDatasourceWorkflowDto, WorkflowDatasourcePayloadDto, WorkflowWorkspaceCreateDto, WorkflowWorkspacePayloadDto +Cohesion: 0.5 +Nodes (0): ### Community 79 - "Community 79" -Cohesion: 0.4 -Nodes (1): RunV2LangGraphStage +Cohesion: 0.5 +Nodes (0): ### Community 80 - "Community 80" -Cohesion: 0.4 -Nodes (1): RunV2StateMachineStage +Cohesion: 0.67 +Nodes (2): listRagProviderPresets(), resolveRagProviderPreset() ### Community 81 - "Community 81" Cohesion: 0.5 -Nodes (0): +Nodes (1): TimeoutEvaluator ### Community 82 - "Community 82" -Cohesion: 0.5 -Nodes (0): +Cohesion: 0.83 +Nodes (3): buildModelingRepository(), buildModelingService(), buildRelationshipService() ### Community 83 - "Community 83" Cohesion: 0.5 -Nodes (0): +Nodes (1): ListPromptTemplatesQueryDto ### Community 84 - "Community 84" -Cohesion: 0.67 -Nodes (2): listRagProviderPresets(), resolveRagProviderPreset() +Cohesion: 0.5 +Nodes (1): UpsertModelingSetupDto ### Community 85 - "Community 85" Cohesion: 0.5 -Nodes (1): TimeoutEvaluator +Nodes (1): CreateGlossaryTermDto ### Community 86 - "Community 86" -Cohesion: 0.83 -Nodes (3): buildModelingRepository(), buildModelingService(), buildRelationshipService() +Cohesion: 0.5 +Nodes (1): UpdateGlossaryTermDto ### Community 87 - "Community 87" Cohesion: 0.5 -Nodes (1): RagModule +Nodes (3): ContextEnvelopeDto, ContextEnvelopeEntityMappingDto, ContextEnvelopeTimeRangeDto ### Community 88 - "Community 88" Cohesion: 0.5 -Nodes (1): ListPromptTemplatesQueryDto +Nodes (1): PostRunHooksStage ### Community 89 - "Community 89" -Cohesion: 0.5 -Nodes (1): UpsertModelingSetupDto +Cohesion: 0.67 +Nodes (0): ### Community 90 - "Community 90" -Cohesion: 0.5 -Nodes (1): CreateGlossaryTermDto +Cohesion: 0.67 +Nodes (0): ### Community 91 - "Community 91" -Cohesion: 0.5 -Nodes (1): UpdateGlossaryTermDto +Cohesion: 1.0 +Nodes (2): createDatasourceAndOpenSetupWizard(), openCreateToStep2() ### Community 92 - "Community 92" -Cohesion: 0.5 -Nodes (3): ContextEnvelopeDto, ContextEnvelopeEntityMappingDto, ContextEnvelopeTimeRangeDto +Cohesion: 0.67 +Nodes (0): ### Community 93 - "Community 93" -Cohesion: 0.5 -Nodes (1): PostRunHooksStage +Cohesion: 0.67 +Nodes (0): ### Community 94 - "Community 94" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 95 - "Community 95" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 96 - "Community 96" -Cohesion: 1.0 -Nodes (2): createDatasourceAndOpenSetupWizard(), openCreateToStep2() +Cohesion: 0.67 +Nodes (0): ### Community 97 - "Community 97" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 98 - "Community 98" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 99 - "Community 99" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 100 - "Community 100" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 101 - "Community 101" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 102 - "Community 102" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 103 - "Community 103" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 104 - "Community 104" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 105 - "Community 105" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 106 - "Community 106" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 107 - "Community 107" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 108 - "Community 108" -Cohesion: 0.67 -Nodes (0): +Cohesion: 1.0 +Nodes (2): resolveBadgeLabel(), resolveBadgeVariant() ### Community 109 - "Community 109" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 110 - "Community 110" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 111 - "Community 111" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 112 - "Community 112" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 113 - "Community 113" -Cohesion: 1.0 -Nodes (2): resolveBadgeLabel(), resolveBadgeVariant() +Cohesion: 0.67 +Nodes (0): ### Community 114 - "Community 114" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 115 - "Community 115" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 116 - "Community 116" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 117 - "Community 117" Cohesion: 0.67 -Nodes (0): +Nodes (0): ### Community 118 - "Community 118" Cohesion: 0.67 -Nodes (0): +Nodes (1): DomainError ### Community 119 - "Community 119" Cohesion: 0.67 -Nodes (0): +Nodes (1): OpenRouterAdapter ### Community 120 - "Community 120" Cohesion: 0.67 -Nodes (0): +Nodes (1): KimiAdapter ### Community 121 - "Community 121" Cohesion: 0.67 -Nodes (0): +Nodes (1): VolcengineAdapter ### Community 122 - "Community 122" Cohesion: 0.67 -Nodes (1): DomainError +Nodes (1): MinimaxAdapter ### Community 123 - "Community 123" Cohesion: 0.67 -Nodes (1): OpenRouterAdapter +Nodes (1): TencentHunyuanAdapter ### Community 124 - "Community 124" Cohesion: 0.67 -Nodes (1): KimiAdapter +Nodes (1): SiliconflowAdapter ### Community 125 - "Community 125" Cohesion: 0.67 -Nodes (1): VolcengineAdapter +Nodes (1): DeepSeekAdapter ### Community 126 - "Community 126" Cohesion: 0.67 -Nodes (1): MinimaxAdapter +Nodes (1): OpenAiAdapter ### Community 127 - "Community 127" Cohesion: 0.67 -Nodes (1): TencentHunyuanAdapter +Nodes (1): TongyiAdapter ### Community 128 - "Community 128" Cohesion: 0.67 -Nodes (1): SiliconflowAdapter +Nodes (1): MemoryModule ### Community 129 - "Community 129" Cohesion: 0.67 -Nodes (1): DeepSeekAdapter +Nodes (1): AdminOnlyGuard ### Community 130 - "Community 130" Cohesion: 0.67 -Nodes (1): OpenAiAdapter +Nodes (1): PlatformChatRuntimeFacade ### Community 131 - "Community 131" Cohesion: 0.67 -Nodes (1): TongyiAdapter +Nodes (1): GlossaryModule ### Community 132 - "Community 132" Cohesion: 0.67 -Nodes (1): MemoryModule +Nodes (1): RelationshipImpactSimulatorService ### Community 133 - "Community 133" Cohesion: 0.67 -Nodes (1): AdminOnlyGuard +Nodes (1): SemanticRegistryModule ### Community 134 - "Community 134" Cohesion: 0.67 -Nodes (1): PlatformChatRuntimeFacade +Nodes (1): GovernanceChatAccessFacade ### Community 135 - "Community 135" Cohesion: 0.67 -Nodes (1): GlossaryModule +Nodes (2): PreviewDatasourcePayloadDto, PreviewDatasourceTablesDto ### Community 136 - "Community 136" Cohesion: 0.67 -Nodes (1): RelationshipImpactSimulatorService +Nodes (1): UpdatePromptTemplateDto ### Community 137 - "Community 137" Cohesion: 0.67 -Nodes (1): SemanticRegistryModule +Nodes (1): CreatePromptTemplateDto ### Community 138 - "Community 138" Cohesion: 0.67 -Nodes (1): GovernanceChatAccessFacade +Nodes (2): CheckRagTaskConfigDraftDto, CheckRagTaskConfigHealthDto ### Community 139 - "Community 139" Cohesion: 0.67 -Nodes (2): PreviewDatasourcePayloadDto, PreviewDatasourceTablesDto +Nodes (1): UpsertModelingGraphDto ### Community 140 - "Community 140" Cohesion: 0.67 -Nodes (1): UpdatePromptTemplateDto +Nodes (1): ListWorkspaceDatasourceTablePermissionsDto ### Community 141 - "Community 141" Cohesion: 0.67 -Nodes (1): CreatePromptTemplateDto +Nodes (1): ResolveModelingSchemaChangeDto ### Community 142 - "Community 142" Cohesion: 0.67 -Nodes (2): CheckRagTaskConfigDraftDto, CheckRagTaskConfigHealthDto +Nodes (1): ReplaceWorkspaceDatasourceTablePermissionsDto ### Community 143 - "Community 143" Cohesion: 0.67 -Nodes (1): UpsertModelingGraphDto +Nodes (1): ListUsersDto ### Community 144 - "Community 144" Cohesion: 0.67 -Nodes (1): ListWorkspaceDatasourceTablePermissionsDto +Nodes (1): ListGlossaryTermsQueryDto ### Community 145 - "Community 145" Cohesion: 0.67 -Nodes (1): ResolveModelingSchemaChangeDto +Nodes (1): RollbackGlossaryAnchorDto ### Community 146 - "Community 146" -Cohesion: 0.67 -Nodes (1): ReplaceWorkspaceDatasourceTablePermissionsDto +Cohesion: 1.0 +Nodes (0): ### Community 147 - "Community 147" -Cohesion: 0.67 -Nodes (1): ListUsersDto +Cohesion: 1.0 +Nodes (0): ### Community 148 - "Community 148" -Cohesion: 0.67 -Nodes (1): ListGlossaryTermsQueryDto +Cohesion: 1.0 +Nodes (0): ### Community 149 - "Community 149" -Cohesion: 0.67 -Nodes (1): RollbackGlossaryAnchorDto +Cohesion: 1.0 +Nodes (0): ### Community 150 - "Community 150" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 151 - "Community 151" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 152 - "Community 152" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 153 - "Community 153" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 154 - "Community 154" Cohesion: 1.0 -Nodes (0): +Nodes (1): ELK ### Community 155 - "Community 155" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 156 - "Community 156" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 157 - "Community 157" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 158 - "Community 158" Cohesion: 1.0 -Nodes (1): ELK +Nodes (0): ### Community 159 - "Community 159" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 160 - "Community 160" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 161 - "Community 161" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 162 - "Community 162" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 163 - "Community 163" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 164 - "Community 164" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 165 - "Community 165" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 166 - "Community 166" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 167 - "Community 167" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 168 - "Community 168" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 169 - "Community 169" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 170 - "Community 170" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 171 - "Community 171" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 172 - "Community 172" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 173 - "Community 173" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 174 - "Community 174" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 175 - "Community 175" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 176 - "Community 176" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 177 - "Community 177" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 178 - "Community 178" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 179 - "Community 179" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 180 - "Community 180" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 181 - "Community 181" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 182 - "Community 182" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 183 - "Community 183" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 184 - "Community 184" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 185 - "Community 185" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 186 - "Community 186" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 187 - "Community 187" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 188 - "Community 188" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 189 - "Community 189" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 190 - "Community 190" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 191 - "Community 191" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 192 - "Community 192" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 193 - "Community 193" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 194 - "Community 194" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 195 - "Community 195" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 196 - "Community 196" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 197 - "Community 197" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 198 - "Community 198" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 199 - "Community 199" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 200 - "Community 200" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 201 - "Community 201" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 202 - "Community 202" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 203 - "Community 203" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 204 - "Community 204" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 205 - "Community 205" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 206 - "Community 206" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 207 - "Community 207" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 208 - "Community 208" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 209 - "Community 209" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 210 - "Community 210" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 211 - "Community 211" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 212 - "Community 212" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 213 - "Community 213" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 214 - "Community 214" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 215 - "Community 215" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 216 - "Community 216" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 217 - "Community 217" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 218 - "Community 218" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 219 - "Community 219" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 220 - "Community 220" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 221 - "Community 221" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 222 - "Community 222" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 223 - "Community 223" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 224 - "Community 224" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 225 - "Community 225" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 226 - "Community 226" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 227 - "Community 227" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 228 - "Community 228" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 229 - "Community 229" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 230 - "Community 230" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 231 - "Community 231" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 232 - "Community 232" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 233 - "Community 233" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 234 - "Community 234" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 235 - "Community 235" Cohesion: 1.0 -Nodes (0): +Nodes (1): AppModule ### Community 236 - "Community 236" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 237 - "Community 237" Cohesion: 1.0 -Nodes (0): +Nodes (1): LlmModule ### Community 238 - "Community 238" Cohesion: 1.0 -Nodes (1): AppModule +Nodes (1): ApplyMemoryFeedbackDto ### Community 239 - "Community 239" Cohesion: 1.0 -Nodes (0): +Nodes (1): AppConfigModule ### Community 240 - "Community 240" Cohesion: 1.0 -Nodes (1): LlmModule +Nodes (1): PlatformModule ### Community 241 - "Community 241" Cohesion: 1.0 -Nodes (1): ApplyMemoryFeedbackDto +Nodes (1): PlatformLlmModule ### Community 242 - "Community 242" Cohesion: 1.0 -Nodes (1): AppConfigModule +Nodes (1): PlatformConfigModule ### Community 243 - "Community 243" Cohesion: 1.0 -Nodes (1): PlatformModule +Nodes (1): PlatformObservabilityModule ### Community 244 - "Community 244" Cohesion: 1.0 -Nodes (1): PlatformLlmModule +Nodes (1): PlatformDataAccessModule ### Community 245 - "Community 245" Cohesion: 1.0 -Nodes (1): PlatformConfigModule +Nodes (1): PlatformDataQueryModule ### Community 246 - "Community 246" Cohesion: 1.0 -Nodes (1): PlatformObservabilityModule +Nodes (1): PlatformDataModule ### Community 247 - "Community 247" Cohesion: 1.0 -Nodes (1): PlatformDataAccessModule +Nodes (1): PlatformDataPersistenceModule ### Community 248 - "Community 248" Cohesion: 1.0 -Nodes (1): PlatformDataQueryModule +Nodes (1): ObservabilityModule ### Community 249 - "Community 249" Cohesion: 1.0 -Nodes (1): PlatformDataModule +Nodes (1): SystemModule ### Community 250 - "Community 250" Cohesion: 1.0 -Nodes (1): PlatformDataPersistenceModule +Nodes (1): KnowledgeModule ### Community 251 - "Community 251" Cohesion: 1.0 -Nodes (1): ObservabilityModule +Nodes (1): SemanticSpineModule ### Community 252 - "Community 252" Cohesion: 1.0 -Nodes (1): SystemModule +Nodes (1): SkillRegistryModule ### Community 253 - "Community 253" Cohesion: 1.0 -Nodes (1): KnowledgeModule +Nodes (1): GovernanceModule ### Community 254 - "Community 254" Cohesion: 1.0 -Nodes (1): SemanticSpineModule +Nodes (1): DatasourceModule ### Community 255 - "Community 255" Cohesion: 1.0 -Nodes (1): SkillRegistryModule +Nodes (1): CreateDatasourceDto ### Community 256 - "Community 256" Cohesion: 1.0 -Nodes (1): GovernanceModule +Nodes (1): UploadFileDatasourceDto ### Community 257 - "Community 257" Cohesion: 1.0 -Nodes (1): DatasourceModule +Nodes (1): UpdateDatasourceDto ### Community 258 - "Community 258" Cohesion: 1.0 -Nodes (1): CreateDatasourceDto +Nodes (1): SettingsModule ### Community 259 - "Community 259" Cohesion: 1.0 -Nodes (1): UploadFileDatasourceDto +Nodes (1): UpsertRagTaskConfigDto ### Community 260 - "Community 260" Cohesion: 1.0 -Nodes (1): UpdateDatasourceDto +Nodes (1): PreviewRagProviderModelsDto ### Community 261 - "Community 261" Cohesion: 1.0 -Nodes (1): SettingsModule +Nodes (1): BatchUpdateModelStatusDto ### Community 262 - "Community 262" Cohesion: 1.0 -Nodes (1): UpsertRagTaskConfigDto +Nodes (1): UpdateModelStatusDto ### Community 263 - "Community 263" Cohesion: 1.0 -Nodes (1): PreviewRagProviderModelsDto +Nodes (1): RefreshProviderModelsDto ### Community 264 - "Community 264" Cohesion: 1.0 -Nodes (1): BatchUpdateModelStatusDto +Nodes (1): CreateProviderDto ### Community 265 - "Community 265" Cohesion: 1.0 -Nodes (1): UpdateModelStatusDto +Nodes (1): GovernanceAccessModule ### Community 266 - "Community 266" Cohesion: 1.0 -Nodes (1): RefreshProviderModelsDto +Nodes (1): WorkspaceModule ### Community 267 - "Community 267" Cohesion: 1.0 -Nodes (1): CreateProviderDto +Nodes (1): RemoveWorkspaceMembersDto ### Community 268 - "Community 268" Cohesion: 1.0 -Nodes (1): GovernanceAccessModule +Nodes (1): RenameWorkspaceDto ### Community 269 - "Community 269" Cohesion: 1.0 -Nodes (1): WorkspaceModule +Nodes (1): GetModelingPreviewDto ### Community 270 - "Community 270" Cohesion: 1.0 -Nodes (1): RemoveWorkspaceMembersDto +Nodes (1): UpdateWorkspaceMemberRoleDto ### Community 271 - "Community 271" Cohesion: 1.0 -Nodes (1): RenameWorkspaceDto +Nodes (1): DetectModelingSchemaChangeDto ### Community 272 - "Community 272" Cohesion: 1.0 -Nodes (1): GetModelingPreviewDto +Nodes (1): ListWorkspaceMembersDto ### Community 273 - "Community 273" Cohesion: 1.0 -Nodes (1): UpdateWorkspaceMemberRoleDto +Nodes (1): DeployModelingGraphDto ### Community 274 - "Community 274" Cohesion: 1.0 -Nodes (1): DetectModelingSchemaChangeDto +Nodes (1): AddWorkspaceMemberDto ### Community 275 - "Community 275" Cohesion: 1.0 -Nodes (1): ListWorkspaceMembersDto +Nodes (1): CreateWorkspaceDto ### Community 276 - "Community 276" Cohesion: 1.0 -Nodes (1): DeployModelingGraphDto +Nodes (1): UserModule ### Community 277 - "Community 277" Cohesion: 1.0 -Nodes (1): AddWorkspaceMemberDto +Nodes (1): BatchDeleteUsersDto ### Community 278 - "Community 278" Cohesion: 1.0 -Nodes (1): CreateWorkspaceDto +Nodes (1): UpdateUserStatusDto ### Community 279 - "Community 279" Cohesion: 1.0 -Nodes (1): UserModule +Nodes (1): UpdateUserDto ### Community 280 - "Community 280" Cohesion: 1.0 -Nodes (1): BatchDeleteUsersDto +Nodes (1): UserVariableDto ### Community 281 - "Community 281" Cohesion: 1.0 -Nodes (1): UpdateUserStatusDto +Nodes (1): CreateUserDto ### Community 282 - "Community 282" Cohesion: 1.0 -Nodes (1): UpdateUserDto +Nodes (1): EvalModule ### Community 283 - "Community 283" Cohesion: 1.0 -Nodes (1): UserVariableDto +Nodes (1): RunEvaluationDto ### Community 284 - "Community 284" Cohesion: 1.0 -Nodes (1): CreateUserDto +Nodes (1): ConversationModule ### Community 285 - "Community 285" Cohesion: 1.0 -Nodes (1): EvalModule +Nodes (1): ChatModule ### Community 286 - "Community 286" Cohesion: 1.0 -Nodes (1): RunEvaluationDto +Nodes (1): SendMessageDto ### Community 287 - "Community 287" Cohesion: 1.0 -Nodes (1): ConversationModule +Nodes (1): ListSessionsDto ### Community 288 - "Community 288" Cohesion: 1.0 -Nodes (1): ChatModule +Nodes (1): CreateSessionDto ### Community 289 - "Community 289" Cohesion: 1.0 -Nodes (1): SendMessageDto +Nodes (1): RenameSessionDto ### Community 290 - "Community 290" Cohesion: 1.0 -Nodes (1): ListSessionsDto +Nodes (1): SaveViewFromRunDto ### Community 291 - "Community 291" Cohesion: 1.0 -Nodes (1): CreateSessionDto +Nodes (1): Text2SqlModule ### Community 292 - "Community 292" Cohesion: 1.0 -Nodes (1): RenameSessionDto +Nodes (1): AgentModule ### Community 293 - "Community 293" Cohesion: 1.0 -Nodes (1): SaveViewFromRunDto +Nodes (0): ### Community 294 - "Community 294" Cohesion: 1.0 -Nodes (1): Text2SqlModule +Nodes (0): ### Community 295 - "Community 295" Cohesion: 1.0 -Nodes (1): AgentModule +Nodes (0): ### Community 296 - "Community 296" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 297 - "Community 297" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 298 - "Community 298" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 299 - "Community 299" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 300 - "Community 300" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 301 - "Community 301" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 302 - "Community 302" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 303 - "Community 303" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 304 - "Community 304" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 305 - "Community 305" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 306 - "Community 306" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 307 - "Community 307" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 308 - "Community 308" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 309 - "Community 309" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 310 - "Community 310" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 311 - "Community 311" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 312 - "Community 312" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 313 - "Community 313" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 314 - "Community 314" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 315 - "Community 315" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 316 - "Community 316" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 317 - "Community 317" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 318 - "Community 318" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 319 - "Community 319" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 320 - "Community 320" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 321 - "Community 321" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 322 - "Community 322" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 323 - "Community 323" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 324 - "Community 324" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 325 - "Community 325" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 326 - "Community 326" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 327 - "Community 327" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 328 - "Community 328" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 329 - "Community 329" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 330 - "Community 330" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 331 - "Community 331" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 332 - "Community 332" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 333 - "Community 333" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 334 - "Community 334" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 335 - "Community 335" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 336 - "Community 336" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 337 - "Community 337" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 338 - "Community 338" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 339 - "Community 339" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 340 - "Community 340" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 341 - "Community 341" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 342 - "Community 342" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 343 - "Community 343" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 344 - "Community 344" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 345 - "Community 345" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 346 - "Community 346" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 347 - "Community 347" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 348 - "Community 348" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 349 - "Community 349" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 350 - "Community 350" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 351 - "Community 351" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 352 - "Community 352" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 353 - "Community 353" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 354 - "Community 354" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 355 - "Community 355" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 356 - "Community 356" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 357 - "Community 357" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 358 - "Community 358" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 359 - "Community 359" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 360 - "Community 360" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 361 - "Community 361" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 362 - "Community 362" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 363 - "Community 363" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 364 - "Community 364" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 365 - "Community 365" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 366 - "Community 366" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 367 - "Community 367" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 368 - "Community 368" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 369 - "Community 369" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 370 - "Community 370" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 371 - "Community 371" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 372 - "Community 372" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 373 - "Community 373" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 374 - "Community 374" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 375 - "Community 375" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 376 - "Community 376" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 377 - "Community 377" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 378 - "Community 378" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 379 - "Community 379" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 380 - "Community 380" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 381 - "Community 381" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 382 - "Community 382" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 383 - "Community 383" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 384 - "Community 384" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 385 - "Community 385" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 386 - "Community 386" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 387 - "Community 387" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 388 - "Community 388" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 389 - "Community 389" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 390 - "Community 390" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 391 - "Community 391" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 392 - "Community 392" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 393 - "Community 393" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 394 - "Community 394" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 395 - "Community 395" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 396 - "Community 396" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 397 - "Community 397" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 398 - "Community 398" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 399 - "Community 399" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 400 - "Community 400" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 401 - "Community 401" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 402 - "Community 402" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 403 - "Community 403" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 404 - "Community 404" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 405 - "Community 405" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 406 - "Community 406" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 407 - "Community 407" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 408 - "Community 408" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 409 - "Community 409" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 410 - "Community 410" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 411 - "Community 411" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 412 - "Community 412" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 413 - "Community 413" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 414 - "Community 414" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 415 - "Community 415" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 416 - "Community 416" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 417 - "Community 417" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 418 - "Community 418" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 419 - "Community 419" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 420 - "Community 420" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 421 - "Community 421" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 422 - "Community 422" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 423 - "Community 423" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 424 - "Community 424" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 425 - "Community 425" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 426 - "Community 426" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 427 - "Community 427" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 428 - "Community 428" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 429 - "Community 429" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 430 - "Community 430" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 431 - "Community 431" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 432 - "Community 432" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 433 - "Community 433" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 434 - "Community 434" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 435 - "Community 435" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 436 - "Community 436" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 437 - "Community 437" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 438 - "Community 438" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 439 - "Community 439" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 440 - "Community 440" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 441 - "Community 441" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 442 - "Community 442" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 443 - "Community 443" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 444 - "Community 444" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 445 - "Community 445" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 446 - "Community 446" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 447 - "Community 447" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 448 - "Community 448" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 449 - "Community 449" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 450 - "Community 450" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 451 - "Community 451" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 452 - "Community 452" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 453 - "Community 453" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 454 - "Community 454" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 455 - "Community 455" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 456 - "Community 456" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 457 - "Community 457" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 458 - "Community 458" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 459 - "Community 459" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 460 - "Community 460" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 461 - "Community 461" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 462 - "Community 462" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 463 - "Community 463" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 464 - "Community 464" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 465 - "Community 465" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 466 - "Community 466" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 467 - "Community 467" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 468 - "Community 468" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 469 - "Community 469" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 470 - "Community 470" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 471 - "Community 471" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 472 - "Community 472" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 473 - "Community 473" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 474 - "Community 474" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 475 - "Community 475" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 476 - "Community 476" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 477 - "Community 477" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 478 - "Community 478" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 479 - "Community 479" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 480 - "Community 480" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 481 - "Community 481" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 482 - "Community 482" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 483 - "Community 483" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 484 - "Community 484" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 485 - "Community 485" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 486 - "Community 486" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 487 - "Community 487" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 488 - "Community 488" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 489 - "Community 489" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 490 - "Community 490" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 491 - "Community 491" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 492 - "Community 492" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 493 - "Community 493" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 494 - "Community 494" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 495 - "Community 495" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 496 - "Community 496" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 497 - "Community 497" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 498 - "Community 498" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 499 - "Community 499" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 500 - "Community 500" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 501 - "Community 501" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 502 - "Community 502" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 503 - "Community 503" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 504 - "Community 504" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 505 - "Community 505" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 506 - "Community 506" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 507 - "Community 507" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 508 - "Community 508" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 509 - "Community 509" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 510 - "Community 510" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 511 - "Community 511" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 512 - "Community 512" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 513 - "Community 513" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 514 - "Community 514" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 515 - "Community 515" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 516 - "Community 516" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 517 - "Community 517" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 518 - "Community 518" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 519 - "Community 519" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 520 - "Community 520" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 521 - "Community 521" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 522 - "Community 522" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 523 - "Community 523" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 524 - "Community 524" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 525 - "Community 525" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 526 - "Community 526" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 527 - "Community 527" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 528 - "Community 528" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 529 - "Community 529" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 530 - "Community 530" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 531 - "Community 531" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 532 - "Community 532" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 533 - "Community 533" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 534 - "Community 534" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 535 - "Community 535" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 536 - "Community 536" Cohesion: 1.0 -Nodes (0): +Nodes (0): ### Community 537 - "Community 537" Cohesion: 1.0 -Nodes (0): +Nodes (0): + +### Community 538 - "Community 538" +Cohesion: 1.0 +Nodes (0): + +### Community 539 - "Community 539" +Cohesion: 1.0 +Nodes (0): + +### Community 540 - "Community 540" +Cohesion: 1.0 +Nodes (0): + +### Community 541 - "Community 541" +Cohesion: 1.0 +Nodes (0): + +### Community 542 - "Community 542" +Cohesion: 1.0 +Nodes (0): + +### Community 543 - "Community 543" +Cohesion: 1.0 +Nodes (0): + +### Community 544 - "Community 544" +Cohesion: 1.0 +Nodes (0): + +### Community 545 - "Community 545" +Cohesion: 1.0 +Nodes (0): ## Knowledge Gaps - **93 isolated node(s):** `ELK`, `ElkLayoutTimeoutError`, `AppModule`, `LlmModule`, `ApplyMemoryFeedbackDto` (+88 more) These have ≤1 connection - possible missing edges or undocumented components. -- **Thin community `Community 150`** (2 nodes): `pickSelectOption()`, `settings-modeling-relationship-editor.spec.tsx` +- **Thin community `Community 146`** (2 nodes): `pickSelectOption()`, `settings-modeling-relationship-editor.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 151`** (2 nodes): `SectionHarness()`, `rag-delivery-section.spec.tsx` +- **Thin community `Community 147`** (2 nodes): `SectionHarness()`, `rag-delivery-section.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 152`** (2 nodes): `waitForWorkspaceDatasourceReady()`, `settings-modeling-workspace.spec.tsx` +- **Thin community `Community 148`** (2 nodes): `waitForWorkspaceDatasourceReady()`, `settings-modeling-workspace.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 153`** (2 nodes): `ChatBIResultTabsHarness()`, `chatbi-result-tabs.spec.tsx` +- **Thin community `Community 149`** (2 nodes): `ChatBIResultTabsHarness()`, `chatbi-result-tabs.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 154`** (2 nodes): `createDelivery()`, `rag-delivery-panel.spec.tsx` +- **Thin community `Community 150`** (2 nodes): `createDelivery()`, `rag-delivery-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 155`** (2 nodes): `[ +- **Thin community `Community 151`** (2 nodes): `[ { id: "model.orders", tableName: "orders", @@ -2751,407 +2791,415 @@ Nodes (0): } ]()`, `settings-modeling-details-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 156`** (2 nodes): `onUnhandledRejection()`, `chat-panel.spec.tsx` +- **Thin community `Community 152`** (2 nodes): `onUnhandledRejection()`, `chat-panel.spec.tsx` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 153`** (2 nodes): `createSettingsView()`, `settings-mobile-smoke.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 157`** (2 nodes): `createSettingsView()`, `settings-mobile-smoke.spec.tsx` +- **Thin community `Community 154`** (2 nodes): `ELK`, `elkjs.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 158`** (2 nodes): `ELK`, `elkjs.d.ts` +- **Thin community `Community 155`** (2 nodes): `RootLayout()`, `layout.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 159`** (2 nodes): `RootLayout()`, `layout.tsx` +- **Thin community `Community 156`** (2 nodes): `ChatPage()`, `page.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 160`** (2 nodes): `ChatPage()`, `page.tsx` +- **Thin community `Community 157`** (2 nodes): `AppShell()`, `app-shell.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 161`** (2 nodes): `AppShell()`, `app-shell.tsx` +- **Thin community `Community 158`** (2 nodes): `AspectRatio()`, `aspect-ratio.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 162`** (2 nodes): `AspectRatio()`, `aspect-ratio.tsx` +- **Thin community `Community 159`** (2 nodes): `StateBlock()`, `state-block.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 163`** (2 nodes): `StateBlock()`, `state-block.tsx` +- **Thin community `Community 160`** (2 nodes): `DirectionProvider()`, `direction.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 164`** (2 nodes): `DirectionProvider()`, `direction.tsx` +- **Thin community `Community 161`** (2 nodes): `cn()`, `tabs.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 165`** (2 nodes): `cn()`, `tabs.tsx` +- **Thin community `Community 162`** (2 nodes): `ButtonGroup()`, `button-group.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 166`** (2 nodes): `ButtonGroup()`, `button-group.tsx` +- **Thin community `Community 163`** (2 nodes): `cn()`, `card.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 167`** (2 nodes): `cn()`, `card.tsx` +- **Thin community `Community 164`** (2 nodes): `InputGroup()`, `input-group.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 168`** (2 nodes): `InputGroup()`, `input-group.tsx` +- **Thin community `Community 165`** (2 nodes): `cn()`, `input-otp.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 169`** (2 nodes): `cn()`, `input-otp.tsx` +- **Thin community `Community 166`** (2 nodes): `HoverCard()`, `hover-card.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 170`** (2 nodes): `HoverCard()`, `hover-card.tsx` +- **Thin community `Community 167`** (2 nodes): `cn()`, `field.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 171`** (2 nodes): `cn()`, `field.tsx` +- **Thin community `Community 168`** (2 nodes): `Label()`, `label.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 172`** (2 nodes): `Label()`, `label.tsx` +- **Thin community `Community 169`** (2 nodes): `cn()`, `empty.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 173`** (2 nodes): `cn()`, `empty.tsx` +- **Thin community `Community 170`** (2 nodes): `TooltipContent()`, `tooltip.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 174`** (2 nodes): `TooltipContent()`, `tooltip.tsx` +- **Thin community `Community 171`** (2 nodes): `cn()`, `alert.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 175`** (2 nodes): `cn()`, `alert.tsx` +- **Thin community `Community 172`** (2 nodes): `Switch()`, `switch.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 176`** (2 nodes): `Switch()`, `switch.tsx` +- **Thin community `Community 173`** (2 nodes): `cn()`, `command.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 177`** (2 nodes): `cn()`, `command.tsx` +- **Thin community `Community 174`** (2 nodes): `ToggleGroup()`, `toggle-group.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 178`** (2 nodes): `ToggleGroup()`, `toggle-group.tsx` +- **Thin community `Community 175`** (2 nodes): `Avatar()`, `avatar.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 179`** (2 nodes): `Avatar()`, `avatar.tsx` +- **Thin community `Community 176`** (2 nodes): `cn()`, `kbd.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 180`** (2 nodes): `cn()`, `kbd.tsx` +- **Thin community `Community 177`** (2 nodes): `SectionHeader()`, `section-header.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 181`** (2 nodes): `SectionHeader()`, `section-header.tsx` +- **Thin community `Community 178`** (2 nodes): `Badge()`, `badge.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 182`** (2 nodes): `Badge()`, `badge.tsx` +- **Thin community `Community 179`** (2 nodes): `cn()`, `table.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 183`** (2 nodes): `cn()`, `table.tsx` +- **Thin community `Community 180`** (2 nodes): `Separator()`, `separator.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 184`** (2 nodes): `Separator()`, `separator.tsx` +- **Thin community `Community 181`** (2 nodes): `Checkbox()`, `checkbox.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 185`** (2 nodes): `Checkbox()`, `checkbox.tsx` +- **Thin community `Community 182`** (2 nodes): `Spinner()`, `spinner.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 186`** (2 nodes): `Spinner()`, `spinner.tsx` +- **Thin community `Community 183`** (2 nodes): `Collapsible()`, `collapsible.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 187`** (2 nodes): `Collapsible()`, `collapsible.tsx` +- **Thin community `Community 184`** (2 nodes): `cn()`, `textarea.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 188`** (2 nodes): `cn()`, `textarea.tsx` +- **Thin community `Community 185`** (2 nodes): `Input()`, `input.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 189`** (2 nodes): `Input()`, `input.tsx` +- **Thin community `Community 186`** (2 nodes): `Skeleton()`, `skeleton.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 190`** (2 nodes): `Skeleton()`, `skeleton.tsx` +- **Thin community `Community 187`** (2 nodes): `workspace-editor-dialog.tsx`, `WorkspaceEditorDialog()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 191`** (2 nodes): `workspace-editor-dialog.tsx`, `WorkspaceEditorDialog()` +- **Thin community `Community 188`** (2 nodes): `handleKeyDown()`, `modeling-context-drawer.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 192`** (2 nodes): `handleKeyDown()`, `modeling-context-drawer.tsx` +- **Thin community `Community 189`** (2 nodes): `resolveBlockingReasonGuidance()`, `modeling-deploy-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 193`** (2 nodes): `resolveBlockingReasonGuidance()`, `modeling-deploy-panel.tsx` +- **Thin community `Community 190`** (2 nodes): `TextPart()`, `assistant-message.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 194`** (2 nodes): `TextPart()`, `assistant-message.tsx` +- **Thin community `Community 191`** (2 nodes): `isRecord()`, `sql-inline-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 195`** (2 nodes): `isRecord()`, `sql-inline-panel.tsx` +- **Thin community `Community 192`** (2 nodes): `resolveMessageRunId()`, `assistant-thread.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 196`** (2 nodes): `resolveMessageRunId()`, `assistant-thread.tsx` +- **Thin community `Community 193`** (2 nodes): `ModelSelector()`, `model-selector.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 197`** (2 nodes): `ModelSelector()`, `model-selector.tsx` +- **Thin community `Community 194`** (2 nodes): `SessionActions()`, `session-actions.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 198`** (2 nodes): `SessionActions()`, `session-actions.tsx` +- **Thin community `Community 195`** (2 nodes): `if()`, `assistant-composer.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 199`** (2 nodes): `SqlDebugDetails()`, `sql-debug-details.tsx` +- **Thin community `Community 196`** (2 nodes): `SqlDebugDetails()`, `sql-debug-details.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 200`** (2 nodes): `SqlRunSummary()`, `sql-run-summary.tsx` +- **Thin community `Community 197`** (2 nodes): `SqlRunSummary()`, `sql-run-summary.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 201`** (2 nodes): `isNumericValue()`, `sql-result-table.tsx` +- **Thin community `Community 198`** (2 nodes): `isNumericValue()`, `sql-result-table.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 202`** (2 nodes): `statusVariant()`, `sql-execution-timeline.tsx` +- **Thin community `Community 199`** (2 nodes): `statusVariant()`, `sql-execution-timeline.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 203`** (2 nodes): `useIsMobile()`, `use-mobile.ts` +- **Thin community `Community 200`** (2 nodes): `useIsMobile()`, `use-mobile.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 204`** (2 nodes): `utils.ts`, `cn()` +- **Thin community `Community 201`** (2 nodes): `utils.ts`, `cn()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 205`** (2 nodes): `createService()`, `sql-generation.service.spec.ts` +- **Thin community `Community 202`** (2 nodes): `createService()`, `sql-generation.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 206`** (2 nodes): `createPreparedRunContext()`, `text2sql-v2-langgraph-result.mapper.spec.ts` +- **Thin community `Community 203`** (2 nodes): `createPreparedRunContext()`, `text2sql-v2-langgraph-result.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 207`** (2 nodes): `createService()`, `semantic-registry.service.spec.ts` +- **Thin community `Community 204`** (2 nodes): `createService()`, `semantic-registry.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 208`** (2 nodes): `createBaseRun()`, `langgraph-runtime.spec.ts` +- **Thin community `Community 205`** (2 nodes): `createBaseRun()`, `langgraph-runtime.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 209`** (2 nodes): `createConfigServiceMock()`, `rag-task-health-probe.service.spec.ts` +- **Thin community `Community 206`** (2 nodes): `createConfigServiceMock()`, `rag-task-health-probe.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 210`** (2 nodes): `createBaseRun()`, `delivery-contract.mapper.spec.ts` +- **Thin community `Community 207`** (2 nodes): `createBaseRun()`, `delivery-contract.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 211`** (2 nodes): `createRequest()`, `request-actor.middleware.spec.ts` +- **Thin community `Community 208`** (2 nodes): `createRequest()`, `request-actor.middleware.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 212`** (2 nodes): `createBaseRun()`, `text2sql-v2-runner.spec.ts` +- **Thin community `Community 209`** (2 nodes): `createBaseRun()`, `text2sql-v2-runner.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 213`** (2 nodes): `workspace-admin.guard.spec.ts`, `createContext()` +- **Thin community `Community 210`** (2 nodes): `workspace-admin.guard.spec.ts`, `createContext()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 214`** (2 nodes): `reset()`, `langsmith-config.spec.ts` +- **Thin community `Community 211`** (2 nodes): `reset()`, `langsmith-config.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 215`** (2 nodes): `createNode()`, `build-semantic-query.node.spec.ts` +- **Thin community `Community 212`** (2 nodes): `createNode()`, `build-semantic-query.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 216`** (2 nodes): `createContext()`, `admin-only.guard.spec.ts` +- **Thin community `Community 213`** (2 nodes): `createContext()`, `admin-only.guard.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 217`** (2 nodes): `createService()`, `text2sql-task-profile-policy.spec.ts` +- **Thin community `Community 214`** (2 nodes): `createService()`, `text2sql-task-profile-policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 218`** (2 nodes): `resetAppEnv()`, `config.module.spec.ts` +- **Thin community `Community 215`** (2 nodes): `resetAppEnv()`, `config.module.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 219`** (2 nodes): `writeRepoFile()`, `capability-boundary-check.spec.ts` +- **Thin community `Community 216`** (2 nodes): `writeRepoFile()`, `capability-boundary-check.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 220`** (2 nodes): `createPolicy()`, `clarification-fusion.policy.spec.ts` +- **Thin community `Community 217`** (2 nodes): `createPolicy()`, `clarification-fusion.policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 221`** (2 nodes): `createRepository()`, `rag-task-config.repository.spec.ts` +- **Thin community `Community 218`** (2 nodes): `createRepository()`, `rag-task-config.repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 222`** (2 nodes): `buildLongSchemaText()`, `rag-chunking.service.spec.ts` +- **Thin community `Community 219`** (2 nodes): `buildLongSchemaText()`, `rag-chunking.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 223`** (2 nodes): `datasource()`, `query-executor-router.spec.ts` +- **Thin community `Community 220`** (2 nodes): `datasource()`, `query-executor-router.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 224`** (2 nodes): `createNode()`, `build-intent-plan.node.spec.ts` +- **Thin community `Community 221`** (2 nodes): `createNode()`, `build-intent-plan.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 225`** (2 nodes): `createSimpleSqliteDb()`, `query-executors.spec.ts` +- **Thin community `Community 222`** (2 nodes): `createSimpleSqliteDb()`, `query-executors.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 226`** (2 nodes): `createBaseRun()`, `sandbox-failover.spec.ts` +- **Thin community `Community 223`** (2 nodes): `createBaseRun()`, `sandbox-failover.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 227`** (2 nodes): `workspace-relationship-api.spec.ts`, `buildService()` +- **Thin community `Community 224`** (2 nodes): `workspace-relationship-api.spec.ts`, `buildService()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 228`** (2 nodes): `buildArtifact()`, `sandbox-isolation.spec.ts` +- **Thin community `Community 225`** (2 nodes): `buildArtifact()`, `sandbox-isolation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 229`** (2 nodes): `datasource()`, `query-executor-router-table-permissions.spec.ts` +- **Thin community `Community 226`** (2 nodes): `datasource()`, `query-executor-router-table-permissions.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 230`** (2 nodes): `sleep()`, `rag-datasource-gate-metrics.spec.ts` +- **Thin community `Community 227`** (2 nodes): `sleep()`, `rag-datasource-gate-metrics.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 231`** (2 nodes): `buildSnapshot()`, `semantic-spine-compiler.spec.ts` +- **Thin community `Community 228`** (2 nodes): `buildSnapshot()`, `semantic-spine-compiler.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 232`** (2 nodes): `workspace-modeling-setup.spec.ts`, `buildService()` +- **Thin community `Community 229`** (2 nodes): `workspace-modeling-setup.spec.ts`, `buildService()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 233`** (2 nodes): `createRun()`, `memory-promotion.spec.ts` +- **Thin community `Community 230`** (2 nodes): `createRun()`, `memory-promotion.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 234`** (2 nodes): `buildSamples()`, `glossary-selected-context-gate.spec.ts` +- **Thin community `Community 231`** (2 nodes): `buildSamples()`, `glossary-selected-context-gate.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 235`** (2 nodes): `readErrorMessage()`, `user-workspace-authz.spec.ts` +- **Thin community `Community 232`** (2 nodes): `readErrorMessage()`, `user-workspace-authz.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 236`** (2 nodes): `readErrorMessage()`, `rule-group-authz.spec.ts` +- **Thin community `Community 233`** (2 nodes): `readErrorMessage()`, `rule-group-authz.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 237`** (2 nodes): `workspace-datasource-authz.spec.ts`, `readErrorMessage()` +- **Thin community `Community 234`** (2 nodes): `workspace-datasource-authz.spec.ts`, `readErrorMessage()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 238`** (2 nodes): `AppModule`, `app.module.ts` +- **Thin community `Community 235`** (2 nodes): `AppModule`, `app.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 239`** (2 nodes): `requestIdMiddleware()`, `request-id.middleware.ts` +- **Thin community `Community 236`** (2 nodes): `requestIdMiddleware()`, `request-id.middleware.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 240`** (2 nodes): `LlmModule`, `llm.module.ts` +- **Thin community `Community 237`** (2 nodes): `LlmModule`, `llm.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 241`** (2 nodes): `ApplyMemoryFeedbackDto`, `apply-memory-feedback.dto.ts` +- **Thin community `Community 238`** (2 nodes): `ApplyMemoryFeedbackDto`, `apply-memory-feedback.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 242`** (2 nodes): `AppConfigModule`, `config.module.ts` +- **Thin community `Community 239`** (2 nodes): `AppConfigModule`, `config.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 243`** (2 nodes): `PlatformModule`, `platform.module.ts` +- **Thin community `Community 240`** (2 nodes): `PlatformModule`, `platform.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 244`** (2 nodes): `PlatformLlmModule`, `llm.module.ts` +- **Thin community `Community 241`** (2 nodes): `PlatformLlmModule`, `llm.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 245`** (2 nodes): `PlatformConfigModule`, `config.module.ts` +- **Thin community `Community 242`** (2 nodes): `PlatformConfigModule`, `config.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 246`** (2 nodes): `PlatformObservabilityModule`, `observability.module.ts` +- **Thin community `Community 243`** (2 nodes): `PlatformObservabilityModule`, `observability.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 247`** (2 nodes): `PlatformDataAccessModule`, `access.module.ts` +- **Thin community `Community 244`** (2 nodes): `PlatformDataAccessModule`, `access.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 248`** (2 nodes): `PlatformDataQueryModule`, `query.module.ts` +- **Thin community `Community 245`** (2 nodes): `PlatformDataQueryModule`, `query.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 249`** (2 nodes): `PlatformDataModule`, `data.module.ts` +- **Thin community `Community 246`** (2 nodes): `PlatformDataModule`, `data.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 250`** (2 nodes): `PlatformDataPersistenceModule`, `persistence.module.ts` +- **Thin community `Community 247`** (2 nodes): `PlatformDataPersistenceModule`, `persistence.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 251`** (2 nodes): `ObservabilityModule`, `observability.module.ts` +- **Thin community `Community 248`** (2 nodes): `ObservabilityModule`, `observability.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 252`** (2 nodes): `SystemModule`, `system.module.ts` +- **Thin community `Community 249`** (2 nodes): `SystemModule`, `system.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 253`** (2 nodes): `KnowledgeModule`, `knowledge.module.ts` +- **Thin community `Community 250`** (2 nodes): `KnowledgeModule`, `knowledge.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 254`** (2 nodes): `SemanticSpineModule`, `semantic-spine.module.ts` +- **Thin community `Community 251`** (2 nodes): `SemanticSpineModule`, `semantic-spine.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 255`** (2 nodes): `SkillRegistryModule`, `skill-registry.module.ts` +- **Thin community `Community 252`** (2 nodes): `SkillRegistryModule`, `skill-registry.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 256`** (2 nodes): `GovernanceModule`, `governance.module.ts` +- **Thin community `Community 253`** (2 nodes): `GovernanceModule`, `governance.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 257`** (2 nodes): `DatasourceModule`, `datasource.module.ts` +- **Thin community `Community 254`** (2 nodes): `DatasourceModule`, `datasource.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 258`** (2 nodes): `CreateDatasourceDto`, `create-datasource.dto.ts` +- **Thin community `Community 255`** (2 nodes): `CreateDatasourceDto`, `create-datasource.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 259`** (2 nodes): `UploadFileDatasourceDto`, `upload-file-datasource.dto.ts` +- **Thin community `Community 256`** (2 nodes): `UploadFileDatasourceDto`, `upload-file-datasource.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 260`** (2 nodes): `UpdateDatasourceDto`, `update-datasource.dto.ts` +- **Thin community `Community 257`** (2 nodes): `UpdateDatasourceDto`, `update-datasource.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 261`** (2 nodes): `SettingsModule`, `settings.module.ts` +- **Thin community `Community 258`** (2 nodes): `SettingsModule`, `settings.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 262`** (2 nodes): `UpsertRagTaskConfigDto`, `upsert-rag-task-config.dto.ts` +- **Thin community `Community 259`** (2 nodes): `UpsertRagTaskConfigDto`, `upsert-rag-task-config.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 263`** (2 nodes): `PreviewRagProviderModelsDto`, `preview-rag-provider-models.dto.ts` +- **Thin community `Community 260`** (2 nodes): `PreviewRagProviderModelsDto`, `preview-rag-provider-models.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 264`** (2 nodes): `BatchUpdateModelStatusDto`, `batch-update-model-status.dto.ts` +- **Thin community `Community 261`** (2 nodes): `BatchUpdateModelStatusDto`, `batch-update-model-status.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 265`** (2 nodes): `UpdateModelStatusDto`, `update-model-status.dto.ts` +- **Thin community `Community 262`** (2 nodes): `UpdateModelStatusDto`, `update-model-status.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 266`** (2 nodes): `RefreshProviderModelsDto`, `refresh-provider-models.dto.ts` +- **Thin community `Community 263`** (2 nodes): `RefreshProviderModelsDto`, `refresh-provider-models.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 267`** (2 nodes): `CreateProviderDto`, `create-provider.dto.ts` +- **Thin community `Community 264`** (2 nodes): `CreateProviderDto`, `create-provider.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 268`** (2 nodes): `GovernanceAccessModule`, `access.module.ts` +- **Thin community `Community 265`** (2 nodes): `GovernanceAccessModule`, `access.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 269`** (2 nodes): `workspace.module.ts`, `WorkspaceModule` +- **Thin community `Community 266`** (2 nodes): `workspace.module.ts`, `WorkspaceModule` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 270`** (2 nodes): `RemoveWorkspaceMembersDto`, `remove-workspace-members.dto.ts` +- **Thin community `Community 267`** (2 nodes): `RemoveWorkspaceMembersDto`, `remove-workspace-members.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 271`** (2 nodes): `RenameWorkspaceDto`, `rename-workspace.dto.ts` +- **Thin community `Community 268`** (2 nodes): `RenameWorkspaceDto`, `rename-workspace.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 272`** (2 nodes): `GetModelingPreviewDto`, `get-modeling-preview.dto.ts` +- **Thin community `Community 269`** (2 nodes): `GetModelingPreviewDto`, `get-modeling-preview.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 273`** (2 nodes): `UpdateWorkspaceMemberRoleDto`, `update-workspace-member-role.dto.ts` +- **Thin community `Community 270`** (2 nodes): `UpdateWorkspaceMemberRoleDto`, `update-workspace-member-role.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 274`** (2 nodes): `DetectModelingSchemaChangeDto`, `detect-modeling-schema-change.dto.ts` +- **Thin community `Community 271`** (2 nodes): `DetectModelingSchemaChangeDto`, `detect-modeling-schema-change.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 275`** (2 nodes): `ListWorkspaceMembersDto`, `list-workspace-members.dto.ts` +- **Thin community `Community 272`** (2 nodes): `ListWorkspaceMembersDto`, `list-workspace-members.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 276`** (2 nodes): `DeployModelingGraphDto`, `deploy-modeling-graph.dto.ts` +- **Thin community `Community 273`** (2 nodes): `DeployModelingGraphDto`, `deploy-modeling-graph.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 277`** (2 nodes): `AddWorkspaceMemberDto`, `add-workspace-member.dto.ts` +- **Thin community `Community 274`** (2 nodes): `AddWorkspaceMemberDto`, `add-workspace-member.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 278`** (2 nodes): `CreateWorkspaceDto`, `create-workspace.dto.ts` +- **Thin community `Community 275`** (2 nodes): `CreateWorkspaceDto`, `create-workspace.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 279`** (2 nodes): `UserModule`, `user.module.ts` +- **Thin community `Community 276`** (2 nodes): `UserModule`, `user.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 280`** (2 nodes): `BatchDeleteUsersDto`, `batch-delete-users.dto.ts` +- **Thin community `Community 277`** (2 nodes): `BatchDeleteUsersDto`, `batch-delete-users.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 281`** (2 nodes): `UpdateUserStatusDto`, `update-user-status.dto.ts` +- **Thin community `Community 278`** (2 nodes): `UpdateUserStatusDto`, `update-user-status.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 282`** (2 nodes): `UpdateUserDto`, `update-user.dto.ts` +- **Thin community `Community 279`** (2 nodes): `UpdateUserDto`, `update-user.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 283`** (2 nodes): `UserVariableDto`, `user-variable.dto.ts` +- **Thin community `Community 280`** (2 nodes): `UserVariableDto`, `user-variable.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 284`** (2 nodes): `CreateUserDto`, `create-user.dto.ts` +- **Thin community `Community 281`** (2 nodes): `CreateUserDto`, `create-user.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 285`** (2 nodes): `EvalModule`, `eval.module.ts` +- **Thin community `Community 282`** (2 nodes): `EvalModule`, `eval.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 286`** (2 nodes): `RunEvaluationDto`, `run-evaluation.dto.ts` +- **Thin community `Community 283`** (2 nodes): `RunEvaluationDto`, `run-evaluation.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 287`** (2 nodes): `ConversationModule`, `conversation.module.ts` +- **Thin community `Community 284`** (2 nodes): `ConversationModule`, `conversation.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 288`** (2 nodes): `ChatModule`, `chat.module.ts` +- **Thin community `Community 285`** (2 nodes): `ChatModule`, `chat.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 289`** (2 nodes): `SendMessageDto`, `send-message.dto.ts` +- **Thin community `Community 286`** (2 nodes): `SendMessageDto`, `send-message.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 290`** (2 nodes): `ListSessionsDto`, `list-sessions.dto.ts` +- **Thin community `Community 287`** (2 nodes): `ListSessionsDto`, `list-sessions.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 291`** (2 nodes): `CreateSessionDto`, `create-session.dto.ts` +- **Thin community `Community 288`** (2 nodes): `CreateSessionDto`, `create-session.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 292`** (2 nodes): `RenameSessionDto`, `rename-session.dto.ts` +- **Thin community `Community 289`** (2 nodes): `RenameSessionDto`, `rename-session.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 293`** (2 nodes): `SaveViewFromRunDto`, `save-view-from-run.dto.ts` +- **Thin community `Community 290`** (2 nodes): `SaveViewFromRunDto`, `save-view-from-run.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 294`** (2 nodes): `Text2SqlModule`, `text2sql.module.ts` +- **Thin community `Community 291`** (2 nodes): `Text2SqlModule`, `text2sql.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 295`** (2 nodes): `AgentModule`, `agent.module.ts` +- **Thin community `Community 292`** (2 nodes): `AgentModule`, `agent.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 296`** (1 nodes): `semantic-spine.ts` +- **Thin community `Community 293`** (1 nodes): `semantic-spine.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 297`** (1 nodes): `modeling.ts` +- **Thin community `Community 294`** (1 nodes): `modeling.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 298`** (1 nodes): `api.d.ts` +- **Thin community `Community 295`** (1 nodes): `api.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 299`** (1 nodes): `api.ts` +- **Thin community `Community 296`** (1 nodes): `api.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 300`** (1 nodes): `index.ts` +- **Thin community `Community 297`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 301`** (1 nodes): `index.d.ts` +- **Thin community `Community 298`** (1 nodes): `index.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 302`** (1 nodes): `api.contract.spec.ts` +- **Thin community `Community 299`** (1 nodes): `api.contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 303`** (1 nodes): `next-env.d.ts` +- **Thin community `Community 300`** (1 nodes): `jest.config.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 304`** (1 nodes): `vitest.config.ts` +- **Thin community `Community 301`** (1 nodes): `protocol.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 305`** (1 nodes): `assistant-thinking-panel.spec.tsx` +- **Thin community `Community 302`** (1 nodes): `types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 306`** (1 nodes): `session-sidebar.spec.tsx` +- **Thin community `Community 303`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 307`** (1 nodes): `run-visibility-mapper.spec.ts` +- **Thin community `Community 304`** (1 nodes): `next-env.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 308`** (1 nodes): `chat-rag-sync-stream-consistency.spec.tsx` +- **Thin community `Community 305`** (1 nodes): `vitest.config.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 309`** (1 nodes): `settings-modeling-page-context.spec.tsx` +- **Thin community `Community 306`** (1 nodes): `assistant-thinking-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 310`** (1 nodes): `settings-relationship-publish-gate.spec.tsx` +- **Thin community `Community 307`** (1 nodes): `session-sidebar.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 311`** (1 nodes): `settings-users-management.spec.tsx` +- **Thin community `Community 308`** (1 nodes): `run-visibility-mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 312`** (1 nodes): `settings-modeling-model-drawer.spec.tsx` +- **Thin community `Community 309`** (1 nodes): `chat-rag-sync-stream-consistency.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 313`** (1 nodes): `admin-api-client-modeling-contract.spec.ts` +- **Thin community `Community 310`** (1 nodes): `settings-modeling-page-context.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 314`** (1 nodes): `settings-modeling-deploy-panel.spec.tsx` +- **Thin community `Community 311`** (1 nodes): `settings-relationship-publish-gate.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 315`** (1 nodes): `settings-modeling-schema-change-panel.spec.tsx` +- **Thin community `Community 312`** (1 nodes): `settings-users-management.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 316`** (1 nodes): `settings-modeling-sidebar-tree.spec.tsx` +- **Thin community `Community 313`** (1 nodes): `settings-modeling-model-drawer.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 317`** (1 nodes): `settings-modeling-deploy.spec.tsx` +- **Thin community `Community 314`** (1 nodes): `admin-api-client-modeling-contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 318`** (1 nodes): `settings-workspace-management.spec.tsx` +- **Thin community `Community 315`** (1 nodes): `settings-modeling-deploy-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 319`** (1 nodes): `chat-context-envelope-input.spec.tsx` +- **Thin community `Community 316`** (1 nodes): `settings-modeling-schema-change-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 320`** (1 nodes): `chat-sql-preview.integration.spec.tsx` +- **Thin community `Community 317`** (1 nodes): `settings-modeling-sidebar-tree.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 321`** (1 nodes): `settings-workspace-datasource-table-permissions.spec.tsx` +- **Thin community `Community 318`** (1 nodes): `settings-modeling-deploy.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 322`** (1 nodes): `rag-provider-card-grid.spec.tsx` +- **Thin community `Community 319`** (1 nodes): `settings-workspace-management.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 323`** (1 nodes): `rag-rerank-config-panel.spec.tsx` +- **Thin community `Community 320`** (1 nodes): `chat-context-envelope-input.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 324`** (1 nodes): `settings-modeling-view-sync.spec.tsx` +- **Thin community `Community 321`** (1 nodes): `chat-sql-preview.integration.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 325`** (1 nodes): `settings-retired-governance-management.spec.tsx` +- **Thin community `Community 322`** (1 nodes): `settings-workspace-datasource-table-permissions.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 326`** (1 nodes): `sql-preview.spec.tsx` +- **Thin community `Community 323`** (1 nodes): `rag-provider-card-grid.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 327`** (1 nodes): `settings-relationship-modeling-canvas.spec.tsx` +- **Thin community `Community 324`** (1 nodes): `rag-rerank-config-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 328`** (1 nodes): `settings-modeling-schema-change.spec.tsx` +- **Thin community `Community 325`** (1 nodes): `settings-modeling-view-sync.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 329`** (1 nodes): `app-shell.spec.tsx` +- **Thin community `Community 326`** (1 nodes): `settings-retired-governance-management.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 330`** (1 nodes): `settings-workspace-table-permission-entry.spec.tsx` +- **Thin community `Community 327`** (1 nodes): `sql-preview.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 331`** (1 nodes): `settings-modeling-calculated-field-editor.spec.tsx` +- **Thin community `Community 328`** (1 nodes): `settings-relationship-modeling-canvas.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 332`** (1 nodes): `rag-embedding-config-panel.spec.tsx` +- **Thin community `Community 329`** (1 nodes): `settings-modeling-schema-change.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 333`** (1 nodes): `settings-modeling-complete-parity.spec.tsx` +- **Thin community `Community 330`** (1 nodes): `app-shell.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 334`** (1 nodes): `chatbi-result-panel.spec.tsx` +- **Thin community `Community 331`** (1 nodes): `settings-workspace-table-permission-entry.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 335`** (1 nodes): `chat-save-as-view.spec.tsx` +- **Thin community `Community 332`** (1 nodes): `settings-modeling-calculated-field-editor.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 336`** (1 nodes): `rag-foundation-status-card.spec.tsx` +- **Thin community `Community 333`** (1 nodes): `rag-embedding-config-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 337`** (1 nodes): `settings-modeling-metadata-editor.spec.tsx` +- **Thin community `Community 334`** (1 nodes): `settings-modeling-complete-parity.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 338`** (1 nodes): `settings-modeling-flow-node-edge.spec.tsx` +- **Thin community `Community 335`** (1 nodes): `chatbi-result-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 339`** (1 nodes): `chat-demo-flow.spec.tsx` +- **Thin community `Community 336`** (1 nodes): `chat-save-as-view.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 340`** (1 nodes): `chat-mobile-smoke.spec.tsx` +- **Thin community `Community 337`** (1 nodes): `rag-foundation-status-card.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 341`** (1 nodes): `platform-pages-smoke.spec.tsx` +- **Thin community `Community 338`** (1 nodes): `settings-modeling-metadata-editor.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 342`** (1 nodes): `page.tsx` +- **Thin community `Community 339`** (1 nodes): `settings-modeling-flow-node-edge.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 343`** (1 nodes): `sql-preview.tsx` +- **Thin community `Community 340`** (1 nodes): `chat-demo-flow.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 344`** (1 nodes): `steps.tsx` +- **Thin community `Community 341`** (1 nodes): `chat-mobile-smoke.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 345`** (1 nodes): `slider.tsx` +- **Thin community `Community 342`** (1 nodes): `platform-pages-smoke.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 346`** (1 nodes): `progress.tsx` +- **Thin community `Community 343`** (1 nodes): `page.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 347`** (1 nodes): `sonner.tsx` +- **Thin community `Community 344`** (1 nodes): `sql-preview.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 348`** (1 nodes): `button.tsx` +- **Thin community `Community 345`** (1 nodes): `steps.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 349`** (1 nodes): `toggle.tsx` +- **Thin community `Community 346`** (1 nodes): `slider.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 350`** (1 nodes): `workspace-datasource-table-permissions-dialog.tsx` +- **Thin community `Community 347`** (1 nodes): `progress.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 351`** (1 nodes): `relationship-publish-panel.tsx` +- **Thin community `Community 348`** (1 nodes): `sonner.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 352`** (1 nodes): `modeling-flow-toolbar.tsx` +- **Thin community `Community 349`** (1 nodes): `button.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 353`** (1 nodes): `modeling-schema-change-panel.tsx` +- **Thin community `Community 350`** (1 nodes): `toggle.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 354`** (1 nodes): `elk-layout.types.ts` +- **Thin community `Community 351`** (1 nodes): `workspace-datasource-table-permissions-dialog.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 355`** (1 nodes): `message-list.tsx` +- **Thin community `Community 352`** (1 nodes): `relationship-publish-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 356`** (1 nodes): `assistant-composer.tsx` +- **Thin community `Community 353`** (1 nodes): `modeling-flow-toolbar.tsx` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 354`** (1 nodes): `modeling-schema-change-panel.tsx` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 355`** (1 nodes): `elk-layout.types.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 356`** (1 nodes): `message-list.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. - **Thin community `Community 357`** (1 nodes): `workspace-selector-inline.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. @@ -3177,358 +3225,374 @@ Nodes (0): Too small to be a meaningful cluster - may be noise or needs more connections extracted. - **Thin community `Community 368`** (1 nodes): `chat.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 369`** (1 nodes): `text2sql-v2-langgraph-nodes.spec.ts` +- **Thin community `Community 369`** (1 nodes): `semantic-asset-preparer.service.spec.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 370`** (1 nodes): `text2sql-v2-langgraph-nodes.spec.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 371`** (1 nodes): `openai-compatible.client.spec.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 372`** (1 nodes): `chartbi-intent-parser.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 370`** (1 nodes): `openai-compatible.client.spec.ts` +- **Thin community `Community 373`** (1 nodes): `slot-filling-context.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 371`** (1 nodes): `chartbi-intent-parser.spec.ts` +- **Thin community `Community 374`** (1 nodes): `sql-prompt.builder.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 372`** (1 nodes): `slot-filling-context.spec.ts` +- **Thin community `Community 375`** (1 nodes): `resolve-saved-prior-sql.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 373`** (1 nodes): `sql-prompt.builder.spec.ts` +- **Thin community `Community 376`** (1 nodes): `tool-execution-guard.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 374`** (1 nodes): `resolve-saved-prior-sql.node.spec.ts` +- **Thin community `Community 377`** (1 nodes): `clarify.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 375`** (1 nodes): `tool-execution-guard.spec.ts` +- **Thin community `Community 378`** (1 nodes): `text2sql-v2-semantic-context-pack.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 376`** (1 nodes): `clarify.node.spec.ts` +- **Thin community `Community 379`** (1 nodes): `planner-version-lock.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 377`** (1 nodes): `text2sql-v2-semantic-context-pack.spec.ts` +- **Thin community `Community 380`** (1 nodes): `planner-cache.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 378`** (1 nodes): `planner-version-lock.service.spec.ts` +- **Thin community `Community 381`** (1 nodes): `semantic-asset-manifest.builder.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 379`** (1 nodes): `planner-cache.service.spec.ts` +- **Thin community `Community 382`** (1 nodes): `text2sql-stream-event.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 380`** (1 nodes): `text2sql-stream-event.mapper.spec.ts` +- **Thin community `Community 383`** (1 nodes): `tool-events.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 381`** (1 nodes): `tool-events.mapper.spec.ts` +- **Thin community `Community 384`** (1 nodes): `tool-registry.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 382`** (1 nodes): `tool-registry.service.spec.ts` +- **Thin community `Community 385`** (1 nodes): `text2sql-stage-contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 383`** (1 nodes): `text2sql-stage-contract.spec.ts` +- **Thin community `Community 386`** (1 nodes): `chartbi-validator.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 384`** (1 nodes): `chartbi-validator.spec.ts` +- **Thin community `Community 387`** (1 nodes): `semantic-spine-context-pack.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 385`** (1 nodes): `semantic-spine-context-pack.mapper.spec.ts` +- **Thin community `Community 388`** (1 nodes): `llm-model-factory.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 386`** (1 nodes): `llm-model-factory.spec.ts` +- **Thin community `Community 389`** (1 nodes): `rag-event-consumer.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 387`** (1 nodes): `rag-event-consumer.spec.ts` +- **Thin community `Community 390`** (1 nodes): `bootstrap.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 388`** (1 nodes): `bootstrap.spec.ts` +- **Thin community `Community 391`** (1 nodes): `skill-registry.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 389`** (1 nodes): `skill-registry.service.spec.ts` +- **Thin community `Community 392`** (1 nodes): `rrf-fusion.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 390`** (1 nodes): `rrf-fusion.spec.ts` +- **Thin community `Community 393`** (1 nodes): `sql-output-extractor.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 391`** (1 nodes): `sql-output-extractor.spec.ts` +- **Thin community `Community 394`** (1 nodes): `safety-check.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 392`** (1 nodes): `safety-check.node.spec.ts` +- **Thin community `Community 395`** (1 nodes): `text2sql-v2-semantic-plan.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 393`** (1 nodes): `text2sql-v2-semantic-plan.spec.ts` +- **Thin community `Community 396`** (1 nodes): `knowledge-facade-contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 394`** (1 nodes): `knowledge-facade-contract.spec.ts` +- **Thin community `Community 397`** (1 nodes): `chat-langsmith-trace.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 395`** (1 nodes): `chat-langsmith-trace.spec.ts` +- **Thin community `Community 398`** (1 nodes): `text2sql-v2-closeout-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 396`** (1 nodes): `text2sql-v2-closeout-flow.spec.ts` +- **Thin community `Community 399`** (1 nodes): `audit-log.repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 397`** (1 nodes): `audit-log.repository.spec.ts` +- **Thin community `Community 400`** (1 nodes): `rag-foundation-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 398`** (1 nodes): `rag-foundation-replay.spec.ts` +- **Thin community `Community 401`** (1 nodes): `rag-retrieval.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 399`** (1 nodes): `rag-retrieval.service.spec.ts` +- **Thin community `Community 402`** (1 nodes): `user-workspace-repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 400`** (1 nodes): `user-workspace-repository.spec.ts` +- **Thin community `Community 403`** (1 nodes): `rag-foundation-observability.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 401`** (1 nodes): `rag-foundation-observability.spec.ts` +- **Thin community `Community 404`** (1 nodes): `rag-incremental-refresh.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 402`** (1 nodes): `rag-incremental-refresh.spec.ts` +- **Thin community `Community 405`** (1 nodes): `graph-acceleration-fallback.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 403`** (1 nodes): `graph-acceleration-fallback.spec.ts` +- **Thin community `Community 406`** (1 nodes): `agent-rag-main-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 404`** (1 nodes): `agent-rag-main-flow.spec.ts` +- **Thin community `Community 407`** (1 nodes): `text2sql-v2-characterization-gate.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 405`** (1 nodes): `text2sql-v2-characterization-gate.spec.ts` +- **Thin community `Community 408`** (1 nodes): `data-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 406`** (1 nodes): `data-flow.spec.ts` +- **Thin community `Community 409`** (1 nodes): `rule-group-schema.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 407`** (1 nodes): `rule-group-schema.spec.ts` +- **Thin community `Community 410`** (1 nodes): `workspace-modeling-deploy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 408`** (1 nodes): `workspace-modeling-deploy.spec.ts` +- **Thin community `Community 411`** (1 nodes): `session-repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 409`** (1 nodes): `session-repository.spec.ts` +- **Thin community `Community 412`** (1 nodes): `trace-lineage.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 410`** (1 nodes): `trace-lineage.spec.ts` +- **Thin community `Community 413`** (1 nodes): `rag-rerank.integration.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 411`** (1 nodes): `rag-rerank.integration.spec.ts` +- **Thin community `Community 414`** (1 nodes): `relationship-impact-simulation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 412`** (1 nodes): `relationship-impact-simulation.spec.ts` +- **Thin community `Community 415`** (1 nodes): `rag-multi-datasource-orchestrator.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 413`** (1 nodes): `rag-multi-datasource-orchestrator.spec.ts` +- **Thin community `Community 416`** (1 nodes): `text2sql-semantic-asset-reindex.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 414`** (1 nodes): `text2sql-semantic-asset-reindex.spec.ts` +- **Thin community `Community 417`** (1 nodes): `rag-run-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 415`** (1 nodes): `rag-run-replay.spec.ts` +- **Thin community `Community 418`** (1 nodes): `workspace-modeling-schema-change.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 416`** (1 nodes): `workspace-modeling-schema-change.spec.ts` +- **Thin community `Community 419`** (1 nodes): `skill-registry-rag.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 417`** (1 nodes): `skill-registry-rag.spec.ts` +- **Thin community `Community 420`** (1 nodes): `semantic-registry.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 418`** (1 nodes): `semantic-registry.spec.ts` +- **Thin community `Community 421`** (1 nodes): `chat-run-template-trace.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 419`** (1 nodes): `chat-run-template-trace.spec.ts` +- **Thin community `Community 422`** (1 nodes): `workspace-datasource-table-permissions-schema.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 420`** (1 nodes): `workspace-datasource-table-permissions-schema.spec.ts` +- **Thin community `Community 423`** (1 nodes): `policy-evaluator.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 421`** (1 nodes): `policy-evaluator.spec.ts` +- **Thin community `Community 424`** (1 nodes): `rag-cache-version-invalidation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 422`** (1 nodes): `rag-cache-version-invalidation.spec.ts` +- **Thin community `Community 425`** (1 nodes): `rag-index-builder.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 423`** (1 nodes): `rag-index-builder.spec.ts` +- **Thin community `Community 426`** (1 nodes): `prisma-graph-baseline.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 424`** (1 nodes): `prisma-graph-baseline.spec.ts` +- **Thin community `Community 427`** (1 nodes): `agent-rag-degrade-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 425`** (1 nodes): `agent-rag-degrade-flow.spec.ts` +- **Thin community `Community 428`** (1 nodes): `langsmith-coverage.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 426`** (1 nodes): `langsmith-coverage.spec.ts` +- **Thin community `Community 429`** (1 nodes): `session-management.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 427`** (1 nodes): `session-management.spec.ts` +- **Thin community `Community 430`** (1 nodes): `text2sql-v2-eval-gate.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 428`** (1 nodes): `text2sql-v2-eval-gate.spec.ts` +- **Thin community `Community 431`** (1 nodes): `planner-cache-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 429`** (1 nodes): `planner-cache-replay.spec.ts` +- **Thin community `Community 432`** (1 nodes): `planner-version-lock.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 430`** (1 nodes): `planner-version-lock.spec.ts` +- **Thin community `Community 433`** (1 nodes): `agent-context-pack.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 431`** (1 nodes): `agent-context-pack.spec.ts` +- **Thin community `Community 434`** (1 nodes): `graph-acceleration-circuit-breaker.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 432`** (1 nodes): `graph-acceleration-circuit-breaker.spec.ts` +- **Thin community `Community 435`** (1 nodes): `agent-sql-prompt-template-runtime.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 433`** (1 nodes): `agent-sql-prompt-template-runtime.spec.ts` +- **Thin community `Community 436`** (1 nodes): `agent-main-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 434`** (1 nodes): `agent-main-flow.spec.ts` +- **Thin community `Community 437`** (1 nodes): `datasource-repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 435`** (1 nodes): `datasource-repository.spec.ts` +- **Thin community `Community 438`** (1 nodes): `rag-performance-budget.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 436`** (1 nodes): `rag-performance-budget.spec.ts` +- **Thin community `Community 439`** (1 nodes): `chat-context-envelope-accuracy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 437`** (1 nodes): `chat-context-envelope-accuracy.spec.ts` +- **Thin community `Community 440`** (1 nodes): `rag-index-activation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 438`** (1 nodes): `rag-index-activation.spec.ts` +- **Thin community `Community 441`** (1 nodes): `datasource-access-policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 439`** (1 nodes): `datasource-access-policy.spec.ts` +- **Thin community `Community 442`** (1 nodes): `eval-langsmith-trace.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 440`** (1 nodes): `eval-langsmith-trace.spec.ts` +- **Thin community `Community 443`** (1 nodes): `rag-quality.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 441`** (1 nodes): `rag-quality.spec.ts` +- **Thin community `Community 444`** (1 nodes): `eval-report.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 442`** (1 nodes): `eval-report.spec.ts` +- **Thin community `Community 445`** (1 nodes): `semantic-spine-shadow-gate.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 443`** (1 nodes): `semantic-spine-shadow-gate.spec.ts` +- **Thin community `Community 446`** (1 nodes): `chat-persistence-retry.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 444`** (1 nodes): `chat-persistence-retry.spec.ts` +- **Thin community `Community 447`** (1 nodes): `saved-prior-sql-ingestion.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 445`** (1 nodes): `saved-prior-sql-ingestion.spec.ts` +- **Thin community `Community 448`** (1 nodes): `r3-gate-rehearsal.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 446`** (1 nodes): `r3-gate-rehearsal.spec.ts` +- **Thin community `Community 449`** (1 nodes): `r3-semantic-registry-migration.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 447`** (1 nodes): `r3-semantic-registry-migration.spec.ts` +- **Thin community `Community 450`** (1 nodes): `workspace-service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 448`** (1 nodes): `workspace-service.spec.ts` +- **Thin community `Community 451`** (1 nodes): `delivery-contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 449`** (1 nodes): `rag-document-factory.spec.ts` +- **Thin community `Community 452`** (1 nodes): `rag-document-factory.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 450`** (1 nodes): `r6-gate-readiness.spec.ts` +- **Thin community `Community 453`** (1 nodes): `r6-gate-readiness.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 451`** (1 nodes): `user-service.spec.ts` +- **Thin community `Community 454`** (1 nodes): `user-service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 452`** (1 nodes): `provider-fallback.spec.ts` +- **Thin community `Community 455`** (1 nodes): `provider-fallback.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 453`** (1 nodes): `saved-prior-sql-quality.spec.ts` +- **Thin community `Community 456`** (1 nodes): `saved-prior-sql-quality.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 454`** (1 nodes): `workspace-api.spec.ts` +- **Thin community `Community 457`** (1 nodes): `workspace-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 455`** (1 nodes): `r6-release-rollback-rehearsal.e2e-spec.ts` +- **Thin community `Community 458`** (1 nodes): `r6-release-rollback-rehearsal.e2e-spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 456`** (1 nodes): `datasource-visibility-api.spec.ts` +- **Thin community `Community 459`** (1 nodes): `datasource-visibility-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 457`** (1 nodes): `workspace-datasource-api.spec.ts` +- **Thin community `Community 460`** (1 nodes): `chat-stream-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 458`** (1 nodes): `graph-acceleration-chaos.e2e-spec.ts` +- **Thin community `Community 461`** (1 nodes): `workspace-datasource-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 459`** (1 nodes): `clarification-hybrid-balance.acceptance.spec.ts` +- **Thin community `Community 462`** (1 nodes): `graph-acceleration-chaos.e2e-spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 460`** (1 nodes): `datasource-workflow-api.spec.ts` +- **Thin community `Community 463`** (1 nodes): `clarification-hybrid-balance.acceptance.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 461`** (1 nodes): `rag-multi-datasource-isolation.spec.ts` +- **Thin community `Community 464`** (1 nodes): `datasource-workflow-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 462`** (1 nodes): `settings-api.spec.ts` +- **Thin community `Community 465`** (1 nodes): `rag-multi-datasource-isolation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 463`** (1 nodes): `workspace-datasource-audit.spec.ts` +- **Thin community `Community 466`** (1 nodes): `settings-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 464`** (1 nodes): `chat-api.spec.ts` +- **Thin community `Community 467`** (1 nodes): `workspace-datasource-audit.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 465`** (1 nodes): `rule-group-api.spec.ts` +- **Thin community `Community 468`** (1 nodes): `chat-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 466`** (1 nodes): `chat-table-permissions-policy.spec.ts` +- **Thin community `Community 469`** (1 nodes): `rule-group-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 467`** (1 nodes): `rag-r6-mixed-load.spec.ts` +- **Thin community `Community 470`** (1 nodes): `chat-table-permissions-policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 468`** (1 nodes): `rag-cache-budget-load.spec.ts` +- **Thin community `Community 471`** (1 nodes): `rag-r6-mixed-load.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 469`** (1 nodes): `browser.ts` +- **Thin community `Community 472`** (1 nodes): `rag-cache-budget-load.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 470`** (1 nodes): `client.ts` +- **Thin community `Community 473`** (1 nodes): `browser.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 471`** (1 nodes): `models.ts` +- **Thin community `Community 474`** (1 nodes): `client.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 472`** (1 nodes): `commonInputTypes.ts` +- **Thin community `Community 475`** (1 nodes): `models.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 473`** (1 nodes): `enums.ts` +- **Thin community `Community 476`** (1 nodes): `commonInputTypes.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 474`** (1 nodes): `prismaNamespace.ts` +- **Thin community `Community 477`** (1 nodes): `enums.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 475`** (1 nodes): `prismaNamespaceBrowser.ts` +- **Thin community `Community 478`** (1 nodes): `prismaNamespace.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 476`** (1 nodes): `WorkspaceDatasourceTablePermission.ts` +- **Thin community `Community 479`** (1 nodes): `prismaNamespaceBrowser.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 477`** (1 nodes): `WorkspaceDatasourceTablePermissionSet.ts` +- **Thin community `Community 480`** (1 nodes): `WorkspaceDatasourceTablePermission.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 478`** (1 nodes): `ModelCatalog.ts` +- **Thin community `Community 481`** (1 nodes): `WorkspaceDatasourceTablePermissionSet.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 479`** (1 nodes): `RagDocument.ts` +- **Thin community `Community 482`** (1 nodes): `ModelCatalog.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 480`** (1 nodes): `WorkspaceDatasourceBinding.ts` +- **Thin community `Community 483`** (1 nodes): `RagDocument.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 481`** (1 nodes): `AgentAuditLog.ts` +- **Thin community `Community 484`** (1 nodes): `WorkspaceDatasourceBinding.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 482`** (1 nodes): `SqlRun.ts` +- **Thin community `Community 485`** (1 nodes): `AgentAuditLog.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 483`** (1 nodes): `Workspace.ts` +- **Thin community `Community 486`** (1 nodes): `SqlRun.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 484`** (1 nodes): `WorkspaceMember.ts` +- **Thin community `Community 487`** (1 nodes): `Workspace.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 485`** (1 nodes): `RagChunkIndexEntry.ts` +- **Thin community `Community 488`** (1 nodes): `WorkspaceMember.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 486`** (1 nodes): `SemanticEdge.ts` +- **Thin community `Community 489`** (1 nodes): `RagChunkIndexEntry.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 487`** (1 nodes): `GraphSnapshot.ts` +- **Thin community `Community 490`** (1 nodes): `SemanticEdge.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 488`** (1 nodes): `RagIndexVersion.ts` +- **Thin community `Community 491`** (1 nodes): `GraphSnapshot.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 489`** (1 nodes): `GlossaryTerm.ts` +- **Thin community `Community 492`** (1 nodes): `RagIndexVersion.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 490`** (1 nodes): `SemanticMemory.ts` +- **Thin community `Community 493`** (1 nodes): `GlossaryTerm.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 491`** (1 nodes): `Datasource.ts` +- **Thin community `Community 494`** (1 nodes): `SemanticMemory.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 492`** (1 nodes): `ProviderConfig.ts` +- **Thin community `Community 495`** (1 nodes): `Datasource.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 493`** (1 nodes): `WorkspaceModelingGraphRevision.ts` +- **Thin community `Community 496`** (1 nodes): `ProviderConfig.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 494`** (1 nodes): `SemanticSpineSnapshot.ts` +- **Thin community `Community 497`** (1 nodes): `WorkspaceModelingGraphRevision.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 495`** (1 nodes): `Message.ts` +- **Thin community `Community 498`** (1 nodes): `SemanticSpineSnapshot.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 496`** (1 nodes): `Session.ts` +- **Thin community `Community 499`** (1 nodes): `Message.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 497`** (1 nodes): `RagChunk.ts` +- **Thin community `Community 500`** (1 nodes): `Session.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 498`** (1 nodes): `SemanticRegistryTerm.ts` +- **Thin community `Community 501`** (1 nodes): `RagChunk.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 499`** (1 nodes): `GlossaryAnchor.ts` +- **Thin community `Community 502`** (1 nodes): `SemanticRegistryTerm.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 500`** (1 nodes): `PlatformUser.ts` +- **Thin community `Community 503`** (1 nodes): `GlossaryAnchor.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 501`** (1 nodes): `RagTaskConfig.ts` +- **Thin community `Community 504`** (1 nodes): `PlatformUser.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 502`** (1 nodes): `SemanticRegistryVersion.ts` +- **Thin community `Community 505`** (1 nodes): `RagTaskConfig.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 503`** (1 nodes): `RagRunReplay.ts` +- **Thin community `Community 506`** (1 nodes): `SemanticRegistryVersion.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 504`** (1 nodes): `EvaluationReport.ts` +- **Thin community `Community 507`** (1 nodes): `RagRunReplay.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 505`** (1 nodes): `PromptTemplate.ts` +- **Thin community `Community 508`** (1 nodes): `EvaluationReport.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 506`** (1 nodes): `express.d.ts` +- **Thin community `Community 509`** (1 nodes): `PromptTemplate.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 507`** (1 nodes): `knowledge.ts` +- **Thin community `Community 510`** (1 nodes): `express.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 508`** (1 nodes): `embedding-gateway.interface.ts` +- **Thin community `Community 511`** (1 nodes): `knowledge.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 509`** (1 nodes): `llm-gateway.interface.ts` +- **Thin community `Community 512`** (1 nodes): `embedding-gateway.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 510`** (1 nodes): `provider-capabilities.ts` +- **Thin community `Community 513`** (1 nodes): `llm-gateway.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 511`** (1 nodes): `provider-adapter.interface.ts` +- **Thin community `Community 514`** (1 nodes): `provider-capabilities.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 512`** (1 nodes): `index.ts` +- **Thin community `Community 515`** (1 nodes): `provider-adapter.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 513`** (1 nodes): `index.ts` +- **Thin community `Community 516`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 514`** (1 nodes): `modeling-graph.types.ts` +- **Thin community `Community 517`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 515`** (1 nodes): `index.ts` +- **Thin community `Community 518`** (1 nodes): `modeling-graph.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 516`** (1 nodes): `langsmith.types.ts` +- **Thin community `Community 519`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 517`** (1 nodes): `rag-retrieval.types.ts` +- **Thin community `Community 520`** (1 nodes): `langsmith.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 518`** (1 nodes): `knowledge-rag.contract.ts` +- **Thin community `Community 521`** (1 nodes): `rag-retrieval.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 519`** (1 nodes): `knowledge-memory.contract.ts` +- **Thin community `Community 522`** (1 nodes): `knowledge-rag.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 520`** (1 nodes): `knowledge-facade.contract.ts` +- **Thin community `Community 523`** (1 nodes): `knowledge-rag-payload.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 521`** (1 nodes): `knowledge-semantic-registry.contract.ts` +- **Thin community `Community 524`** (1 nodes): `knowledge-memory.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 522`** (1 nodes): `knowledge-glossary.contract.ts` +- **Thin community `Community 525`** (1 nodes): `knowledge-facade.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 523`** (1 nodes): `rag-retrieval.types.ts` +- **Thin community `Community 526`** (1 nodes): `knowledge-semantic-registry.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 524`** (1 nodes): `modeling-graph.validator.ts` +- **Thin community `Community 527`** (1 nodes): `knowledge-glossary.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 525`** (1 nodes): `modeling-graph.types.ts` +- **Thin community `Community 528`** (1 nodes): `rag-retrieval.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 526`** (1 nodes): `semantic-spine.types.ts` +- **Thin community `Community 529`** (1 nodes): `semantic-asset-manifest.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 527`** (1 nodes): `index.ts` +- **Thin community `Community 530`** (1 nodes): `semantic-asset-source-snapshot.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 528`** (1 nodes): `query-executor.interface.ts` +- **Thin community `Community 531`** (1 nodes): `semantic-asset-reason-codes.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 529`** (1 nodes): `text2sql-v2.types.ts` +- **Thin community `Community 532`** (1 nodes): `modeling-graph.validator.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 530`** (1 nodes): `text2sql-smart-defaults.bundle.ts` +- **Thin community `Community 533`** (1 nodes): `modeling-graph.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 531`** (1 nodes): `text2sql-workflow-runner.service.ts` +- **Thin community `Community 534`** (1 nodes): `semantic-spine.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 532`** (1 nodes): `run-v2-langgraph.stage.ts` +- **Thin community `Community 535`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 533`** (1 nodes): `text2sql-stage-name.ts` +- **Thin community `Community 536`** (1 nodes): `query-executor.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 534`** (1 nodes): `text2sql-stage.interface.ts` +- **Thin community `Community 537`** (1 nodes): `text2sql-v2.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 535`** (1 nodes): `text2sql-stage-result.ts` +- **Thin community `Community 538`** (1 nodes): `text2sql-smart-defaults.bundle.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 536`** (1 nodes): `text2sql-run-context.ts` +- **Thin community `Community 539`** (1 nodes): `text2sql-workflow-runner.service.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 537`** (1 nodes): `run-view-support.guard.ts` +- **Thin community `Community 540`** (1 nodes): `run-v2-langgraph.stage.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 541`** (1 nodes): `text2sql-stage-name.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 542`** (1 nodes): `text2sql-stage.interface.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 543`** (1 nodes): `text2sql-stage-result.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 544`** (1 nodes): `text2sql-run-context.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 545`** (1 nodes): `run-view-support.guard.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. ## Suggested Questions _Questions this graph is uniquely positioned to answer:_ -- **Why does `ok()` connect `Community 5` to `Community 24`, `Community 1`, `Community 6`, `Community 15`?** - _High betweenness centrality (0.028) - this node is a cross-community bridge._ -- **Why does `collectClarificationBalanceGate()` connect `Community 34` to `Community 1`, `Community 2`, `Community 4`, `Community 8`, `Community 26`?** - _High betweenness centrality (0.013) - this node is a cross-community bridge._ +- **Why does `ok()` connect `Community 4` to `Community 1`, `Community 12`, `Community 23`?** + _High betweenness centrality (0.023) - this node is a cross-community bridge._ +- **Why does `DatasourceService` connect `Community 7` to `Community 1`?** + _High betweenness centrality (0.020) - this node is a cross-community bridge._ - **Are the 89 inferred relationships involving `ok()` (e.g. with `.applyFeedback()` and `.report()`) actually correct?** _`ok()` has 89 INFERRED edges - model-reasoned connections that need verification._ - **What connects `ELK`, `ElkLayoutTimeoutError`, `AppModule` to the rest of the system?** _93 weakly-connected nodes found - possible documentation gaps or missing edges._ - **Should `Community 0` be split into smaller, more focused modules?** - _Cohesion score 0.02 - nodes in this community are weakly interconnected._ + _Cohesion score 0.01 - nodes in this community are weakly interconnected._ - **Should `Community 1` be split into smaller, more focused modules?** _Cohesion score 0.02 - nodes in this community are weakly interconnected._ - **Should `Community 2` be split into smaller, more focused modules?** diff --git a/graphify-out/graph.html b/graphify-out/graph.html index df5f783..6f976a3 100644 --- a/graphify-out/graph.html +++ b/graphify-out/graph.html @@ -50,12 +50,12 @@

Node Info

Communities

-
4314 nodes · 8504 edges · 538 communities
+
4480 nodes · 8904 edges · 546 communities