diff --git a/README.md b/README.md index 6f5fe35..d0fc57e 100644 --- a/README.md +++ b/README.md @@ -252,6 +252,10 @@ pnpm test:frontend - 质量门禁：`pnpm --filter @text2sql/backend run lint && pnpm --filter @text2sql/backend run build && pnpm --filter @text2sql/backend run test` - R1 离线 Gate：`pnpm --filter @text2sql/backend exec jest test/e2e/stage1-acceptance.spec.ts --runInBand` - 术语 selected_context 门禁：`pnpm --filter @text2sql/backend test -- glossary-selected-context-gate.spec.ts --runInBand` +- clarification hybrid balance gate： + - 观测模式：`pnpm --filter @text2sql/backend run collect:clarification-balance-gate` + - 强门禁模式（失败返回非 0）：`pnpm --filter @text2sql/backend run collect:clarification-balance-gate:strict` + - 样本定义：`apps/backend/test/fixtures/clarification-balance-cases.json` - modeling parity shadow gate： - 观测模式：`pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate` - 强门禁模式（失败返回非 0）：`pnpm --filter @text2sql/backend run collect:modeling-parity-shadow-gate:strict` @@ -265,6 +269,24 @@ pnpm test:frontend - `GET /api/v1/settings/models`（管理员上下文） - CI 可参考：`.github/workflows/backend-prisma-quality.yml` +## Clarification Hybrid Balanced Rollout Runbook + +1. 混合门控质量信号采集（观测）： +```bash +pnpm --filter @text2sql/backend run collect:clarification-balance-gate +``` +2. 发布门禁（严格）： +```bash +pnpm --filter @text2sql/backend run collect:clarification-balance-gate:strict +``` +3. 一键回退（rules-only）： + - 设置 `CLARIFICATION_HYBRID_KILL_SWITCH_RULES_ONLY=true` + - 重启后端 +4. 回退后验证： + - `node tests/smoke/nginx-dev-gateway-smoke.mjs` + - `pnpm --filter @text2sql/backend run collect:clarification-balance-gate:strict` +5. 详细步骤与阈值说明：`docs/runbooks/clarification-hybrid-balanced-rollout.md` + ## Modeling Parity Shadow Gate Rollout Runbook 1. 采集并生成报告（观测模式）： diff --git a/apps/backend/.env.example b/apps/backend/.env.example index 6b21094..ea00fc7 100644 --- a/apps/backend/.env.example +++ b/apps/backend/.env.example @@ -26,6 +26,27 @@ LLM_TIMEOUT_MS=30000 LLM_MOCK_MODE=false LLM_FALLBACK_PROVIDERS=siliconflow,minimax AGENT_PLANNING_SCAFFOLD_ENABLED=false + +# Clarification hybrid gate defaults: +# - enabled by default +# - one-click rollback via kill switch (rules-only) +# - timeout fallback to rules-only path +CLARIFICATION_HYBRID_ENABLED=true +CLARIFICATION_HYBRID_KILL_SWITCH_RULES_ONLY=false +CLARIFICATION_HYBRID_TIMEOUT_MS=1200 + +# Clarification balance gate thresholds (collect-clarification-balance-gate.ts) +# Strict mode returns non-zero when gate fails. +CLARIFICATION_BALANCE_GATE_MIN_SAMPLES=4 +CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE=0.2 +CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE=0.8 +CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE=0.2 +CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE=0.2 +CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE=0.7 +CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS=1.5 +CLARIFICATION_BALANCE_GATE_STRICT=false +CLARIFICATION_BALANCE_GATE_REPORT_PATH=../../data/reports/clarification-hybrid-balance/gate-summary.json + SQL_SAFETY_SOFT_WARN_MAX_LENGTH=600 R1_GATE_WINDOW_MINUTES=60 R1_GATE_MIN_SAMPLES=10 diff --git a/apps/backend/data/reports/modeling-parity-shadow/gate-summary.json b/apps/backend/data/reports/modeling-parity-shadow/gate-summary.json index dde9cbf..5d35b64 100644 --- a/apps/backend/data/reports/modeling-parity-shadow/gate-summary.json +++ b/apps/backend/data/reports/modeling-parity-shadow/gate-summary.json @@ -1,5 +1,5 @@ { - "generatedAt": "2026-04-24T04:43:07.359Z", + "generatedAt": "2026-04-25T06:47:56.724Z", "inputs": { "relationshipPlatform": "/Users/lienli/Documents/GitHub/text2sql/apps/backend/data/reports/relationship-platform-shadow/samples.json", "semanticSpine": "/Users/lienli/Documents/GitHub/text2sql/apps/backend/data/reports/semantic-spine-shadow/samples.json", @@ -10,7 +10,7 @@ "semanticSpine": "/Users/lienli/Documents/GitHub/text2sql/apps/backend/data/reports/semantic-spine-shadow/gate-summary.json", "modelingWorkspace": "/Users/lienli/Documents/GitHub/text2sql/apps/backend/data/reports/modeling-parity-shadow/gate-summary.json" }, - "strictMode": false, + "strictMode": true, "gatePass": false, "reasons": [ "relationship_platform:sample_not_ready", diff --git a/apps/backend/data/reports/relationship-platform-shadow/gate-summary.json b/apps/backend/data/reports/relationship-platform-shadow/gate-summary.json index 3c32bfb..8f176c4 100644 --- a/apps/backend/data/reports/relationship-platform-shadow/gate-summary.json +++ b/apps/backend/data/reports/relationship-platform-shadow/gate-summary.json @@ -1,5 +1,5 @@ { - "generatedAt": "2026-04-24T04:43:07.331Z", + "generatedAt": "2026-04-25T06:47:56.690Z", "inputPath": "/Users/lienli/Documents/GitHub/text2sql/apps/backend/data/reports/relationship-platform-shadow/samples.json", "thresholds": { "minSamples": 30, diff --git a/apps/backend/data/reports/semantic-spine-shadow/gate-summary.json b/apps/backend/data/reports/semantic-spine-shadow/gate-summary.json index 5eeb0eb..0f4c1c9 100644 --- a/apps/backend/data/reports/semantic-spine-shadow/gate-summary.json +++ b/apps/backend/data/reports/semantic-spine-shadow/gate-summary.json @@ -1,5 +1,5 @@ { - "generatedAt": "2026-04-24T04:43:07.354Z", + "generatedAt": "2026-04-25T06:47:56.719Z", "inputPath": "/Users/lienli/Documents/GitHub/text2sql/apps/backend/data/reports/semantic-spine-shadow/samples.json", "thresholds": { "minSamples": 30, diff --git a/apps/backend/package.json b/apps/backend/package.json index 9aec490..5e0be91 100644 --- a/apps/backend/package.json +++ b/apps/backend/package.json @@ -18,6 +18,8 @@ "prisma:migrate": "node scripts/prisma-with-database-url.cjs migrate dev", "prisma:studio": "node scripts/prisma-with-database-url.cjs studio", "prisma:verify-empty-db": "node scripts/prisma-verify-empty-db.cjs", + "collect:clarification-balance-gate": "TS_NODE_TRANSPILE_ONLY=1 ts-node --project tsconfig.json scripts/collect-clarification-balance-gate.ts", + "collect:clarification-balance-gate:strict": "TS_NODE_TRANSPILE_ONLY=1 ts-node --project tsconfig.json scripts/collect-clarification-balance-gate.ts --strict", "collect:modeling-parity-shadow-gate": "node scripts/collect-modeling-parity-shadow-gate.mjs", "collect:modeling-parity-shadow-gate:strict": "node scripts/collect-modeling-parity-shadow-gate.mjs --fail-on-gate" }, diff --git a/apps/backend/scripts/collect-clarification-balance-gate.ts b/apps/backend/scripts/collect-clarification-balance-gate.ts new file mode 100644 index 0000000..dea034f --- /dev/null +++ b/apps/backend/scripts/collect-clarification-balance-gate.ts @@ -0,0 +1,652 @@ +import { randomUUID } from "node:crypto"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { dirname, resolve } from "node:path"; +import { Test, type TestingModule } from "@nestjs/testing"; +import { AppModule } from "../src/app.module"; +import { GraphBuilderService } from "../src/modules/conversation/agent/graph/graph.builder"; +import { createSeededSqliteFixture } from "../test/support/sqlite-fixture"; + +type RunStatus = + | "clarification" + | "executionResult" + | "rejected" + | "failed"; + +type ScenarioLabel = + | "clarify_to_aggregate" + | "clarify_to_compare" + | "skip_but_strict" + | "metadata_bypass"; + +interface ClarificationBalanceCase { + id: string; + scenario: ScenarioLabel; + initialQuestion: string; + followUpQuestion?: string; + expected: { + shouldClarify: boolean; + metadataBypass: boolean; + strictSemanticPath: boolean; + maxClarificationRounds: number; + }; +} + +interface ClarificationBalanceFixture { + version: string; + generatedAt: string; + cases: ClarificationBalanceCase[]; +} + +interface GateThresholds { + minSamples: number; + minTriggerRate: number; + maxTriggerRate: number; + maxFalsePositiveRate: number; + maxFalseNegativeRate: number; + minPostClarifySemanticPassRate: number; + maxAverageClarificationRounds: number; +} + +interface CaseDiagnostics { + caseId: string; + scenario: ScenarioLabel; + expectedShouldClarify: boolean; + triggeredClarification: boolean; + clarificationRounds: number; + firstRunStatus: RunStatus; + followUpRunStatus?: RunStatus; + metadataBypassDetected: boolean; + strictSemanticPathDetected: boolean; + postClarifySemanticPass: boolean; + semanticPlanStatus?: string; + errors: string[]; +} + +interface ClarificationBalanceGateReport { + generatedAt: string; + strictMode: boolean; + fixture: { + path: string; + version: string; + generatedAt: string; + sampleSize: number; + }; + sampleReady: boolean; + gatePass: boolean; + reasons: string[]; + thresholds: GateThresholds; + metrics: { + sampleSize: number; + clarificationTriggered: number; + triggerRate: number; + falsePositiveRate: number; + falseNegativeRate: number; + postClarifySemanticPassRate: number; + averageClarificationRounds: number; + }; + diagnostics: { + falsePositiveCaseIds: string[]; + falseNegativeCaseIds: string[]; + metadataBypassMismatchCaseIds: string[]; + strictSemanticPathMismatchCaseIds: string[]; + clarificationRoundExceededCaseIds: string[]; + caseErrors: Array<{ caseId: string; message: string }>; + cases: CaseDiagnostics[]; + }; +} + +interface CollectOptions { + fixturePath?: string; + outputPath?: string; + strictMode?: boolean; +} + +interface ParsedArgs { + fixturePath: string; + outputPath: string; + strictMode: boolean; +} + +const DEFAULT_FIXTURE_PATH = resolve( + process.cwd(), + "test/fixtures/clarification-balance-cases.json" +); +const DEFAULT_OUTPUT_PATH = resolve( + process.cwd(), + "../../data/reports/clarification-hybrid-balance/gate-summary.json" +); + +function parseBoolean(value: string | undefined): boolean { + if (!value) { + return false; + } + const normalized = value.trim().toLowerCase(); + return normalized === "1" || normalized === "true" || normalized === "yes"; +} + +function toNumber(value: string | undefined, fallback: number): number { + const parsed = Number(value); + return Number.isFinite(parsed) ? parsed : fallback; +} + +function toRate(value: number): number { + if (!Number.isFinite(value)) { + return 0; + } + return Number(value.toFixed(6)); +} + +function readThresholds(): GateThresholds { + return { + minSamples: Math.max( + 1, + Math.floor(toNumber(process.env.CLARIFICATION_BALANCE_GATE_MIN_SAMPLES, 4)) + ), + minTriggerRate: toNumber(process.env.CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE, 0.2), + maxTriggerRate: toNumber(process.env.CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE, 0.8), + maxFalsePositiveRate: toNumber( + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE, + 0.2 + ), + maxFalseNegativeRate: toNumber( + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE, + 0.2 + ), + minPostClarifySemanticPassRate: toNumber( + process.env.CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE, + 0.7 + ), + maxAverageClarificationRounds: toNumber( + process.env.CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS, + 1.5 + ) + }; +} + +function parseArgs(argv: string[]): ParsedArgs { + const args = argv.slice(2); + const strictByArg = args.includes("--strict") || args.includes("--fail-on-gate"); + const positional = args.filter( + (item) => item !== "--strict" && item !== "--fail-on-gate" + ); + return { + fixturePath: resolve(positional[0] ?? DEFAULT_FIXTURE_PATH), + outputPath: resolve( + positional[1] ?? + process.env.CLARIFICATION_BALANCE_GATE_REPORT_PATH ?? + DEFAULT_OUTPUT_PATH + ), + strictMode: + strictByArg || parseBoolean(process.env.CLARIFICATION_BALANCE_GATE_STRICT) + }; +} + +function ensureFixture(input: unknown, fixturePath: string): ClarificationBalanceFixture { + if (typeof input !== "object" || input === null || Array.isArray(input)) { + throw new Error(`Invalid fixture format: ${fixturePath}`); + } + const payload = input as Record; + if (typeof payload.version !== "string" || payload.version.trim().length === 0) { + throw new Error(`Invalid fixture version in ${fixturePath}`); + } + if ( + typeof payload.generatedAt !== "string" || + payload.generatedAt.trim().length === 0 + ) { + throw new Error(`Invalid fixture generatedAt in ${fixturePath}`); + } + if (!Array.isArray(payload.cases)) { + throw new Error(`Invalid fixture cases array in ${fixturePath}`); + } + if (payload.cases.length === 0) { + throw new Error(`Empty fixture cases in ${fixturePath}`); + } + + const knownScenarios = new Set([ + "clarify_to_aggregate", + "clarify_to_compare", + "skip_but_strict", + "metadata_bypass" + ]); + const cases = payload.cases.map((item, index) => { + if (typeof item !== "object" || item === null || Array.isArray(item)) { + throw new Error(`Invalid case at index ${index} in ${fixturePath}`); + } + const raw = item as Record; + const caseId = typeof raw.id === "string" ? raw.id : `index-${index}`; + if (!knownScenarios.has(raw.scenario as ScenarioLabel)) { + throw new Error( + `Unknown scenario label for case ${caseId}: ${String(raw.scenario)}` + ); + } + if (typeof raw.initialQuestion !== "string" || raw.initialQuestion.trim().length === 0) { + throw new Error(`Case ${caseId} has empty initialQuestion`); + } + if (typeof raw.expected !== "object" || raw.expected === null || Array.isArray(raw.expected)) { + throw new Error(`Case ${caseId} has invalid expected block`); + } + const expected = raw.expected as Record; + if (typeof expected.shouldClarify !== "boolean") { + throw new Error(`Case ${caseId} expected.shouldClarify must be boolean`); + } + if (typeof expected.metadataBypass !== "boolean") { + throw new Error(`Case ${caseId} expected.metadataBypass must be boolean`); + } + if (typeof expected.strictSemanticPath !== "boolean") { + throw new Error(`Case ${caseId} expected.strictSemanticPath must be boolean`); + } + if ( + typeof expected.maxClarificationRounds !== "number" || + !Number.isFinite(expected.maxClarificationRounds) + ) { + throw new Error( + `Case ${caseId} expected.maxClarificationRounds must be a number` + ); + } + return { + id: caseId, + scenario: raw.scenario as ScenarioLabel, + initialQuestion: raw.initialQuestion, + followUpQuestion: + typeof raw.followUpQuestion === "string" ? raw.followUpQuestion : undefined, + expected: { + shouldClarify: expected.shouldClarify, + metadataBypass: expected.metadataBypass, + strictSemanticPath: expected.strictSemanticPath, + maxClarificationRounds: expected.maxClarificationRounds + } + } satisfies ClarificationBalanceCase; + }); + + return { + version: payload.version, + generatedAt: payload.generatedAt, + cases + }; +} + +function parseOutputSummary( + value: string | undefined +): Record | undefined { + if (!value || value.trim().length === 0) { + return undefined; + } + try { + const parsed = JSON.parse(value); + if (typeof parsed === "object" && parsed !== null && !Array.isArray(parsed)) { + return parsed as Record; + } + } catch { + return undefined; + } + return undefined; +} + +function findTraceStep(run: { trace?: { steps?: unknown[] } }, node: string) { + const steps = run.trace?.steps; + if (!Array.isArray(steps)) { + return undefined; + } + return steps.find((item) => { + if (typeof item !== "object" || item === null || Array.isArray(item)) { + return false; + } + return (item as Record).node === node; + }) as Record | undefined; +} + +function readRunStatus(value: unknown): RunStatus { + if ( + value === "clarification" || + value === "executionResult" || + value === "rejected" || + value === "failed" + ) { + return value; + } + return "failed"; +} + +function detectMetadataBypass(run: { + status: RunStatus; + trace?: { + steps?: unknown[]; + clarificationDecision?: { + decisionSource?: unknown; + bypassed?: unknown; + bypassReasonCode?: unknown; + }; + }; +}): boolean { + if (run.status === "clarification") { + return false; + } + const decisionSource = run.trace?.clarificationDecision?.decisionSource; + const bypassed = run.trace?.clarificationDecision?.bypassed; + const bypassReasonCode = run.trace?.clarificationDecision?.bypassReasonCode; + if ( + decisionSource === "metadata-intent" || + bypassed === true || + bypassReasonCode === "bypass_metadata_intent" + ) { + return true; + } + const generateSqlStep = findTraceStep(run, "generate-sql"); + const outputSummary = parseOutputSummary( + typeof generateSqlStep?.outputSummary === "string" + ? generateSqlStep.outputSummary + : undefined + ); + return outputSummary?.semanticIntent === "metadata"; +} + +function detectStrictSemanticPath(run: { trace?: { steps?: unknown[] } }): boolean { + const semanticStep = findTraceStep(run, "build-semantic-query"); + if (!semanticStep) { + return false; + } + return semanticStep.status !== "skipped"; +} + +function extractSemanticPlanStatus(run: { trace?: { steps?: unknown[] } }): string | undefined { + const semanticStep = findTraceStep(run, "build-semantic-query"); + const outputSummary = parseOutputSummary( + typeof semanticStep?.outputSummary === "string" + ? semanticStep.outputSummary + : undefined + ); + if (typeof outputSummary?.status === "string") { + return outputSummary.status; + } + return undefined; +} + +function evaluatePostClarifySemanticPass(run: { + status: RunStatus; + trace?: { steps?: unknown[] }; +}): boolean { + if (run.status === "clarification" || run.status === "failed") { + return false; + } + const semanticStep = findTraceStep(run, "build-semantic-query"); + if (!semanticStep) { + return false; + } + return semanticStep.status === "success"; +} + +async function readFixture(filePath: string): Promise { + const raw = await readFile(filePath, "utf8"); + return ensureFixture(JSON.parse(raw), filePath); +} + +function applyCollectorDefaults(): void { + process.env.LLM_MOCK_MODE = process.env.LLM_MOCK_MODE ?? "true"; + process.env.LLM_PROVIDER = process.env.LLM_PROVIDER ?? "volcengine"; + process.env.AGENT_PLANNING_SCAFFOLD_ENABLED = + process.env.AGENT_PLANNING_SCAFFOLD_ENABLED ?? "true"; + process.env.CLARIFICATION_HYBRID_ENABLED = + process.env.CLARIFICATION_HYBRID_ENABLED ?? "true"; + process.env.CLARIFICATION_HYBRID_KILL_SWITCH_RULES_ONLY = + process.env.CLARIFICATION_HYBRID_KILL_SWITCH_RULES_ONLY ?? "false"; + process.env.CLARIFICATION_HYBRID_TIMEOUT_MS = + process.env.CLARIFICATION_HYBRID_TIMEOUT_MS ?? "1200"; + process.env.LANGSMITH_TRACING = process.env.LANGSMITH_TRACING ?? "false"; +} + +export async function collectClarificationBalanceGate( + options: CollectOptions = {} +): Promise { + applyCollectorDefaults(); + const fixturePath = resolve(options.fixturePath ?? DEFAULT_FIXTURE_PATH); + const outputPath = resolve( + options.outputPath ?? + process.env.CLARIFICATION_BALANCE_GATE_REPORT_PATH ?? + DEFAULT_OUTPUT_PATH + ); + const strictMode = + options.strictMode ?? parseBoolean(process.env.CLARIFICATION_BALANCE_GATE_STRICT); + + const fixture = await readFixture(fixturePath); + const thresholds = readThresholds(); + const caseDiagnostics: CaseDiagnostics[] = []; + const falsePositiveCaseIds: string[] = []; + const falseNegativeCaseIds: string[] = []; + const metadataBypassMismatchCaseIds: string[] = []; + const strictSemanticPathMismatchCaseIds: string[] = []; + const clarificationRoundExceededCaseIds: string[] = []; + const caseErrors: Array<{ caseId: string; message: string }> = []; + + let clarificationTriggered = 0; + let falsePositiveCount = 0; + let falseNegativeCount = 0; + let postClarifySemanticTotal = 0; + let postClarifySemanticPassed = 0; + let clarificationRoundTotal = 0; + + const sqliteFixture = await createSeededSqliteFixture("clarification-balance-gate"); + process.env.SQLITE_PATH = sqliteFixture.dbPath; + let moduleRef: TestingModule | undefined; + + try { + const builtModule = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + moduleRef = builtModule; + const graph = builtModule.get(GraphBuilderService); + + for (const testCase of fixture.cases) { + const diagnostics: CaseDiagnostics = { + caseId: testCase.id, + scenario: testCase.scenario, + expectedShouldClarify: testCase.expected.shouldClarify, + triggeredClarification: false, + clarificationRounds: 0, + firstRunStatus: "failed", + metadataBypassDetected: false, + strictSemanticPathDetected: false, + postClarifySemanticPass: false, + errors: [] + }; + + try { + const firstRun = await graph.run({ + runId: randomUUID(), + sessionId: `clarification-balance:${testCase.id}`, + question: testCase.initialQuestion, + datasourceId: "sqlite_main", + datasourceType: "sqlite" + }); + const firstRunStatus = readRunStatus(firstRun.status); + diagnostics.firstRunStatus = firstRunStatus; + + const triggeredClarification = firstRunStatus === "clarification"; + diagnostics.triggeredClarification = triggeredClarification; + if (triggeredClarification) { + clarificationTriggered += 1; + diagnostics.clarificationRounds = 1; + } + + if (triggeredClarification && !testCase.expected.shouldClarify) { + falsePositiveCount += 1; + falsePositiveCaseIds.push(testCase.id); + } + if (!triggeredClarification && testCase.expected.shouldClarify) { + falseNegativeCount += 1; + falseNegativeCaseIds.push(testCase.id); + } + + diagnostics.metadataBypassDetected = detectMetadataBypass({ + status: firstRunStatus, + trace: firstRun.trace + }); + if (diagnostics.metadataBypassDetected !== testCase.expected.metadataBypass) { + metadataBypassMismatchCaseIds.push(testCase.id); + } + + diagnostics.strictSemanticPathDetected = detectStrictSemanticPath({ + trace: firstRun.trace + }); + if (triggeredClarification && testCase.followUpQuestion) { + const followUpRun = await graph.run({ + runId: randomUUID(), + sessionId: `clarification-balance:${testCase.id}`, + question: testCase.followUpQuestion, + datasourceId: "sqlite_main", + datasourceType: "sqlite" + }); + diagnostics.followUpRunStatus = readRunStatus(followUpRun.status); + diagnostics.strictSemanticPathDetected = + diagnostics.strictSemanticPathDetected || + detectStrictSemanticPath({ trace: followUpRun.trace }); + diagnostics.semanticPlanStatus = extractSemanticPlanStatus({ + trace: followUpRun.trace + }); + diagnostics.postClarifySemanticPass = evaluatePostClarifySemanticPass({ + status: diagnostics.followUpRunStatus, + trace: followUpRun.trace + }); + postClarifySemanticTotal += 1; + if (diagnostics.postClarifySemanticPass) { + postClarifySemanticPassed += 1; + } + } else if (triggeredClarification && !testCase.followUpQuestion) { + diagnostics.errors.push( + "followUpQuestion is required when expected.shouldClarify=true" + ); + } + + if ( + testCase.expected.strictSemanticPath && + !diagnostics.strictSemanticPathDetected + ) { + strictSemanticPathMismatchCaseIds.push(testCase.id); + } + + clarificationRoundTotal += diagnostics.clarificationRounds; + if (diagnostics.clarificationRounds > testCase.expected.maxClarificationRounds) { + clarificationRoundExceededCaseIds.push(testCase.id); + } + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + diagnostics.errors.push(message); + caseErrors.push({ caseId: testCase.id, message }); + } + + caseDiagnostics.push(diagnostics); + } + } finally { + await moduleRef?.close(); + await sqliteFixture.cleanup(); + } + + const sampleSize = fixture.cases.length; + const triggerRate = sampleSize === 0 ? 0 : clarificationTriggered / sampleSize; + const falsePositiveRate = sampleSize === 0 ? 0 : falsePositiveCount / sampleSize; + const falseNegativeRate = sampleSize === 0 ? 0 : falseNegativeCount / sampleSize; + const postClarifySemanticPassRate = + postClarifySemanticTotal === 0 ? 0 : postClarifySemanticPassed / postClarifySemanticTotal; + const averageClarificationRounds = + clarificationTriggered === 0 ? 0 : clarificationRoundTotal / clarificationTriggered; + const sampleReady = sampleSize >= thresholds.minSamples; + + const reasons: string[] = []; + if (!sampleReady) { + reasons.push("sample_not_ready"); + } + if (triggerRate < thresholds.minTriggerRate) { + reasons.push("trigger_rate_below_threshold"); + } + if (triggerRate > thresholds.maxTriggerRate) { + reasons.push("trigger_rate_above_threshold"); + } + if (falsePositiveRate > thresholds.maxFalsePositiveRate) { + reasons.push("false_positive_rate_exceeded"); + } + if (falseNegativeRate > thresholds.maxFalseNegativeRate) { + reasons.push("false_negative_rate_exceeded"); + } + if (postClarifySemanticPassRate < thresholds.minPostClarifySemanticPassRate) { + reasons.push("post_clarify_semantic_pass_rate_below_threshold"); + } + if (averageClarificationRounds > thresholds.maxAverageClarificationRounds) { + reasons.push("avg_clarification_rounds_exceeded"); + } + if (metadataBypassMismatchCaseIds.length > 0) { + reasons.push("metadata_bypass_expectation_mismatch"); + } + if (strictSemanticPathMismatchCaseIds.length > 0) { + reasons.push("strict_semantic_path_expectation_mismatch"); + } + if (clarificationRoundExceededCaseIds.length > 0) { + reasons.push("clarification_rounds_exceeded"); + } + if (caseErrors.length > 0) { + reasons.push("case_execution_error"); + } + + const report: ClarificationBalanceGateReport = { + generatedAt: new Date().toISOString(), + strictMode, + fixture: { + path: fixturePath, + version: fixture.version, + generatedAt: fixture.generatedAt, + sampleSize + }, + sampleReady, + gatePass: sampleReady && reasons.length === 0, + reasons, + thresholds, + metrics: { + sampleSize, + clarificationTriggered, + triggerRate: toRate(triggerRate), + falsePositiveRate: toRate(falsePositiveRate), + falseNegativeRate: toRate(falseNegativeRate), + postClarifySemanticPassRate: toRate(postClarifySemanticPassRate), + averageClarificationRounds: toRate(averageClarificationRounds) + }, + diagnostics: { + falsePositiveCaseIds, + falseNegativeCaseIds, + metadataBypassMismatchCaseIds, + strictSemanticPathMismatchCaseIds, + clarificationRoundExceededCaseIds, + caseErrors, + cases: caseDiagnostics + } + }; + + await mkdir(dirname(outputPath), { recursive: true }); + await writeFile(outputPath, JSON.stringify(report, null, 2), "utf8"); + + return report; +} + +async function main(): Promise { + const args = parseArgs(process.argv); + const report = await collectClarificationBalanceGate({ + fixturePath: args.fixturePath, + outputPath: args.outputPath, + strictMode: args.strictMode + }); + process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); + if (report.strictMode && !report.gatePass) { + process.stderr.write( + `clarification balance gate failed under strict mode: ${ + report.reasons.join(", ") || "unknown_reason" + }\n` + ); + process.exitCode = 2; + } +} + +if (require.main === module) { + main().catch((error) => { + process.stderr.write( + `${error instanceof Error ? error.stack ?? error.message : String(error)}\n` + ); + process.exit(1); + }); +} diff --git a/apps/backend/src/modules/config/app-config.service.ts b/apps/backend/src/modules/config/app-config.service.ts index 30f365b..907dda8 100644 --- a/apps/backend/src/modules/config/app-config.service.ts +++ b/apps/backend/src/modules/config/app-config.service.ts @@ -133,6 +133,34 @@ export class AppConfigService { return this.config.get("AGENT_PLANNING_SCAFFOLD_ENABLED", "false") === "true"; } + get clarificationHybridEnabled(): boolean { + return this.config.get("CLARIFICATION_HYBRID_ENABLED", "true") === "true"; + } + + get clarificationRulesOnlyKillSwitch(): boolean { + return ( + this.config.get( + "CLARIFICATION_HYBRID_KILL_SWITCH_RULES_ONLY", + this.config.get("CLARIFICATION_RULES_ONLY_KILL_SWITCH", "false") + ) === "true" + ); + } + + get clarificationSemanticTimeoutMs(): number { + const raw = this.config.get( + "CLARIFICATION_HYBRID_TIMEOUT_MS", + this.config.get("CLARIFICATION_SEMANTIC_TIMEOUT_MS", "1200") + ); + const parsed = Number(raw); + if (Number.isFinite(parsed) && parsed > 0) { + return Math.floor(parsed); + } + this.logger.warn( + `CLARIFICATION_HYBRID_TIMEOUT_MS 配置无效（${raw}），已回退默认值 1200。` + ); + return 1200; + } + get sqlSafetySoftWarnMaxLength(): number { return this.readNumber("SQL_SAFETY_SOFT_WARN_MAX_LENGTH", 600); } diff --git a/apps/backend/src/modules/conversation/agent/agent.module.ts b/apps/backend/src/modules/conversation/agent/agent.module.ts index c737222..7f01026 100644 --- a/apps/backend/src/modules/conversation/agent/agent.module.ts +++ b/apps/backend/src/modules/conversation/agent/agent.module.ts @@ -15,6 +15,8 @@ import { SemanticRegistryService } from "../../semantic-registry/semantic-regist import { SettingsModule } from "../../governance/settings/settings.module"; import { BuildIntentPlanNode } from "./nodes/build-intent-plan.node"; import { ClarifyNode } from "./nodes/clarify.node"; +import { ClarificationFusionPolicy } from "./nodes/clarification-fusion.policy"; +import { ClarificationSemanticEvaluatorService } from "./nodes/clarification-semantic-evaluator.service"; import { FormatAnswerNode } from "./nodes/format-answer.node"; import { RetrieveKnowledgeNode } from "./nodes/retrieve-knowledge.node"; import { BuildPhysicalPlanNode } from "./nodes/build-physical-plan.node"; @@ -60,6 +62,8 @@ import { PlannerCacheService } from "./planner/planner-cache.service"; GraphBuilderService, LangGraphRuntimeService, ClarifyNode, + ClarificationSemanticEvaluatorService, + ClarificationFusionPolicy, RetrieveKnowledgeNode, BuildIntentPlanNode, PlannerVersionLockService, diff --git a/apps/backend/src/modules/conversation/agent/graph/graph.builder.ts b/apps/backend/src/modules/conversation/agent/graph/graph.builder.ts index 78b30e1..66497e1 100644 --- a/apps/backend/src/modules/conversation/agent/graph/graph.builder.ts +++ b/apps/backend/src/modules/conversation/agent/graph/graph.builder.ts @@ -1,6 +1,7 @@ import { Injectable } from "@nestjs/common"; import type { ContextEnvelope, + ContextEnvelopePinningEvidence, ExecutionTraceStep, SqlRun } from "@text2sql/shared-types"; @@ -31,12 +32,22 @@ export interface GraphRunOptions { } interface GraphContextEvidence { - effectiveContextSummary?: GraphEffectiveContextSummary; + effectiveContextSummary?: GraphEffectiveContextSummaryWithPinning; conflictHint?: GraphContextConflictHint; } const CONTEXT_CONFLICT_REGEX = /conflict|mismatch|contradict|inconsistent|冲突/i; +interface GraphEffectiveContextSummaryWithPinning extends GraphEffectiveContextSummary { + userEnvelope: GraphEffectiveContextSummary["userEnvelope"] & { + pinnedTableCount?: number; + pinnedColumnCount?: number; + }; + retrievalContext?: GraphEffectiveContextSummary["retrievalContext"] & { + pinning?: ContextEnvelopePinningEvidence; + }; +} + @Injectable() export class GraphBuilderService { constructor( @@ -264,8 +275,10 @@ export class GraphBuilderService { const includeTableCount = this.countNonEmptyStrings(envelope?.mustIncludeTables); const excludeTableCount = this.countNonEmptyStrings(envelope?.mustExcludeTables); + const pinnedTableCount = this.countNonEmptyStrings(envelope?.pinnedTables); + const pinnedColumnCount = this.countNonEmptyStrings(envelope?.pinnedColumns); const entityMappingCount = this.countEntityMappings(envelope?.entityMappings); - const effectiveContextSummary: GraphEffectiveContextSummary = { + const effectiveContextSummary: GraphEffectiveContextSummaryWithPinning = { sourcePriority: "user_explicit_over_system", userEnvelope: { metricDefinitionProvided: this.hasNonEmptyString(envelope?.metricDefinition), @@ -273,11 +286,18 @@ export class GraphBuilderService { entityMappingCount, includeTableCount, excludeTableCount, + ...(pinnedTableCount > 0 ? { pinnedTableCount } : {}), + ...(pinnedColumnCount > 0 ? { pinnedColumnCount } : {}), businessConstraintCount: this.countNonEmptyStrings(envelope?.businessConstraints) }, retrievalContext: { status: state.retrievalBundle?.status, - selectedContextCount: state.retrievalBundle?.selected_context?.length + selectedContextCount: state.retrievalBundle?.selected_context?.length, + ...(state.retrievedKnowledge?.pinning + ? { + pinning: state.retrievedKnowledge.pinning + } + : {}) } }; @@ -339,6 +359,8 @@ export class GraphBuilderService { this.countEntityMappings(input.entityMappings) > 0 || this.countNonEmptyStrings(input.mustIncludeTables) > 0 || this.countNonEmptyStrings(input.mustExcludeTables) > 0 || + this.countNonEmptyStrings(input.pinnedTables) > 0 || + this.countNonEmptyStrings(input.pinnedColumns) > 0 || this.countNonEmptyStrings(input.businessConstraints) > 0 ); } diff --git a/apps/backend/src/modules/conversation/agent/graph/langgraph.node-handlers.ts b/apps/backend/src/modules/conversation/agent/graph/langgraph.node-handlers.ts index 91f9595..89113e5 100644 --- a/apps/backend/src/modules/conversation/agent/graph/langgraph.node-handlers.ts +++ b/apps/backend/src/modules/conversation/agent/graph/langgraph.node-handlers.ts @@ -1,5 +1,6 @@ import type { LangGraphRunnableConfig } from "@langchain/langgraph"; import type { ExecutionTraceStep } from "@text2sql/shared-types"; +import { DomainError } from "../../../../common/domain-error"; import { BuildIntentPlanNode } from "../nodes/build-intent-plan.node"; import { BuildPhysicalPlanNode } from "../nodes/build-physical-plan.node"; import { BuildSemanticQueryNode } from "../nodes/build-semantic-query.node"; @@ -18,6 +19,7 @@ import { appendStep, type LangGraphState } from "./langgraph.state"; +import type { SqlGenerationExplicitPinningEvidence } from "../sql/sql-generation.service"; interface RuntimeCallbacks { streamMode?: boolean; @@ -116,8 +118,82 @@ const appendPlanningWarning = ( return [...(state.planningWarnings ?? []), warning]; }; +const appendPlanningWarnings = ( + state: LangGraphState, + stage: "intent" | "semantic", + warnings: string[] +): string[] => { + const normalized = warnings + .map((warning) => warning.trim()) + .filter((warning) => warning.length > 0) + .map((warning) => `${stage}:${warning}`); + return [...(state.planningWarnings ?? []), ...normalized]; +}; + +const normalizeIdentifier = (value: string | undefined): string | undefined => { + if (!value) { + return undefined; + } + const normalized = value + .trim() + .replace(/^[`"'[\]]+|[`"'[\]]+$/g, "") + .replace(/\s+/g, ""); + if (!normalized) { + return undefined; + } + return normalized.toLowerCase(); +}; + +const extractQualifiedColumns = (value: string | undefined): string[] => { + if (!value) { + return []; + } + const result: string[] = []; + const regex = /\b([a-zA-Z_][\w$]*)\.([a-zA-Z_][\w$]*)\b/g; + for (const match of value.matchAll(regex)) { + const table = normalizeIdentifier(match[1]); + const column = normalizeIdentifier(match[2]); + if (!table || !column) { + continue; + } + result.push(`${table}.${column}`); + } + return result; +}; + +const unique = (values: string[]): string[] => Array.from(new Set(values)); + +const buildExplicitPinningEvidence = ( + state: LangGraphState +): SqlGenerationExplicitPinningEvidence | undefined => { + const envelopePinnedTables = state.contextEnvelope?.pinnedTables ?? []; + const envelopePinnedColumns = state.contextEnvelope?.pinnedColumns ?? []; + const tables = unique( + envelopePinnedTables + .map((item) => normalizeIdentifier(item)) + .filter((item): item is string => Boolean(item)) + ); + const mappedColumns = unique([ + ...envelopePinnedColumns + .map((item) => normalizeIdentifier(item)) + .filter((item): item is string => Boolean(item)), + ...((state.contextEnvelope?.entityMappings ?? []) + .map((item) => extractQualifiedColumns(item.mappedTo)) + .flat()), + ...extractQualifiedColumns(state.contextEnvelope?.metricDefinition) + ]); + if (tables.length === 0 && mappedColumns.length === 0) { + return undefined; + } + return { + source: "context_envelope", + tables, + columns: mappedColumns + }; +}; + export interface LangGraphNodeDependencies { - clarifyNode: Pick; + clarifyNode: Pick; retrieveKnowledgeNode: Pick; buildIntentPlanNode: Pick; buildSemanticQueryNode: Pick; @@ -138,14 +214,33 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => await runtime.emitStep( buildRunningStep(state, "clarify", startedAt, "正在理解问题") ); - const clarification = deps.clarifyNode.run( + const decision = deps.clarifyNode.evaluate( state.question, state.contextEnvelope ); + const clarificationDecision = { + decision: decision.decision, + triggerPath: decision.triggerPath, + decisionSource: decision.decisionSource, + bypassed: decision.bypassed, + ...(decision.bypassReasonCode ? { bypassReasonCode: decision.bypassReasonCode } : {}), + confidenceLevel: decision.confidenceLevel, + missingCriticalSlots: decision.missingCriticalSlots, + conflictDetected: false, + ...(decision.reasonCodes.length > 0 ? { reasonCodes: decision.reasonCodes } : {}), + question: decision.question, + reason: decision.reason + }; + const clarification = decision.shouldClarify + ? { + ...clarificationDecision + } + : undefined; const endedAt = new Date().toISOString(); if (clarification) { const outputs = { - clarificationQuestion: clarification.question + clarificationQuestion: clarification.question, + clarificationDecision }; const trace = appendStep(state, { step: { @@ -161,6 +256,10 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => await runtime.emitStep(latestStep(trace)); return { ...trace, + trace: { + ...trace.trace, + clarificationDecision + }, clarification, answer: clarification.question, terminalStatus: "clarification" @@ -170,12 +269,20 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => step: { node: "clarify", status: "skipped", - detail: "问题信息充足，跳过澄清。", - ...withTiming(startedAt, endedAt) - } + detail: decision.reason || "问题信息充足，跳过澄清。", + ...withTiming(startedAt, endedAt), + outputSummary: summarize({ clarificationDecision }) + }, + outputs: { clarificationDecision } }); await runtime.emitStep(latestStep(trace)); - return trace; + return { + ...trace, + trace: { + ...trace.trace, + clarificationDecision + } + }; }; const retrieveKnowledge = async ( @@ -210,6 +317,9 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => datasourceId: state.datasourceId, runId: state.runId, workspaceId: state.accessContext?.workspaceId, + allowedTables: state.accessContext?.allowedTables, + pinnedTables: state.contextEnvelope?.pinnedTables ?? state.contextEnvelope?.mustIncludeTables, + pinnedColumns: state.contextEnvelope?.pinnedColumns, modelCatalogId: state.modelCatalogId }); const endedAt = new Date().toISOString(); @@ -312,15 +422,19 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => } try { - const intentPlan = deps.buildIntentPlanNode.run( + const intentPlan = await deps.buildIntentPlanNode.run( state.question, - state.retrievedKnowledge + state.retrievedKnowledge, + state.trace.clarificationDecision ); const endedAt = new Date().toISOString(); const outputs = { status: intentPlan.status, intent: intentPlan.intent, - constraints: intentPlan.constraints + constraints: intentPlan.constraints, + uncertaintySignal: intentPlan.uncertaintySignal, + clarificationDecision: intentPlan.clarificationDecision, + riskTags: intentPlan.riskTags }; const trace = appendStep(state, { step: { @@ -335,12 +449,24 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => await runtime.emitStep(latestStep(trace)); return { ...trace, + trace: { + ...trace.trace, + clarificationDecision: + intentPlan.clarificationDecision ?? state.trace.clarificationDecision + }, intentPlan, planningStatus: intentPlan.status === "ready" ? state.planningStatus : "degraded", planningWarnings: - intentPlan.status === "ready" + intentPlan.status === "ready" && + (intentPlan.planningWarnings?.length ?? 0) === 0 ? state.planningWarnings - : appendPlanningWarning(state, intentPlan.summary) + : appendPlanningWarnings( + state, + "intent", + (intentPlan.planningWarnings?.length ?? 0) > 0 + ? (intentPlan.planningWarnings ?? []) + : [intentPlan.summary] + ) }; } catch (error) { const message = toErrorMessage(error); @@ -417,7 +543,8 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => const semanticQueryPlan = await deps.buildSemanticQueryNode.run({ intentPlan: state.intentPlan, question: state.question, - retrievalBundle: state.retrievalBundle + retrievalBundle: state.retrievalBundle, + clarificationDecision: state.trace.clarificationDecision }); const endedAt = new Date().toISOString(); const outputs = { @@ -430,7 +557,11 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => lockStatus: semanticQueryPlan.lockStatus, fallbackApplied: semanticQueryPlan.fallbackApplied, degradeReason: semanticQueryPlan.degradeReason, - riskTags: semanticQueryPlan.riskTags + riskTags: semanticQueryPlan.riskTags, + strictMode: semanticQueryPlan.strictMode, + strictModeReasons: semanticQueryPlan.strictModeReasons, + planningWarnings: semanticQueryPlan.planningWarnings, + clarificationDecision: semanticQueryPlan.clarificationDecision }; const trace = appendStep(state, { step: { @@ -445,13 +576,25 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => await runtime.emitStep(latestStep(trace)); return { ...trace, + trace: { + ...trace.trace, + clarificationDecision: + semanticQueryPlan.clarificationDecision ?? state.trace.clarificationDecision + }, semanticQueryPlan, planningStatus: semanticQueryPlan.status === "ready" ? state.planningStatus : "degraded", planningWarnings: - semanticQueryPlan.status === "ready" + semanticQueryPlan.status === "ready" && + (semanticQueryPlan.planningWarnings?.semantic.length ?? 0) === 0 ? state.planningWarnings - : appendPlanningWarning(state, semanticQueryPlan.summary) + : appendPlanningWarnings( + state, + "semantic", + (semanticQueryPlan.planningWarnings?.semantic.length ?? 0) > 0 + ? (semanticQueryPlan.planningWarnings?.semantic ?? []) + : [semanticQueryPlan.summary] + ) }; } catch (error) { const message = toErrorMessage(error); @@ -594,6 +737,7 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => ); const callbacks = runtime.callbacks; try { + const explicitPinning = buildExplicitPinningEvidence(state); const generated = await deps.generateSqlNode.run( state.question, state.datasourceType, @@ -606,13 +750,15 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => selectedContext: state.retrievalBundle?.selected_context, semanticContextPack: state.contextPack ?? state.retrievalBundle?.context_pack, datasourceId: state.datasourceId, - workspaceId: state.accessContext?.workspaceId + workspaceId: state.accessContext?.workspaceId, + explicitPinning } : { selectedContext: state.retrievalBundle?.selected_context, semanticContextPack: state.contextPack ?? state.retrievalBundle?.context_pack, datasourceId: state.datasourceId, - workspaceId: state.accessContext?.workspaceId + workspaceId: state.accessContext?.workspaceId, + explicitPinning } ); const endedAt = new Date().toISOString(); @@ -632,6 +778,9 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => retrieveSummary: state.retrievedKnowledge?.summary, intent: state.intentPlan?.intent, semanticHints: state.semanticQueryPlan?.semanticHints, + semanticStrictMode: state.semanticQueryPlan?.strictMode, + semanticStrictModeReasons: state.semanticQueryPlan?.strictModeReasons, + clarificationDecision: state.trace.clarificationDecision, physicalStrategy: state.physicalPlan?.strategy, semanticConstraintMode: state.physicalPlan?.semanticConstraintMode, contextPackStatus: state.contextPack?.status ?? state.retrievalBundle?.context_pack?.status @@ -640,11 +789,10 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => provider: generated.provider, model: generated.model, modelCatalogId: generated.modelCatalogId, - sql: generated.sql, - rawText: generated.rawText, promptTemplate: generated.promptTemplate, retryCount: generated.retryCount ?? 0, - semanticIntent: generated.semanticIntent + semanticIntent: generated.semanticIntent, + coverage: generated.coverage }; const trace = appendStep(state, { step: { @@ -696,6 +844,13 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => const inputs = { question: state.question }; + const outputs = + error instanceof DomainError + ? { + errorCode: error.code, + errorDetails: error.details + } + : undefined; const trace = appendStep(state, { step: { node: "generate-sql", @@ -703,10 +858,12 @@ export const createLangGraphNodeHandlers = (deps: LangGraphNodeDependencies) => detail: message, ...withTiming(startedAt, endedAt), inputSummary: summarize(inputs), + outputSummary: summarize(outputs), errorSummary: summarize(message) }, runType: "llm", inputs, + outputs, error: message }); await runtime.emitStep(latestStep(trace)); diff --git a/apps/backend/src/modules/conversation/agent/graph/langgraph.state.ts b/apps/backend/src/modules/conversation/agent/graph/langgraph.state.ts index 4f53fc7..f04e254 100644 --- a/apps/backend/src/modules/conversation/agent/graph/langgraph.state.ts +++ b/apps/backend/src/modules/conversation/agent/graph/langgraph.state.ts @@ -130,7 +130,8 @@ export const createInitialLangGraphState = ( runId: input.runId, provider: fallbackProvider, retryCount: 0, - steps: [] + steps: [], + clarificationDecision: undefined }, spanEvents: [], relationCorrectionRetryCount: 0 diff --git a/apps/backend/src/modules/conversation/agent/nodes/build-intent-plan.node.ts b/apps/backend/src/modules/conversation/agent/nodes/build-intent-plan.node.ts index 570253b..19c6ca6 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/build-intent-plan.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/build-intent-plan.node.ts @@ -1,55 +1,255 @@ import { Injectable } from "@nestjs/common"; +import type { ClarificationDecisionEvidence } from "@text2sql/shared-types"; +import { + ClarificationFusionPolicy, + mapRuleDecisionToEvidence +} from "./clarification-fusion.policy"; +import { ClarificationSemanticEvaluatorService } from "./clarification-semantic-evaluator.service"; import type { RetrievedKnowledge } from "./retrieve-knowledge.node"; +import { decideSlotFilling } from "./slot-filling-context"; + +export interface IntentUncertaintySignal { + level: "low" | "medium" | "high"; + needsStrictSemanticPath: boolean; + reasonCodes: string[]; +} export interface IntentPlan { status: "ready" | "degraded"; intent: "aggregate" | "detail" | "compare" | "unknown"; constraints: string[]; summary: string; + uncertaintySignal?: IntentUncertaintySignal; + clarificationDecision?: ClarificationDecisionEvidence; + riskTags?: string[]; + planningWarnings?: string[]; } @Injectable() export class BuildIntentPlanNode { - run(question: string, knowledge: RetrievedKnowledge): IntentPlan { - const normalized = question.toLowerCase(); + constructor( + private readonly semanticEvaluator: ClarificationSemanticEvaluatorService, + private readonly clarificationFusionPolicy: ClarificationFusionPolicy + ) {} + + async run( + question: string, + knowledge: RetrievedKnowledge, + clarificationDecision?: ClarificationDecisionEvidence + ): Promise { + const normalizedQuestion = question.toLowerCase(); const selectedContextCount = knowledge.retrievalBundle?.selected_context?.length ?? 0; + const pinnedConstraint = + knowledge.pinning?.enabled && knowledge.pinning.status === "applied" + ? ["require_pinned_table_alignment"] + : []; + + const ruleDecision = this.resolveRuleDecision(question, clarificationDecision); + const semanticEvaluation = this.clarificationFusionPolicy.shouldArbitrate(ruleDecision) + ? await this.semanticEvaluator.evaluate({ + question, + ruleDecision + }) + : undefined; + const fusion = this.clarificationFusionPolicy.fuse({ + ruleDecision, + semanticEvaluation + }); + + const bypassedBusinessSemantics = this.isBypassDecision(fusion.decision); + const uncertaintySignal = this.buildUncertaintySignal({ + fusionDecision: fusion.decision, + strictModeReasons: fusion.metadata.strictModeReasons, + knowledgeDegraded: knowledge.status === "degraded", + bypassedBusinessSemantics + }); + + const planningWarnings = this.buildPlanningWarnings({ + fusionDecision: fusion.decision, + fallbackApplied: fusion.metadata.fallbackApplied, + fallbackReason: fusion.metadata.fallbackReason, + strictModeReasons: fusion.metadata.strictModeReasons, + knowledgeSummary: knowledge.summary, + knowledgeStatus: knowledge.status + }); + if (knowledge.status === "degraded") { return { status: "degraded", intent: "unknown", - constraints: selectedContextCount > 0 ? ["fallback_with_partial_context"] : [], - summary: "检索上下文不可用，意图规划降级。" + constraints: [ + ...(selectedContextCount > 0 ? ["fallback_with_partial_context"] : []), + ...pinnedConstraint, + ...(bypassedBusinessSemantics ? ["skip_business_semantic_assertions"] : []) + ], + summary: "检索上下文不可用，意图规划降级。", + uncertaintySignal, + clarificationDecision: fusion.decision, + riskTags: this.unique([ + "intent:knowledge_degraded", + ...fusion.metadata.stageRiskTags.rule.map((tag) => `intent:${tag}`), + ...fusion.metadata.stageRiskTags.semantic.map((tag) => `intent:${tag}`), + ...fusion.metadata.stageRiskTags.fusion.map((tag) => `intent:${tag}`) + ]), + planningWarnings }; } - if (/统计|总数|分布|汇总/.test(normalized)) { + const baseConstraints: string[] = [ + ...(selectedContextCount > 0 ? ["must_use_selected_context"] : []), + ...pinnedConstraint, + ...(bypassedBusinessSemantics ? ["skip_business_semantic_assertions"] : []) + ]; + + if (/统计|总数|分布|汇总/.test(normalizedQuestion)) { return { status: "ready", intent: "aggregate", - constraints: [ - "prefer_group_by", - ...(selectedContextCount > 0 ? ["must_use_selected_context"] : []) - ], - summary: "意图识别为聚合统计。" + constraints: ["prefer_group_by", ...baseConstraints], + summary: "意图识别为聚合统计。", + uncertaintySignal, + clarificationDecision: fusion.decision, + riskTags: this.resolveRiskTags(fusion), + planningWarnings }; } - if (/对比|比较|同比|环比/.test(normalized)) { + if (/对比|比较|同比|环比/.test(normalizedQuestion)) { return { status: "ready", intent: "compare", - constraints: [ - "require_two_dimensions", - ...(selectedContextCount > 0 ? ["must_use_selected_context"] : []) - ], - summary: "意图识别为对比分析。" + constraints: ["require_two_dimensions", ...baseConstraints], + summary: "意图识别为对比分析。", + uncertaintySignal, + clarificationDecision: fusion.decision, + riskTags: this.resolveRiskTags(fusion), + planningWarnings }; } return { status: "ready", intent: "detail", - constraints: selectedContextCount > 0 ? ["must_use_selected_context"] : [], - summary: "意图识别为明细查询。" + constraints: baseConstraints, + summary: "意图识别为明细查询。", + uncertaintySignal, + clarificationDecision: fusion.decision, + riskTags: this.resolveRiskTags(fusion), + planningWarnings }; } + + private resolveRuleDecision( + question: string, + clarificationDecision?: ClarificationDecisionEvidence + ): ClarificationDecisionEvidence { + if (clarificationDecision?.decision) { + return { + ...clarificationDecision, + triggerPath: clarificationDecision.triggerPath ?? "rule", + confidenceLevel: clarificationDecision.confidenceLevel ?? "medium", + missingCriticalSlots: clarificationDecision.missingCriticalSlots ?? [], + conflictDetected: Boolean(clarificationDecision.conflictDetected), + reasonCodes: this.unique(clarificationDecision.reasonCodes ?? []) + }; + } + return mapRuleDecisionToEvidence(decideSlotFilling(question)); + } + + private buildUncertaintySignal(input: { + fusionDecision: ClarificationDecisionEvidence; + strictModeReasons: string[]; + knowledgeDegraded: boolean; + bypassedBusinessSemantics: boolean; + }): IntentUncertaintySignal { + if (input.bypassedBusinessSemantics) { + return { + level: "low", + needsStrictSemanticPath: false, + reasonCodes: this.unique([ + ...(input.fusionDecision.reasonCodes ?? []), + "intent_bypass_business_semantics" + ]) + }; + } + + const confidence = input.fusionDecision.confidenceLevel ?? "medium"; + const level: "low" | "medium" | "high" = + input.knowledgeDegraded || input.fusionDecision.decision === "clarify" + ? "high" + : confidence === "low" || input.fusionDecision.conflictDetected + ? "medium" + : "low"; + + return { + level, + needsStrictSemanticPath: + level !== "low" || input.strictModeReasons.length > 0 || input.knowledgeDegraded, + reasonCodes: this.unique([ + ...(input.fusionDecision.reasonCodes ?? []), + ...input.strictModeReasons, + input.knowledgeDegraded ? "intent_knowledge_degraded" : "intent_knowledge_ready" + ]) + }; + } + + private resolveRiskTags(fusion: { + metadata: { + stageRiskTags: { + rule: string[]; + semantic: string[]; + fusion: string[]; + }; + }; + }): string[] { + return this.unique([ + ...fusion.metadata.stageRiskTags.rule.map((tag) => `intent:${tag}`), + ...fusion.metadata.stageRiskTags.semantic.map((tag) => `intent:${tag}`), + ...fusion.metadata.stageRiskTags.fusion.map((tag) => `intent:${tag}`) + ]); + } + + private buildPlanningWarnings(input: { + fusionDecision: ClarificationDecisionEvidence; + fallbackApplied: boolean; + fallbackReason?: "disabled" | "kill-switch" | "timeout" | "error" | "invalid_payload"; + strictModeReasons: string[]; + knowledgeSummary: string; + knowledgeStatus: RetrievedKnowledge["status"]; + }): string[] { + const warnings: string[] = []; + if (input.fallbackApplied) { + warnings.push( + `clarification fusion fallback applied (${input.fallbackReason ?? "unknown"})` + ); + } + if (input.strictModeReasons.length > 0) { + warnings.push(`strict semantic path required (${input.strictModeReasons.join(", ")})`); + } + if (input.fusionDecision.conflictDetected) { + warnings.push("rule and semantic decisions conflicted during arbitration"); + } + if (input.knowledgeStatus === "degraded") { + warnings.push(input.knowledgeSummary); + } + return this.unique(warnings); + } + + private isBypassDecision(decision: ClarificationDecisionEvidence): boolean { + if (decision.bypassed || decision.decisionSource === "metadata-intent" || decision.decisionSource === "sql-write-intent") { + return true; + } + const reasonCodes = decision.reasonCodes ?? []; + return reasonCodes.some((reasonCode) => { + return ( + reasonCode === "bypass_metadata_intent" || + reasonCode === "bypass_sql_write_intent" || + reasonCode === "rule_source_metadata-intent" || + reasonCode === "rule_source_sql-write-intent" + ); + }); + } + + private unique(values: string[]): string[] { + return [...new Set(values.map((value) => value.trim()).filter(Boolean))]; + } } diff --git a/apps/backend/src/modules/conversation/agent/nodes/build-semantic-query.node.ts b/apps/backend/src/modules/conversation/agent/nodes/build-semantic-query.node.ts index ea414a8..5766c62 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/build-semantic-query.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/build-semantic-query.node.ts @@ -1,4 +1,5 @@ import { Injectable } from "@nestjs/common"; +import type { ClarificationDecisionEvidence } from "@text2sql/shared-types"; import type { IntentPlan } from "./build-intent-plan.node"; import type { RetrievedKnowledge } from "./retrieve-knowledge.node"; import { PlannerVersionLockService } from "../planner/planner-version-lock.service"; @@ -22,6 +23,15 @@ export interface SemanticQueryPlan { fallbackApplied: boolean; degradeReason?: string; riskTags: string[]; + intentRiskTags?: string[]; + semanticRiskTags?: string[]; + planningWarnings?: { + intent: string[]; + semantic: string[]; + }; + strictMode?: boolean; + strictModeReasons?: string[]; + clarificationDecision?: ClarificationDecisionEvidence; summary: string; } @@ -34,15 +44,40 @@ export class BuildSemanticQueryNode { question: string; retrievalBundle?: RagRetrievalBundle; requestedSemanticVersion?: number; + clarificationDecision?: ClarificationDecisionEvidence; }): Promise { const { intentPlan } = input; + const clarificationDecision = + input.clarificationDecision ?? intentPlan.clarificationDecision; + const intentRiskTags = this.unique(intentPlan.riskTags ?? []); + const planningWarnings = { + intent: this.unique(intentPlan.planningWarnings ?? []), + semantic: [] as string[] + }; + + const bypassedBusinessSemantics = intentPlan.constraints.includes( + "skip_business_semantic_assertions" + ); + const strictMode = + !bypassedBusinessSemantics && + Boolean(intentPlan.uncertaintySignal?.needsStrictSemanticPath); + const strictModeReasons = strictMode + ? this.unique(intentPlan.uncertaintySignal?.reasonCodes ?? []) + : []; + if (intentPlan.status === "degraded") { return { status: "degraded", semanticHints: [], lockStatus: "degraded", fallbackApplied: false, - riskTags: [], + riskTags: intentRiskTags, + intentRiskTags, + semanticRiskTags: [], + planningWarnings, + strictMode, + strictModeReasons, + clarificationDecision, summary: "意图规划不可用，语义检索降级。" }; } @@ -66,7 +101,14 @@ export class BuildSemanticQueryNode { modelingRevision, contextPackStatus }); - const riskTags = this.unique([...versionLock.riskTags, ...contextualRiskTags]); + + const semanticRiskTags = this.unique([ + ...versionLock.riskTags, + ...contextualRiskTags, + ...(strictMode ? ["semantic:strict_mode_enabled"] : []), + ...(bypassedBusinessSemantics ? ["semantic:bypass_business_semantics"] : []) + ]); + const riskTags = this.unique([...intentRiskTags, ...semanticRiskTags]); const semanticHints = intentPlan.intent === "aggregate" @@ -95,6 +137,20 @@ export class BuildSemanticQueryNode { ); if (contextPackStatus === "degraded") { semanticHints.push("semantic_context_pack_degraded"); + planningWarnings.semantic.push("context pack degraded, semantic constraints may be partial"); + } + if (strictMode) { + semanticHints.push("enforce_strict_clarification_guards"); + semanticHints.push("require_explicit_slot_grounding"); + planningWarnings.semantic.push( + `strict semantic path enabled (${strictModeReasons.join(", ") || "unknown reason"})` + ); + } + if (bypassedBusinessSemantics) { + semanticHints.push("preserve_clarification_bypass_reason"); + planningWarnings.semantic.push( + "bypass intent detected; business semantic assertions are skipped" + ); } const semanticBindingSummary = contextPack @@ -116,6 +172,7 @@ export class BuildSemanticQueryNode { relationshipBindingCount, contextPackStatus )}`; + planningWarnings.semantic.push(summary); return { status: "degraded", semanticHints, @@ -126,6 +183,15 @@ export class BuildSemanticQueryNode { fallbackApplied: false, degradeReason: versionLock.degradeReason, riskTags, + intentRiskTags, + semanticRiskTags, + planningWarnings: { + intent: this.unique(planningWarnings.intent), + semantic: this.unique(planningWarnings.semantic) + }, + strictMode, + strictModeReasons, + clarificationDecision, summary }; } @@ -136,6 +202,7 @@ export class BuildSemanticQueryNode { relationshipBindingCount, contextPackStatus )}`; + planningWarnings.semantic.push(summary); return { status: "degraded", semanticHints, @@ -146,6 +213,15 @@ export class BuildSemanticQueryNode { fallbackApplied: true, degradeReason: versionLock.degradeReason, riskTags, + intentRiskTags, + semanticRiskTags, + planningWarnings: { + intent: this.unique(planningWarnings.intent), + semantic: this.unique(planningWarnings.semantic) + }, + strictMode, + strictModeReasons, + clarificationDecision, summary }; } @@ -164,6 +240,15 @@ export class BuildSemanticQueryNode { lockStatus: "locked", fallbackApplied: false, riskTags, + intentRiskTags, + semanticRiskTags, + planningWarnings: { + intent: this.unique(planningWarnings.intent), + semantic: this.unique(planningWarnings.semantic) + }, + strictMode, + strictModeReasons, + clarificationDecision, summary }; } diff --git a/apps/backend/src/modules/conversation/agent/nodes/clarification-fusion.policy.ts b/apps/backend/src/modules/conversation/agent/nodes/clarification-fusion.policy.ts new file mode 100644 index 0000000..499839c --- /dev/null +++ b/apps/backend/src/modules/conversation/agent/nodes/clarification-fusion.policy.ts @@ -0,0 +1,354 @@ +import { Injectable } from "@nestjs/common"; +import type { + ClarificationDecision, + ClarificationDecisionEvidence, + ClarificationSlotKey +} from "@text2sql/shared-types"; +import { AppConfigService } from "../../../config/app-config.service"; +import type { + ClarificationSemanticEvaluationResult +} from "./clarification-semantic-evaluator.service"; +import type { SlotFillingDecision } from "./slot-filling-context"; + +export interface ClarificationFusionInput { + ruleDecision: ClarificationDecisionEvidence; + semanticEvaluation?: ClarificationSemanticEvaluationResult; +} + +export interface ClarificationFusionResult { + decision: ClarificationDecisionEvidence; + metadata: { + mode: "rules-only" | "hybrid"; + semanticRequested: boolean; + semanticUsed: boolean; + fallbackApplied: boolean; + fallbackReason?: "disabled" | "kill-switch" | "timeout" | "error" | "invalid_payload"; + strictModeReasons: string[]; + stageRiskTags: { + rule: string[]; + semantic: string[]; + fusion: string[]; + }; + }; +} + +export const mapRuleDecisionToEvidence = ( + ruleDecision: SlotFillingDecision +): ClarificationDecisionEvidence => { + const decision: ClarificationDecision = + ruleDecision.decision === "clarify" ? "clarify" : "continue"; + const missingCriticalSlots: ClarificationSlotKey[] = + ruleDecision.missingCriticalSlots ?? ruleDecision.missingSlots ?? []; + + return { + decision, + triggerPath: "rule", + decisionSource: ruleDecision.decisionSource, + bypassed: ruleDecision.bypassed, + bypassReasonCode: ruleDecision.bypassReasonCode, + confidenceLevel: ruleDecision.confidence, + missingCriticalSlots, + conflictDetected: false, + reasonCodes: buildRuleReasonCodes(ruleDecision), + question: ruleDecision.question, + reason: ruleDecision.reason + }; +}; + +const buildRuleReasonCodes = (ruleDecision: SlotFillingDecision): string[] => { + const reasonCodes: string[] = [ + `rule_source_${ruleDecision.source}`, + `rule_confidence_${ruleDecision.confidence}`, + ...(ruleDecision.reasonCodes ?? []) + ]; + for (const slot of ruleDecision.missingCriticalSlots ?? []) { + reasonCodes.push(`rule_missing_${slot}`); + } + return unique(reasonCodes); +}; + +@Injectable() +export class ClarificationFusionPolicy { + constructor(private readonly appConfig: AppConfigService) {} + + shouldArbitrate(ruleDecision: ClarificationDecisionEvidence): boolean { + if (this.appConfig.clarificationRulesOnlyKillSwitch) { + return false; + } + if (!this.appConfig.clarificationHybridEnabled) { + return false; + } + if (ruleDecision.bypassed || ruleDecision.decisionSource === "metadata-intent" || ruleDecision.decisionSource === "sql-write-intent") { + return false; + } + return ( + ruleDecision.confidenceLevel === "low" || + Boolean(ruleDecision.conflictDetected) || + Boolean(ruleDecision.reasonCodes?.includes("rule_confidence_low")) + ); + } + + fuse(input: ClarificationFusionInput): ClarificationFusionResult { + const ruleDecision = this.normalizeRuleDecision(input.ruleDecision); + const semanticEvaluation = input.semanticEvaluation; + + const hybridDisabled = !this.appConfig.clarificationHybridEnabled; + const rulesOnlyEnabled = this.appConfig.clarificationRulesOnlyKillSwitch; + + if (hybridDisabled || rulesOnlyEnabled) { + return { + decision: { + ...ruleDecision, + triggerPath: "rule", + reasonCodes: unique([ + ...(ruleDecision.reasonCodes ?? []), + hybridDisabled ? "hybrid_disabled" : "rules_only_kill_switch" + ]) + }, + metadata: { + mode: "rules-only", + semanticRequested: false, + semanticUsed: false, + fallbackApplied: true, + fallbackReason: hybridDisabled ? "disabled" : "kill-switch", + strictModeReasons: this.buildStrictModeReasons(ruleDecision), + stageRiskTags: { + rule: this.toStageRiskTags("rule", ruleDecision.reasonCodes), + semantic: [], + fusion: [hybridDisabled ? "fusion:hybrid_disabled" : "fusion:rules_only_kill_switch"] + } + } + }; + } + + if (!semanticEvaluation) { + return { + decision: ruleDecision, + metadata: { + mode: "hybrid", + semanticRequested: false, + semanticUsed: false, + fallbackApplied: false, + strictModeReasons: this.buildStrictModeReasons(ruleDecision), + stageRiskTags: { + rule: this.toStageRiskTags("rule", ruleDecision.reasonCodes), + semantic: [], + fusion: [] + } + } + }; + } + + if (semanticEvaluation.status !== "success" || !semanticEvaluation.decision) { + const fallbackReason = + semanticEvaluation.metadata.fallbackReason ?? this.mapFallbackReason(semanticEvaluation); + return { + decision: { + ...ruleDecision, + triggerPath: "hybrid", + reasonCodes: unique([ + ...(ruleDecision.reasonCodes ?? []), + `semantic_fallback_${fallbackReason}` + ]) + }, + metadata: { + mode: "hybrid", + semanticRequested: true, + semanticUsed: false, + fallbackApplied: true, + fallbackReason, + strictModeReasons: this.buildStrictModeReasons(ruleDecision), + stageRiskTags: { + rule: this.toStageRiskTags("rule", ruleDecision.reasonCodes), + semantic: this.toStageRiskTags("semantic", semanticEvaluation.reasonCodes), + fusion: [`fusion:semantic_fallback_${fallbackReason}`] + } + } + }; + } + + const semanticDecision = this.normalizeSemanticDecision(semanticEvaluation.decision); + const conflictDetected = ruleDecision.decision !== semanticDecision.decision; + const finalDecision = this.resolveConflict(ruleDecision, semanticDecision, conflictDetected); + + const mergedReasonCodes = unique([ + ...(finalDecision.reasonCodes ?? []), + ...(semanticDecision.reasonCodes ?? []), + conflictDetected + ? `fusion_conflict_resolved_${finalDecision.decision ?? "continue"}` + : "fusion_semantic_confirmed" + ]); + + const decision: ClarificationDecisionEvidence = { + ...finalDecision, + triggerPath: "hybrid", + conflictDetected, + reasonCodes: mergedReasonCodes + }; + + return { + decision, + metadata: { + mode: "hybrid", + semanticRequested: true, + semanticUsed: true, + fallbackApplied: false, + strictModeReasons: this.buildStrictModeReasons(decision), + stageRiskTags: { + rule: this.toStageRiskTags("rule", ruleDecision.reasonCodes), + semantic: this.toStageRiskTags("semantic", semanticDecision.reasonCodes), + fusion: conflictDetected + ? ["fusion:rule_semantic_conflict"] + : ["fusion:rule_semantic_agree"] + } + } + }; + } + + private normalizeRuleDecision( + decision: ClarificationDecisionEvidence + ): ClarificationDecisionEvidence { + return { + decision: decision.decision === "clarify" ? "clarify" : "continue", + triggerPath: "rule", + decisionSource: decision.decisionSource, + bypassed: decision.bypassed, + bypassReasonCode: decision.bypassReasonCode, + confidenceLevel: this.normalizeConfidenceLevel(decision.confidenceLevel), + missingCriticalSlots: this.normalizeSlots(decision.missingCriticalSlots), + conflictDetected: Boolean(decision.conflictDetected), + reasonCodes: unique(decision.reasonCodes ?? []), + question: decision.question, + reason: decision.reason + }; + } + + private normalizeSemanticDecision( + decision: ClarificationDecisionEvidence + ): ClarificationDecisionEvidence { + return { + decision: decision.decision === "clarify" ? "clarify" : "continue", + triggerPath: "semantic", + decisionSource: decision.decisionSource, + bypassed: decision.bypassed, + bypassReasonCode: decision.bypassReasonCode, + confidenceLevel: this.normalizeConfidenceLevel(decision.confidenceLevel), + missingCriticalSlots: this.normalizeSlots(decision.missingCriticalSlots), + conflictDetected: Boolean(decision.conflictDetected), + reasonCodes: unique(decision.reasonCodes ?? []), + question: decision.question, + reason: decision.reason + }; + } + + private resolveConflict( + ruleDecision: ClarificationDecisionEvidence, + semanticDecision: ClarificationDecisionEvidence, + conflictDetected: boolean + ): ClarificationDecisionEvidence { + if (!conflictDetected) { + if (ruleDecision.confidenceLevel === "low" && semanticDecision.confidenceLevel !== "low") { + return semanticDecision; + } + return ruleDecision; + } + + const ruleScore = this.scoreConfidence(ruleDecision.confidenceLevel); + const semanticScore = this.scoreConfidence(semanticDecision.confidenceLevel); + + if (semanticScore > ruleScore) { + return semanticDecision; + } + if (ruleScore > semanticScore) { + return ruleDecision; + } + + if (ruleDecision.decision === "clarify" || semanticDecision.decision === "clarify") { + return { + ...semanticDecision, + decision: "clarify", + confidenceLevel: "medium", + question: + semanticDecision.question?.trim() || + ruleDecision.question?.trim() || + "请补充关键分析信息后继续。", + reason: + semanticDecision.reason?.trim() || + ruleDecision.reason?.trim() || + "规则与语义信号冲突，保守进入澄清流程。" + }; + } + + return ruleDecision; + } + + private normalizeConfidenceLevel( + confidenceLevel: ClarificationDecisionEvidence["confidenceLevel"] + ): "high" | "medium" | "low" { + if (confidenceLevel === "high" || confidenceLevel === "medium" || confidenceLevel === "low") { + return confidenceLevel; + } + return "medium"; + } + + private normalizeSlots( + slots: ClarificationDecisionEvidence["missingCriticalSlots"] + ): ClarificationSlotKey[] { + if (!Array.isArray(slots)) { + return []; + } + return slots.filter((slot): slot is ClarificationSlotKey => { + return typeof slot === "string" && slot.trim().length > 0; + }); + } + + private scoreConfidence(confidence: ClarificationDecisionEvidence["confidenceLevel"]): number { + if (confidence === "high") { + return 3; + } + if (confidence === "medium") { + return 2; + } + if (confidence === "low") { + return 1; + } + return 2; + } + + private mapFallbackReason( + semanticEvaluation: ClarificationSemanticEvaluationResult + ): "timeout" | "error" | "invalid_payload" { + if (semanticEvaluation.status === "timeout") { + return "timeout"; + } + if (semanticEvaluation.status === "invalid") { + return "invalid_payload"; + } + return "error"; + } + + private buildStrictModeReasons(decision: ClarificationDecisionEvidence): string[] { + const reasons: string[] = []; + if (decision.decision === "clarify") { + reasons.push("clarification_decision_clarify"); + } + if (decision.confidenceLevel === "low") { + reasons.push("clarification_low_confidence"); + } + if (decision.conflictDetected) { + reasons.push("clarification_rule_semantic_conflict"); + } + return unique(reasons); + } + + private toStageRiskTags( + stage: "rule" | "semantic", + reasonCodes: string[] | undefined + ): string[] { + return (reasonCodes ?? []).map((reasonCode) => `${stage}:${reasonCode}`); + } +} + +const unique = (values: string[]): string[] => { + return [...new Set(values.map((value) => value.trim()).filter(Boolean))]; +}; diff --git a/apps/backend/src/modules/conversation/agent/nodes/clarification-semantic-evaluator.service.ts b/apps/backend/src/modules/conversation/agent/nodes/clarification-semantic-evaluator.service.ts new file mode 100644 index 0000000..6458812 --- /dev/null +++ b/apps/backend/src/modules/conversation/agent/nodes/clarification-semantic-evaluator.service.ts @@ -0,0 +1,316 @@ +import { Injectable } from "@nestjs/common"; +import type { + ClarificationConfidenceLevel, + ClarificationDecision, + ClarificationDecisionEvidence, + ClarificationSlotKey +} from "@text2sql/shared-types"; +import { AppConfigService } from "../../../config/app-config.service"; + +const METRIC_HINT_REGEX = + /(总数|数量|计数|count|金额|交易额|销售额|gmv|平均|均值|占比|比例|转化率|留存|退款率|分布|top|排行|环比|同比)/i; +const SUBJECT_HINT_REGEX = + /(订单|用户|客户|商品|sku|门店|支付|退款|交易|会话|工单|发票|供应商|渠道|地区|市场|销售|商家|账单)/i; +const TIME_HINT_REGEX = + /(今天|昨日|昨天|本周|上周|本月|上月|本季度|上季度|本年|去年|近\d+\s*(天|周|月|年)|最近|过去|between|from|to|日期|时间)/i; +const TREND_HINT_REGEX = /(趋势|变化|对比|同比|环比|走势)/i; +const AMBIGUOUS_REFERENCE_REGEX = /(这个|那个|它|他们|这项|该指标|这个指标)/i; + +class SemanticEvaluatorTimeoutError extends Error { + constructor() { + super("semantic evaluator timed out"); + } +} + +export type ClarificationSemanticEvaluationStatus = + | "success" + | "timeout" + | "error" + | "invalid"; + +export interface ClarificationSemanticEvaluationInput { + question: string; + ruleDecision: ClarificationDecisionEvidence; +} + +export interface ClarificationSemanticEvaluationMetadata { + timeoutMs: number; + elapsedMs: number; + fallbackApplied: boolean; + fallbackReason?: "timeout" | "error" | "invalid_payload"; + errorMessage?: string; +} + +export interface ClarificationSemanticEvaluationResult { + status: ClarificationSemanticEvaluationStatus; + decision?: ClarificationDecisionEvidence; + reasonCodes: string[]; + metadata: ClarificationSemanticEvaluationMetadata; +} + +interface SemanticArbitrationPayload { + decision: ClarificationDecision; + confidenceLevel: ClarificationConfidenceLevel; + missingCriticalSlots: ClarificationSlotKey[]; + reasonCodes: string[]; + reason: string; + question: string; +} + +@Injectable() +export class ClarificationSemanticEvaluatorService { + constructor(private readonly appConfig: AppConfigService) {} + + async evaluate( + input: ClarificationSemanticEvaluationInput + ): Promise { + const timeoutMs = this.appConfig.clarificationSemanticTimeoutMs; + const startedAt = Date.now(); + + try { + const payload = await this.withTimeout( + this.runSemanticArbitration(input), + timeoutMs + ); + const elapsedMs = this.resolveElapsedMs(startedAt); + const normalized = this.normalizePayload(payload, input.ruleDecision); + if (!normalized) { + return { + status: "invalid", + reasonCodes: ["semantic_invalid_payload"], + metadata: { + timeoutMs, + elapsedMs, + fallbackApplied: true, + fallbackReason: "invalid_payload" + } + }; + } + + return { + status: "success", + decision: normalized, + reasonCodes: normalized.reasonCodes ?? [], + metadata: { + timeoutMs, + elapsedMs, + fallbackApplied: false + } + }; + } catch (error) { + const elapsedMs = this.resolveElapsedMs(startedAt); + if (error instanceof SemanticEvaluatorTimeoutError) { + return { + status: "timeout", + reasonCodes: ["semantic_timeout"], + metadata: { + timeoutMs, + elapsedMs, + fallbackApplied: true, + fallbackReason: "timeout" + } + }; + } + return { + status: "error", + reasonCodes: ["semantic_error"], + metadata: { + timeoutMs, + elapsedMs, + fallbackApplied: true, + fallbackReason: "error", + errorMessage: error instanceof Error ? error.message : String(error) + } + }; + } + } + + protected async runSemanticArbitration( + input: ClarificationSemanticEvaluationInput + ): Promise { + const normalizedQuestion = input.question.trim(); + const normalized = normalizedQuestion.toLowerCase(); + + const missingCriticalSlots = this.collectMissingCriticalSlots(normalized); + const decision: ClarificationDecision = + missingCriticalSlots.length > 0 ? "clarify" : "continue"; + + const ambiguousReference = AMBIGUOUS_REFERENCE_REGEX.test(normalized); + const confidenceLevel = this.resolveConfidenceLevel( + decision, + missingCriticalSlots, + ambiguousReference + ); + const reasonCodes = this.resolveReasonCodes({ + decision, + missingCriticalSlots, + ambiguousReference + }); + + return { + decision, + confidenceLevel, + missingCriticalSlots, + reasonCodes, + reason: + decision === "clarify" + ? this.buildClarificationReason(missingCriticalSlots) + : "语义评估通过，关键槽位完整。", + question: + decision === "clarify" + ? this.buildClarificationQuestion(missingCriticalSlots) + : "" + }; + } + + private normalizePayload( + payload: SemanticArbitrationPayload, + ruleDecision: ClarificationDecisionEvidence + ): ClarificationDecisionEvidence | undefined { + if (payload.decision !== "continue" && payload.decision !== "clarify") { + return undefined; + } + + const confidenceLevel = + payload.confidenceLevel === "high" || + payload.confidenceLevel === "medium" || + payload.confidenceLevel === "low" + ? payload.confidenceLevel + : "medium"; + + const missingCriticalSlots = Array.isArray(payload.missingCriticalSlots) + ? payload.missingCriticalSlots.filter((slot): slot is ClarificationSlotKey => { + return typeof slot === "string" && slot.trim().length > 0; + }) + : []; + + const reasonCodes = this.unique([ + "semantic_classifier_v1", + ...(Array.isArray(payload.reasonCodes) + ? payload.reasonCodes.filter( + (item): item is string => typeof item === "string" && item.trim().length > 0 + ) + : []) + ]); + + return { + decision: payload.decision, + triggerPath: "semantic", + confidenceLevel, + missingCriticalSlots, + conflictDetected: + ruleDecision.decision !== undefined && ruleDecision.decision !== payload.decision, + reasonCodes, + question: payload.question, + reason: payload.reason + }; + } + + private collectMissingCriticalSlots(question: string): ClarificationSlotKey[] { + const missing = new Set(); + + if (!SUBJECT_HINT_REGEX.test(question)) { + missing.add("subject"); + } + if (!METRIC_HINT_REGEX.test(question)) { + missing.add("metric"); + } + if (TREND_HINT_REGEX.test(question) && !TIME_HINT_REGEX.test(question)) { + missing.add("time"); + } + + return [...missing]; + } + + private resolveConfidenceLevel( + decision: ClarificationDecision, + missingCriticalSlots: ClarificationSlotKey[], + ambiguousReference: boolean + ): ClarificationConfidenceLevel { + if (decision === "clarify") { + return missingCriticalSlots.length > 1 ? "low" : "medium"; + } + return ambiguousReference ? "medium" : "high"; + } + + private resolveReasonCodes(input: { + decision: ClarificationDecision; + missingCriticalSlots: ClarificationSlotKey[]; + ambiguousReference: boolean; + }): string[] { + const reasonCodes: string[] = []; + if (input.decision === "clarify") { + reasonCodes.push("semantic_needs_clarification"); + if (input.missingCriticalSlots.includes("subject")) { + reasonCodes.push("semantic_missing_subject"); + } + if (input.missingCriticalSlots.includes("metric")) { + reasonCodes.push("semantic_missing_metric"); + } + if (input.missingCriticalSlots.includes("time")) { + reasonCodes.push("semantic_missing_time"); + } + } else { + reasonCodes.push("semantic_ready_to_continue"); + } + if (input.ambiguousReference) { + reasonCodes.push("semantic_ambiguous_reference"); + } + return this.unique(reasonCodes); + } + + private buildClarificationReason(missingCriticalSlots: ClarificationSlotKey[]): string { + const labels: Record = { + subject: "分析对象", + metric: "指标口径", + time: "时间范围" + }; + const missingLabels = missingCriticalSlots + .map((slot) => labels[slot]) + .filter((label): label is string => Boolean(label)); + if (missingLabels.length === 0) { + return "语义评估建议补充关键信息。"; + } + return `语义评估发现关键槽位不足：${missingLabels.join("、")}。`; + } + + private buildClarificationQuestion(missingCriticalSlots: ClarificationSlotKey[]): string { + const missingSet = new Set(missingCriticalSlots); + if (missingSet.has("subject") && missingSet.has("metric")) { + return "请补充分析对象和指标口径，例如“近30天订单总数”或“按渠道统计退款金额”。"; + } + if (missingSet.has("subject")) { + return "请补充分析对象（例如订单、用户、商品或门店）。"; + } + if (missingSet.has("metric")) { + return "请补充要统计的指标口径（例如订单数、GMV、退款金额）。"; + } + if (missingSet.has("time")) { + return "请补充时间范围（例如近30天、本季度或具体起止日期）。"; + } + return "请补充关键分析信息后继续。"; + } + + private withTimeout(promise: Promise, timeoutMs: number): Promise { + return new Promise((resolve, reject) => { + const timeout = setTimeout(() => reject(new SemanticEvaluatorTimeoutError()), timeoutMs); + promise + .then((value) => { + clearTimeout(timeout); + resolve(value); + }) + .catch((error) => { + clearTimeout(timeout); + reject(error); + }); + }); + } + + private resolveElapsedMs(startedAt: number): number { + return Math.max(0, Date.now() - startedAt); + } + + private unique(values: string[]): string[] { + return [...new Set(values.map((value) => value.trim()).filter(Boolean))]; + } +} diff --git a/apps/backend/src/modules/conversation/agent/nodes/clarify.node.ts b/apps/backend/src/modules/conversation/agent/nodes/clarify.node.ts index 76478ca..b957761 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/clarify.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/clarify.node.ts @@ -1,27 +1,40 @@ import { Injectable } from "@nestjs/common"; import type { ClarificationPrompt, ContextEnvelope } from "@text2sql/shared-types"; -import { decideSlotFilling } from "./slot-filling-context"; +import { + buildFallbackSlotFillingDecision, + decideSlotFilling, + type SlotFillingDecision +} from "./slot-filling-context"; @Injectable() export class ClarifyNode { + evaluate(question: string, contextEnvelope?: ContextEnvelope): SlotFillingDecision { + try { + return decideSlotFilling(question, contextEnvelope); + } catch { + return buildFallbackSlotFillingDecision(); + } + } + run( question: string, contextEnvelope?: ContextEnvelope ): ClarificationPrompt | undefined { - try { - const decision = decideSlotFilling(question, contextEnvelope); - if (!decision.shouldClarify) { - return undefined; - } - return { - reason: decision.reason, - question: decision.question - }; - } catch { - return { - reason: "槽位解析异常", - question: "请补充分析对象、指标口径和时间范围后重试。" - }; + const decision = this.evaluate(question, contextEnvelope); + if (!decision.shouldClarify) { + return undefined; } + return { + reason: decision.reason, + question: decision.question, + decision: decision.decision, + triggerPath: decision.triggerPath, + decisionSource: decision.decisionSource, + bypassed: decision.bypassed, + ...(decision.bypassReasonCode ? { bypassReasonCode: decision.bypassReasonCode } : {}), + confidenceLevel: decision.confidenceLevel, + missingCriticalSlots: decision.missingCriticalSlots, + ...(decision.reasonCodes.length > 0 ? { reasonCodes: decision.reasonCodes } : {}) + }; } } diff --git a/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts b/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts index b40003e..7859b9a 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/generate-sql.node.ts @@ -4,7 +4,11 @@ import type { LlmGatewayStreamEvent, LlmGatewayToolDefinition } from "../../../llm/llm-gateway.interface"; -import { SqlGenerationService } from "../sql/sql-generation.service"; +import { + SqlGenerationService, + type SqlEvidenceCoverage, + type SqlGenerationExplicitPinningEvidence +} from "../sql/sql-generation.service"; import type { SqlSemanticIntent } from "../sql/sql-prompt.builder"; import type { RagContextPack, @@ -28,6 +32,7 @@ export class GenerateSqlNode { datasourceId?: string; workspaceId?: string; semanticIntent?: SqlSemanticIntent; + explicitPinning?: SqlGenerationExplicitPinningEvidence; } ): Promise<{ provider: string; @@ -43,6 +48,7 @@ export class GenerateSqlNode { promptTemplate?: PromptTemplateTraceEvidence; retryCount?: number; semanticIntent?: SqlSemanticIntent; + coverage?: SqlEvidenceCoverage; }> { if (options?.stream) { return this.sqlGeneration.stream( @@ -54,7 +60,8 @@ export class GenerateSqlNode { modelCatalogId, selectedContext: options.selectedContext, semanticContextPack: options.semanticContextPack, - semanticIntent: options.semanticIntent + semanticIntent: options.semanticIntent, + explicitPinning: options.explicitPinning }, { tools: options.tools, @@ -69,7 +76,8 @@ export class GenerateSqlNode { modelCatalogId, selectedContext: options?.selectedContext, semanticContextPack: options?.semanticContextPack, - semanticIntent: options?.semanticIntent + semanticIntent: options?.semanticIntent, + explicitPinning: options?.explicitPinning }); } } diff --git a/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts b/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts index 778c6f6..a21b204 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/retrieve-knowledge.node.ts @@ -1,9 +1,11 @@ import { Inject, Injectable } from "@nestjs/common"; +import type { ContextEnvelopePinningEvidence } from "@text2sql/shared-types"; import { KNOWLEDGE_RAG_CONTRACT, type KnowledgeRagContract } from "../../../knowledge/contracts/knowledge-rag.contract"; import type { + RagRetrievalChunkPayload, RagContextPack, RagRetrievalBundle } from "../../../knowledge/rag/retrieval/rag-retrieval.types"; @@ -14,6 +16,18 @@ export interface RetrievedKnowledge { summary: string; retrievalBundle?: RagRetrievalBundle; contextPack?: RagContextPack; + pinning?: ContextEnvelopePinningEvidence; +} + +interface RetrievalPinningConfig { + enabled: boolean; + pinnedTables: Set; + pinnedColumns: Set; +} + +interface RetrievalPinningResult { + bundle: RagRetrievalBundle; + pinning: ContextEnvelopePinningEvidence; } @Injectable() @@ -28,11 +42,18 @@ export class RetrieveKnowledgeNode { datasourceId: string; runId: string; workspaceId?: string; + allowedTables?: string[]; modelCatalogId?: string; + pinnedTables?: string[]; + pinnedColumns?: string[]; }): Promise { const question = input.question.trim(); const datasourceId = input.datasourceId.trim(); const runId = input.runId.trim(); + const pinningConfig = this.normalizePinningConfig({ + pinnedTables: input.pinnedTables, + pinnedColumns: input.pinnedColumns + }); const normalized = question.trim(); if (!normalized) { @@ -98,6 +119,12 @@ export class RetrieveKnowledgeNode { degrade_reasons: ["empty_question"], risk_tags: ["semantic_spine_degraded"] } + }, + pinning: { + enabled: pinningConfig.enabled, + status: "inactive", + candidateFilteredCount: 0, + selectedContextFilteredCount: 0 } }; } @@ -106,13 +133,24 @@ export class RetrieveKnowledgeNode { query: normalized, datasourceId, workspaceId: input.workspaceId, + allowedTables: input.allowedTables, runId }); const reranked = await this.ragContract.rerank.rerank({ retrievalBundle: retrieved.retrieval_bundle, modelCatalogId: input.modelCatalogId }); - const bundle = reranked.retrieval_bundle; + const pinningResult = this.applyPinningConstraints( + reranked.retrieval_bundle, + pinningConfig + ); + const bundle = pinningResult.bundle; + const pinningSummary = + pinningResult.pinning.enabled && pinningResult.pinning.status === "applied" + ? `，pinning[candidates=${pinningResult.pinning.candidateFilteredCount ?? 0},selected=${pinningResult.pinning.selectedContextFilteredCount ?? 0}]` + : ""; + const priorSqlLane = bundle.prior_sql_lane ?? bundle.priorSqlLane; + const priorSqlHitCount = priorSqlLane?.selected_count ?? priorSqlLane?.selectedCount ?? 0; const snippets = (bundle.selected_context ?? bundle.candidates.map((item) => item.chunk)) .slice(0, 3) .map((item) => item.content.slice(0, 200)); @@ -122,10 +160,130 @@ export class RetrieveKnowledgeNode { snippets, summary: bundle.status === "ready" - ? `检索与重排完成，候选=${bundle.candidates.length}，上下文=${bundle.selected_context?.length ?? 0}。` - : `检索链路降级执行，原因=${bundle.degrade_reasons.join(", ") || "unknown"}。`, + ? `检索与重排完成，候选=${bundle.candidates.length}，上下文=${bundle.selected_context?.length ?? 0}，priorSQL=${priorSqlHitCount}${pinningSummary}。` + : `检索链路降级执行，原因=${bundle.degrade_reasons.join(", ") || "unknown"}，priorSQL=${priorSqlHitCount}${pinningSummary}。`, retrievalBundle: bundle, - contextPack: bundle.context_pack + contextPack: bundle.context_pack, + pinning: pinningResult.pinning + }; + } + + private normalizePinningConfig(input: { + pinnedTables?: string[]; + pinnedColumns?: string[]; + }): RetrievalPinningConfig { + const pinnedTables = this.toNormalizedSet(input.pinnedTables); + const pinnedColumns = this.toNormalizedSet(input.pinnedColumns); + return { + enabled: pinnedTables.size > 0 || pinnedColumns.size > 0, + pinnedTables, + pinnedColumns + }; + } + + private applyPinningConstraints( + bundle: RagRetrievalBundle, + pinningConfig: RetrievalPinningConfig + ): RetrievalPinningResult { + if (!pinningConfig.enabled) { + return { + bundle, + pinning: { + enabled: false, + status: "inactive", + candidateFilteredCount: 0, + selectedContextFilteredCount: 0 + } + }; + } + + const filteredCandidates = bundle.candidates.filter((candidate) => + this.matchesPinning(candidate.chunk, pinningConfig) + ); + const allowedCandidateChunkIds = new Set( + filteredCandidates.map((candidate) => candidate.chunk_id) + ); + const filteredSelectedContext = (bundle.selected_context ?? []).filter((chunk) => + this.matchesPinning(chunk, pinningConfig) + ); + const filteredReranked = bundle.reranked?.filter((candidate) => + allowedCandidateChunkIds.has(candidate.chunk_id) + ); + const candidateFilteredCount = Math.max(0, bundle.candidates.length - filteredCandidates.length); + const selectedContextFilteredCount = Math.max( + 0, + (bundle.selected_context?.length ?? 0) - filteredSelectedContext.length + ); + + const nextBundle: RagRetrievalBundle = { + ...bundle, + candidates: filteredCandidates, + selected_context: filteredSelectedContext, + ...(filteredReranked + ? { + reranked: filteredReranked + } + : {}), + context_pack: this.withPinnedContextPack(bundle.context_pack, filteredSelectedContext) + }; + + return { + bundle: nextBundle, + pinning: { + enabled: true, + status: "applied", + candidateFilteredCount, + selectedContextFilteredCount + } }; } + + private withPinnedContextPack( + contextPack: RagContextPack | undefined, + selectedContext: RagRetrievalChunkPayload[] + ): RagContextPack | undefined { + if (!contextPack) { + return contextPack; + } + return { + ...contextPack, + selected_context_summary: { + ...contextPack.selected_context_summary, + count: selectedContext.length, + snippets: selectedContext.map((chunk) => chunk.content.slice(0, 200)) + } + }; + } + + private matchesPinning( + chunk: RagRetrievalChunkPayload, + pinningConfig: RetrievalPinningConfig + ): boolean { + if (pinningConfig.pinnedTables.size > 0) { + const tableNames = chunk.metadata.tableNames.map((item) => item.trim().toLowerCase()); + const tableMatched = tableNames.some((name) => pinningConfig.pinnedTables.has(name)); + if (!tableMatched) { + return false; + } + } + if (pinningConfig.pinnedColumns.size > 0) { + const columnNames = chunk.metadata.columnNames.map((item) => item.trim().toLowerCase()); + const columnMatched = columnNames.some((name) => pinningConfig.pinnedColumns.has(name)); + if (!columnMatched) { + return false; + } + } + return true; + } + + private toNormalizedSet(values?: string[]): Set { + if (!Array.isArray(values) || values.length === 0) { + return new Set(); + } + return new Set( + values + .map((item) => item.trim().toLowerCase()) + .filter((item) => item.length > 0) + ); + } } diff --git a/apps/backend/src/modules/conversation/agent/nodes/slot-filling-context.ts b/apps/backend/src/modules/conversation/agent/nodes/slot-filling-context.ts index 90a9cd7..ca1d075 100644 --- a/apps/backend/src/modules/conversation/agent/nodes/slot-filling-context.ts +++ b/apps/backend/src/modules/conversation/agent/nodes/slot-filling-context.ts @@ -1,6 +1,19 @@ -import type { ContextEnvelope } from "@text2sql/shared-types"; +import type { + ClarificationConfidenceLevel, + ClarificationTriggerPath, + ContextEnvelope +} from "@text2sql/shared-types"; -type SlotKey = "subject" | "metric" | "time" | "dimension" | "filter"; +export type SlotKey = "subject" | "metric" | "time" | "dimension" | "filter"; +type SlotFillingDecisionState = "continue" | "clarify"; +type SlotFillingAction = "proceed" | "ask_clarification"; +type SlotFillingSource = + | "rule" + | "metadata-intent" + | "sql-write-intent" + | "short-input-fallback" + | "exception-fallback"; +type SlotFillingConfidence = ClarificationConfidenceLevel; interface SlotStatus { key: SlotKey; @@ -9,12 +22,115 @@ interface SlotStatus { } export interface SlotFillingDecision { + decision: SlotFillingDecisionState; + action: SlotFillingAction; + source: SlotFillingSource; + decisionSource: SlotFillingSource; + triggerPath: ClarificationTriggerPath; + bypassed: boolean; + bypassReasonCode?: string; + confidence: SlotFillingConfidence; + confidenceLevel: SlotFillingConfidence; + missingSlots: SlotKey[]; shouldClarify: boolean; missingCriticalSlots: SlotKey[]; + reasonCodes: string[]; reason: string; question: string; } +interface SlotFillingDecisionInput { + decision: SlotFillingDecisionState; + source: SlotFillingSource; + confidence: SlotFillingConfidence; + missingSlots: SlotKey[]; + reasonCodes?: string[]; + reason: string; + question?: string; +} + +function buildDecision(input: SlotFillingDecisionInput): SlotFillingDecision { + const shouldClarify = input.decision === "clarify"; + const reasonCodes = uniqueReasonCodes( + input.reasonCodes ?? resolveReasonCodes(input.source, input.missingSlots) + ); + const bypassed = + input.source === "metadata-intent" || input.source === "sql-write-intent"; + return { + decision: input.decision, + action: shouldClarify ? "ask_clarification" : "proceed", + source: input.source, + decisionSource: input.source, + triggerPath: "rule", + bypassed, + bypassReasonCode: bypassed ? reasonCodes[0] : undefined, + confidence: input.confidence, + confidenceLevel: input.confidence, + missingSlots: input.missingSlots, + shouldClarify, + missingCriticalSlots: input.missingSlots, + reasonCodes, + reason: input.reason, + question: input.question ?? "" + }; +} + +const MISSING_SLOT_REASON_CODE: Record = { + subject: "missing_subject_slot", + metric: "missing_metric_slot", + time: "missing_time_slot", + dimension: "missing_dimension_slot", + filter: "missing_filter_slot" +}; + +function resolveReasonCodes(source: SlotFillingSource, missingSlots: SlotKey[]): string[] { + const missingReasonCodes = missingSlots.map((slot) => MISSING_SLOT_REASON_CODE[slot]); + if (source === "metadata-intent") { + return ["bypass_metadata_intent"]; + } + if (source === "sql-write-intent") { + return ["bypass_sql_write_intent"]; + } + if (source === "short-input-fallback") { + return ["fallback_short_input", ...missingReasonCodes]; + } + if (source === "exception-fallback") { + return ["fallback_exception", ...missingReasonCodes]; + } + if (missingReasonCodes.length === 0) { + return ["rule_slots_sufficient"]; + } + return missingReasonCodes; +} + +function uniqueReasonCodes(reasonCodes: string[]): string[] { + return Array.from( + new Set(reasonCodes.filter((reasonCode) => reasonCode.trim().length > 0)) + ); +} + +export function buildFallbackSlotFillingDecision(): SlotFillingDecision { + const missingSlots: SlotKey[] = ["subject", "metric", "time"]; + return buildDecision({ + decision: "clarify", + source: "exception-fallback", + confidence: "low", + missingSlots, + reason: "槽位解析异常", + question: "请补充分析对象、指标口径和时间范围后重试。" + }); +} + +function resolveConfidence(missingSlots: SlotKey[]): SlotFillingConfidence { + if (missingSlots.length === 0) { + return "high"; + } + if (missingSlots.length === 1) { + return "medium"; + } + return "low"; +} + const METRIC_HINT_REGEX = /(总数|数量|计数|count|金额|交易额|销售额|gmv|平均|均值|占比|比例|转化率|留存|退款率|分布|top|排行|环比|同比)/i; const SUBJECT_HINT_REGEX = @@ -145,29 +261,35 @@ export function decideSlotFilling( ): SlotFillingDecision { const trimmedQuestion = question.trim(); if (SQL_WRITE_INTENT_REGEX.test(trimmedQuestion)) { - return { - shouldClarify: false, - missingCriticalSlots: [], + return buildDecision({ + decision: "continue", + source: "sql-write-intent", + confidence: "high", + missingSlots: [], reason: "检测到写操作 SQL 意图，跳过槽位澄清", question: "" - }; + }); } if (METADATA_INTENT_REGEX.test(trimmedQuestion)) { - return { - shouldClarify: false, - missingCriticalSlots: [], + return buildDecision({ + decision: "continue", + source: "metadata-intent", + confidence: "high", + missingSlots: [], reason: "元数据查询意图，无需业务槽位补全", question: "" - }; + }); } if (trimmedQuestion.length < 6) { - const missingCriticalSlots: SlotKey[] = ["subject", "metric", "time"]; - return { - shouldClarify: true, - missingCriticalSlots, - reason: resolveReason(missingCriticalSlots), - question: resolveClarificationQuestion(missingCriticalSlots) - }; + const missingSlots: SlotKey[] = ["subject", "metric", "time"]; + return buildDecision({ + decision: "clarify", + source: "short-input-fallback", + confidence: "low", + missingSlots, + reason: resolveReason(missingSlots), + question: resolveClarificationQuestion(missingSlots) + }); } const slots = resolveSlots(trimmedQuestion, contextEnvelope); @@ -186,11 +308,13 @@ export function decideSlotFilling( missingSet.add("time"); } - const missingCriticalSlots = Array.from(missingSet); - return { - shouldClarify: missingCriticalSlots.length > 0, - missingCriticalSlots, - reason: resolveReason(missingCriticalSlots), - question: resolveClarificationQuestion(missingCriticalSlots) - }; + const missingSlots = Array.from(missingSet); + return buildDecision({ + decision: missingSlots.length > 0 ? "clarify" : "continue", + source: "rule", + confidence: resolveConfidence(missingSlots), + missingSlots, + reason: resolveReason(missingSlots), + question: resolveClarificationQuestion(missingSlots) + }); } diff --git a/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts b/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts index ad4e887..f65c5ff 100644 --- a/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts +++ b/apps/backend/src/modules/conversation/agent/sql/sql-generation.service.ts @@ -27,8 +27,16 @@ interface SqlGenerationSelection { datasourceType?: DatasourceType; modelCatalogId?: string; selectedContext?: RagRetrievalChunkPayload[]; + selected_context?: RagRetrievalChunkPayload[]; semanticContextPack?: RagContextPack; semanticIntent?: SqlSemanticIntent; + explicitPinning?: SqlGenerationExplicitPinningEvidence; +} + +export interface SqlGenerationExplicitPinningEvidence { + source?: string; + tables?: string[]; + columns?: string[]; } const MAX_SEMANTIC_REPAIR_RETRY = 1; @@ -39,6 +47,27 @@ const METADATA_INTENT_REGEX = const COUNT_SQL_REGEX = /\bcount\s*\(/i; const METADATA_SQL_REGEX = /\bsqlite_master\b|\bsqlite_schema\b|\binformation_schema\b|\bpg_catalog\b|\bshow\s+tables\b|\bdescribe\b|\bpragma\b/i; +const SQL_TABLE_REF_REGEX = /\b(?:from|join)\s+([`"'[\]]?[a-zA-Z_][\w$]*(?:\.[a-zA-Z_][\w$]*)?[`"'[\]]?)/gi; +const SQL_QUALIFIED_COLUMN_REGEX = /\b([a-zA-Z_][\w$]*)\.([a-zA-Z_][\w$]*)\b/g; +const SQL_CTE_REGEX = /(?:\bwith\b|,)\s*([a-zA-Z_][\w$]*)\s+as\s*\(/gi; + +export type SqlCoverageTriggerSource = + | "selected_context" + | "semantic_context" + | "explicit_pinning" + | "none"; + +export interface SqlEvidenceCoverage { + gateStatus: + | "passed" + | "failed" + | "skipped_no_evidence" + | "skipped_metadata_intent" + | "skipped_no_sql_objects"; + missingObjects: string[]; + triggerSource: SqlCoverageTriggerSource; + usedObjects: string[]; +} export interface SqlDraft { provider: string; @@ -51,6 +80,7 @@ export interface SqlDraft { promptTemplate?: PromptTemplateTraceEvidence; retryCount?: number; semanticIntent?: SqlSemanticIntent; + coverage?: SqlEvidenceCoverage; } @Injectable() @@ -149,7 +179,7 @@ export class SqlGenerationService { return this.promptBuilder.build( question, selection?.datasourceType, - selection?.selectedContext, + this.resolveSelectedContext(selection), { templateOverlay, semanticGuardrail: { @@ -161,6 +191,12 @@ export class SqlGenerationService { ); } + private resolveSelectedContext( + selection?: SqlGenerationSelection + ): RagRetrievalChunkPayload[] | undefined { + return selection?.selectedContext ?? selection?.selected_context; + } + private async finalizeWithSemanticGuardrails(input: { question: string; selection: SqlGenerationSelection | undefined; @@ -182,6 +218,22 @@ export class SqlGenerationService { extracted.sql ); if (validation.valid) { + const coverage = this.validateEvidenceCoverage({ + sql: extracted.sql, + semanticIntent: input.semanticIntent, + selection: input.selection + }); + if (!coverage.valid) { + throw new DomainError( + "LLM_SQL_EVIDENCE_COVERAGE_FAILED", + "SQL 证据覆盖校验失败，已阻止不具备证据覆盖的 SQL 输出。", + 422, + { + reason: coverage.reason, + coverage: coverage.evidence + } + ); + } return { provider: completion.provider, model: completion.model, @@ -192,7 +244,8 @@ export class SqlGenerationService { prompt, promptTemplate: input.promptTemplate, retryCount, - semanticIntent: input.semanticIntent + semanticIntent: input.semanticIntent, + coverage: coverage.evidence }; } @@ -291,6 +344,421 @@ export class SqlGenerationService { return { valid: true }; } + private validateEvidenceCoverage(input: { + sql: string; + semanticIntent: SqlSemanticIntent; + selection: SqlGenerationSelection | undefined; + }): { + valid: boolean; + reason?: string; + evidence: SqlEvidenceCoverage; + } { + if (input.semanticIntent === "metadata") { + return { + valid: true, + evidence: { + gateStatus: "skipped_metadata_intent", + missingObjects: [], + triggerSource: "none", + usedObjects: [] + } + }; + } + + const evidenceIndex = this.buildCoverageEvidenceIndex(input.selection); + const triggerSource = this.resolveCoverageTriggerSource(evidenceIndex); + if (!evidenceIndex.hasEvidence) { + return { + valid: true, + evidence: { + gateStatus: "skipped_no_evidence", + missingObjects: [], + triggerSource, + usedObjects: [] + } + }; + } + + const references = this.extractSqlReferences(input.sql); + const usedObjects = this.unique([ + ...Array.from(references.tables).map((table) => `table:${table}`), + ...Array.from(references.tableColumns).map((tableColumn) => `column:${tableColumn}`), + ...Array.from(references.columns).map((column) => `column:${column}`) + ]); + + if (usedObjects.length === 0) { + return { + valid: true, + evidence: { + gateStatus: "skipped_no_sql_objects", + missingObjects: [], + triggerSource, + usedObjects: [] + } + }; + } + + const missingObjects = this.unique([ + ...Array.from(references.tables) + .filter((table) => !evidenceIndex.tables.has(table)) + .map((table) => `table:${table}`), + ...Array.from(references.tableColumns) + .filter((tableColumn) => { + const column = tableColumn.split(".")[1]; + return ( + !evidenceIndex.tableColumns.has(tableColumn) && + !evidenceIndex.columns.has(column) + ); + }) + .map((tableColumn) => `column:${tableColumn}`), + ...Array.from(references.columns) + .filter( + (column) => + !evidenceIndex.columns.has(column) && + !this.hasTableColumnFallback(evidenceIndex.tableColumns, column) + ) + .map((column) => `column:${column}`) + ]); + + if (missingObjects.length > 0) { + const shouldEnforceBlock = + evidenceIndex.hasExplicitPinning && + !evidenceIndex.hasSelectedContext && + !evidenceIndex.hasSemanticContext; + if (!shouldEnforceBlock) { + return { + valid: true, + evidence: { + gateStatus: "failed", + missingObjects, + triggerSource, + usedObjects + } + }; + } + return { + valid: false, + reason: `missing evidence for ${missingObjects.join(", ")}`, + evidence: { + gateStatus: "failed", + missingObjects, + triggerSource, + usedObjects + } + }; + } + + return { + valid: true, + evidence: { + gateStatus: "passed", + missingObjects: [], + triggerSource, + usedObjects + } + }; + } + + private buildCoverageEvidenceIndex(selection?: SqlGenerationSelection): { + hasEvidence: boolean; + tables: Set; + columns: Set; + tableColumns: Set; + hasSelectedContext: boolean; + hasSemanticContext: boolean; + hasExplicitPinning: boolean; + } { + const tables = new Set(); + const columns = new Set(); + const tableColumns = new Set(); + const selectedContext = this.resolveSelectedContext(selection); + let hasSelectedContext = false; + for (const chunk of selectedContext ?? []) { + const chunkTables = (chunk.metadata.tableNames ?? []) + .map((item) => this.normalizeIdentifier(item)) + .filter((item): item is string => Boolean(item)); + const chunkColumns = (chunk.metadata.columnNames ?? []) + .map((item) => this.normalizeIdentifier(item)) + .filter((item): item is string => Boolean(item)); + if (chunkTables.length > 0 || chunkColumns.length > 0) { + hasSelectedContext = true; + } + for (const table of chunkTables) { + tables.add(table); + } + for (const column of chunkColumns) { + columns.add(column); + } + if (chunkTables.length === 1) { + const table = chunkTables[0]; + for (const column of chunkColumns) { + tableColumns.add(`${table}.${column}`); + } + } + } + + const semanticEvidence = this.extractSemanticEvidence(selection?.semanticContextPack); + for (const table of semanticEvidence.tables) { + tables.add(table); + } + for (const column of semanticEvidence.columns) { + columns.add(column); + } + for (const tableColumn of semanticEvidence.tableColumns) { + tableColumns.add(tableColumn); + } + + const explicitPinning = selection?.explicitPinning; + const explicitTables = (explicitPinning?.tables ?? []) + .map((item) => this.normalizeIdentifier(item)) + .filter((item): item is string => Boolean(item)); + const explicitColumns = (explicitPinning?.columns ?? []) + .map((item) => this.normalizeIdentifier(item)) + .filter((item): item is string => Boolean(item)); + const hasExplicitPinning = explicitTables.length > 0 || explicitColumns.length > 0; + for (const table of explicitTables) { + tables.add(table); + } + for (const column of explicitColumns) { + columns.add(column); + if (column.includes(".")) { + const [table, field] = column.split("."); + if (table && field) { + tables.add(table); + columns.add(field); + tableColumns.add(`${table}.${field}`); + } + } + } + + return { + hasEvidence: hasSelectedContext || semanticEvidence.hasSemanticContext || hasExplicitPinning, + tables, + columns, + tableColumns, + hasSelectedContext, + hasSemanticContext: semanticEvidence.hasSemanticContext, + hasExplicitPinning + }; + } + + private extractSemanticEvidence(contextPack?: RagContextPack): { + hasSemanticContext: boolean; + tables: Set; + columns: Set; + tableColumns: Set; + } { + const tables = new Set(); + const columns = new Set(); + const tableColumns = new Set(); + if (!contextPack || contextPack.status === "degraded") { + return { + hasSemanticContext: false, + tables, + columns, + tableColumns + }; + } + + const contextPackCompat = contextPack as RagContextPack & { + semanticBindings?: RagContextPack["semantic_bindings"]; + instructionSets?: RagContextPack["instruction_sets"]; + }; + const semanticBindingsRecord = (contextPack.semantic_bindings ?? + contextPackCompat.semanticBindings) as unknown as + | Record + | undefined; + const instructionSetsRecord = (contextPack.instruction_sets ?? + contextPackCompat.instructionSets) as unknown as + | Record + | undefined; + const candidates = this.unique([ + ...this.readStringArray(semanticBindingsRecord?.model_keys), + ...this.readStringArray(semanticBindingsRecord?.relationship_keys), + ...this.readStringArray(semanticBindingsRecord?.metric_keys), + ...this.readStringArray(semanticBindingsRecord?.calculated_field_keys), + ...this.readStringArray(instructionSetsRecord?.model_bindings), + ...this.readStringArray(instructionSetsRecord?.relationship_bindings), + ...this.readStringArray(instructionSetsRecord?.metric_bindings), + ...this.readStringArray(instructionSetsRecord?.calculated_field_bindings) + ]); + + for (const candidate of candidates) { + for (const pair of this.extractQualifiedPairs(candidate)) { + tables.add(pair.table); + columns.add(pair.column); + tableColumns.add(`${pair.table}.${pair.column}`); + } + const modelMatch = /(?:^|[^\w])(model|table)[.:]([a-zA-Z_][\w$]*)/i.exec(candidate); + if (modelMatch?.[2]) { + const table = this.normalizeIdentifier(modelMatch[2]); + if (table) { + tables.add(table); + } + } + } + + return { + hasSemanticContext: candidates.length > 0, + tables, + columns, + tableColumns + }; + } + + private resolveCoverageTriggerSource(input: { + hasSelectedContext: boolean; + hasSemanticContext: boolean; + hasExplicitPinning: boolean; + }): SqlCoverageTriggerSource { + if (input.hasExplicitPinning) { + return "explicit_pinning"; + } + if (input.hasSelectedContext) { + return "selected_context"; + } + if (input.hasSemanticContext) { + return "semantic_context"; + } + return "none"; + } + + private extractSqlReferences(sql: string): { + tables: Set; + columns: Set; + tableColumns: Set; + } { + const tables = new Set(); + const columns = new Set(); + const tableColumns = new Set(); + const cteNames = new Set(); + const cteRegex = new RegExp(SQL_CTE_REGEX.source, "gi"); + for (const match of sql.matchAll(cteRegex)) { + const cteName = this.normalizeIdentifier(match[1]); + if (cteName) { + cteNames.add(cteName); + } + } + + const tableRegex = new RegExp(SQL_TABLE_REF_REGEX.source, "gi"); + for (const match of sql.matchAll(tableRegex)) { + const table = this.normalizeIdentifier(match[1]); + if (!table || cteNames.has(table)) { + continue; + } + tables.add(table); + } + + const qualifiedRegex = new RegExp(SQL_QUALIFIED_COLUMN_REGEX.source, "gi"); + for (const match of sql.matchAll(qualifiedRegex)) { + const table = this.normalizeIdentifier(match[1]); + const column = this.normalizeIdentifier(match[2]); + if (!table || !column) { + continue; + } + if (!cteNames.has(table)) { + tables.add(table); + } + columns.add(column); + tableColumns.add(`${table}.${column}`); + } + + for (const column of this.extractSimpleSelectColumns(sql)) { + columns.add(column); + } + + return { tables, columns, tableColumns }; + } + + private extractSimpleSelectColumns(sql: string): string[] { + const selectMatch = /\bselect\b([\s\S]*?)\bfrom\b/i.exec(sql); + if (!selectMatch?.[1]) { + return []; + } + const chunks = selectMatch[1].split(","); + const parsed: string[] = []; + for (const chunk of chunks) { + let candidate = chunk.trim(); + if (!candidate || candidate === "*") { + continue; + } + if (candidate.includes(".") || candidate.includes("(")) { + continue; + } + candidate = candidate.replace(/\bas\s+[a-zA-Z_][\w$]*$/i, "").trim(); + candidate = candidate.replace(/^distinct\s+/i, "").trim(); + const match = /^([a-zA-Z_][\w$]*)/.exec(candidate); + if (!match?.[1]) { + continue; + } + const normalized = this.normalizeIdentifier(match[1]); + if (normalized) { + parsed.push(normalized); + } + } + return this.unique(parsed); + } + + private extractQualifiedPairs(value: string): Array<{ table: string; column: string }> { + const normalized = value.replace(/->/g, " ").replace(/:/g, " "); + const regex = new RegExp(SQL_QUALIFIED_COLUMN_REGEX.source, "gi"); + const pairs: Array<{ table: string; column: string }> = []; + for (const match of normalized.matchAll(regex)) { + const table = this.normalizeIdentifier(match[1]); + const column = this.normalizeIdentifier(match[2]); + if (!table || !column) { + continue; + } + if (["model", "table", "metric", "relationship", "calculated"].includes(table)) { + continue; + } + pairs.push({ table, column }); + } + return pairs; + } + + private normalizeIdentifier(value: string | undefined): string | undefined { + if (!value) { + return undefined; + } + const normalized = value + .trim() + .replace(/^[`"'[\]]+|[`"'[\]]+$/g, "") + .replace(/\s+/g, ""); + if (!normalized) { + return undefined; + } + const token = normalized.includes(".") + ? normalized.split(".").at(-1) + : normalized; + return token?.toLowerCase(); + } + + private hasTableColumnFallback( + tableColumns: Set, + column: string + ): boolean { + for (const tableColumn of tableColumns) { + if (tableColumn.endsWith(`.${column}`)) { + return true; + } + } + return false; + } + + private readStringArray(value: unknown): string[] { + if (!Array.isArray(value)) { + return []; + } + return value + .map((item) => (typeof item === "string" ? item.trim() : "")) + .filter((item): item is string => item.length > 0); + } + + private unique(values: string[]): string[] { + return Array.from(new Set(values)); + } + private async resolvePromptTemplate(selection?: { datasourceId?: string; workspaceId?: string; diff --git a/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts b/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts index bd93d08..a2a4777 100644 --- a/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts +++ b/apps/backend/src/modules/conversation/agent/sql/sql-prompt.builder.ts @@ -8,6 +8,19 @@ type RagRetrievalChunkPayload = NonNullable< NonNullable["selected_context"] >[number]; +interface SqlPruningColumnHint { + name: string; + reason?: string; +} + +interface SqlPruningEntryHint { + sourceLabel: string; + tableName?: string; + columns: SqlPruningColumnHint[]; + reasons: string[]; + ambiguousOrLowConfidence: boolean; +} + export type SqlSemanticIntent = "count" | "metadata" | "general"; const DIALECT_HINT: Record = { @@ -111,14 +124,356 @@ export class SqlPromptBuilder { if (!selectedContext || selectedContext.length === 0) { return ""; } - const lines = selectedContext.slice(0, 5).map((item, index) => { + const legacyLines = this.buildLegacyContextLines(selectedContext); + const pruningLines = this.buildPruningContextLines(selectedContext); + if (pruningLines.length === 0) { + return ["Retrieved context (trusted evidence):", ...legacyLines].join("\n"); + } + return [ + "Retrieved context (trusted evidence):", + "Selected-context pruning view (preferred when available):", + ...pruningLines, + "Raw context snippets (compatibility fallback):", + ...legacyLines + ].join("\n"); + } + + private buildLegacyContextLines(selectedContext: RagRetrievalChunkPayload[]): string[] { + return selectedContext.slice(0, 5).map((item, index) => { const domain = item.metadata.domain; const chunkId = item.chunk_id; const compact = item.content.replace(/\s+/g, " ").trim(); const excerpt = compact.length > 260 ? `${compact.slice(0, 260)}...` : compact; return `${index + 1}. [${domain}] ${chunkId}: ${excerpt}`; }); - return ["Retrieved context (trusted evidence):", ...lines].join("\n"); + } + + private buildPruningContextLines(selectedContext: RagRetrievalChunkPayload[]): string[] { + return selectedContext + .slice(0, 5) + .flatMap((item, index) => this.toPruningContextLines(item, index)); + } + + private toPruningContextLines( + item: RagRetrievalChunkPayload, + index: number + ): string[] { + const hint = this.readPruningHint(item); + if (!hint) { + return []; + } + const header = [ + `${index + 1}. [${item.metadata.domain}] ${item.chunk_id}`, + `table=${hint.tableName ?? "unknown"}`, + `columns=${this.formatColumnHints(hint.columns)}`, + `source=${hint.sourceLabel}` + ].join(" | "); + const lines = [header]; + if (hint.reasons.length > 0) { + lines.push(` rationale: ${hint.reasons.slice(0, 3).join("; ")}`); + } + if (hint.ambiguousOrLowConfidence) { + lines.push( + " conservative_note: low-confidence/ambiguous pruning detected, full chunk columns retained to avoid dropping required fields." + ); + } + return lines; + } + + private formatColumnHints(columns: SqlPruningColumnHint[]): string { + if (columns.length === 0) { + return "none"; + } + return columns + .map((column) => + column.reason ? `${column.name} (${column.reason})` : column.name + ) + .join(", "); + } + + private readPruningHint(item: RagRetrievalChunkPayload): SqlPruningEntryHint | undefined { + const sourceMetadata = this.asRecord(item.metadata.sourceMetadata); + if (!sourceMetadata) { + return undefined; + } + const source = this.readPruningSource(sourceMetadata); + if (!source) { + return undefined; + } + + const perTableSource = this.readPerTablePruningSource(source, item.metadata.tableNames); + const effectiveSource = perTableSource ?? source; + const tableName = + this.readString(effectiveSource.tableName) ?? + this.readString(effectiveSource.table_name) ?? + this.readString(effectiveSource.table) ?? + this.readString(source.tableName) ?? + this.readString(source.table_name) ?? + this.readString(source.table) ?? + item.metadata.tableNames[0]; + + const explicitColumns = this.readColumns(effectiveSource); + const fallbackColumns = this.readStringArray(item.metadata.columnNames).map((name) => ({ + name + })); + const hasAmbiguousOrLowConfidence = + this.readAmbiguousOrLowConfidence(source) || + this.readAmbiguousOrLowConfidence(effectiveSource); + const columns = this.mergeColumnHints( + explicitColumns, + fallbackColumns, + hasAmbiguousOrLowConfidence + ); + if (columns.length === 0) { + return undefined; + } + + const reasons = this.unique( + [ + ...this.readReasonList(source.reason), + ...this.readReasonList(source.reasons), + ...this.readReasonList(source.selectionReason), + ...this.readReasonList(source.selection_reason), + ...this.readReasonList(source.evidence), + ...this.readReasonList(source.evidences), + ...this.readReasonList(effectiveSource.reason), + ...this.readReasonList(effectiveSource.reasons), + ...this.readReasonList(effectiveSource.selectionReason), + ...this.readReasonList(effectiveSource.selection_reason), + ...this.readReasonList(effectiveSource.evidence), + ...this.readReasonList(effectiveSource.evidences) + ] + .map((reason) => reason.trim()) + .filter((reason) => reason.length > 0) + ); + return { + sourceLabel: + this.readString(source.source) ?? + this.readString(source.sourceLabel) ?? + this.readString(source.source_label) ?? + "selected_context_pruning", + tableName, + columns, + reasons, + ambiguousOrLowConfidence: hasAmbiguousOrLowConfidence + }; + } + + private readPruningSource( + sourceMetadata: Record + ): Record | undefined { + const directKeys = [ + "selectedContextPruning", + "selected_context_pruning", + "prunedContext", + "pruned_context", + "pruning" + ]; + for (const key of directKeys) { + const nested = this.asRecord(sourceMetadata[key]); + if (nested) { + return nested; + } + } + if (this.readColumns(sourceMetadata).length > 0) { + return sourceMetadata; + } + return undefined; + } + + private readPerTablePruningSource( + source: Record, + tableNames: string[] + ): Record | undefined { + const tables = source.tables; + if (!Array.isArray(tables) || tables.length === 0) { + return undefined; + } + const normalizedTargets = tableNames.map((name) => name.trim().toLowerCase()); + const records = tables + .map((item) => this.asRecord(item)) + .filter((item): item is Record => Boolean(item)); + if (records.length === 0) { + return undefined; + } + const exactMatch = records.find((entry) => { + const tableName = + this.readString(entry.tableName) ?? + this.readString(entry.table_name) ?? + this.readString(entry.table); + if (!tableName) { + return false; + } + return normalizedTargets.includes(tableName.toLowerCase()); + }); + return exactMatch ?? records[0]; + } + + private readColumns(source: Record): SqlPruningColumnHint[] { + const columnKeys = [ + "selectedColumns", + "selected_columns", + "retainedColumns", + "retained_columns", + "columns", + "columnNames", + "column_names", + "keptColumns", + "kept_columns" + ]; + for (const key of columnKeys) { + const parsed = this.parseColumnHintList(source[key]); + if (parsed.length > 0) { + return parsed; + } + } + return []; + } + + private parseColumnHintList(raw: unknown): SqlPruningColumnHint[] { + if (!Array.isArray(raw)) { + return []; + } + const parsed: SqlPruningColumnHint[] = []; + for (const item of raw) { + if (typeof item === "string") { + const name = item.trim(); + if (name.length > 0) { + parsed.push({ name }); + } + continue; + } + const record = this.asRecord(item); + if (!record) { + continue; + } + const name = + this.readString(record.name) ?? + this.readString(record.column) ?? + this.readString(record.columnName) ?? + this.readString(record.column_name); + if (!name) { + continue; + } + const reason = + this.readString(record.reason) ?? + this.readString(record.selectionReason) ?? + this.readString(record.selection_reason) ?? + this.readString(record.evidence); + parsed.push({ + name, + ...(reason ? { reason } : {}) + }); + } + return parsed; + } + + private mergeColumnHints( + explicitColumns: SqlPruningColumnHint[], + fallbackColumns: SqlPruningColumnHint[], + includeFallbackColumns: boolean + ): SqlPruningColumnHint[] { + const merged = [...explicitColumns]; + if (includeFallbackColumns || explicitColumns.length === 0) { + for (const fallback of fallbackColumns) { + if (merged.some((entry) => entry.name === fallback.name)) { + continue; + } + merged.push(fallback); + } + } + return merged; + } + + private readAmbiguousOrLowConfidence(source: Record): boolean { + if (this.readBoolean(source.ambiguous) || this.readBoolean(source.lowConfidence)) { + return true; + } + if ( + this.readBoolean(source.low_confidence) || + this.readBoolean(source.ambiguity) + ) { + return true; + } + const confidence = + this.readString(source.confidenceLevel) ?? + this.readString(source.confidence_level) ?? + this.readString(source.confidence); + if (confidence === "low") { + return true; + } + if (typeof source.confidence === "number" && source.confidence < 0.6) { + return true; + } + return false; + } + + private readReasonList(raw: unknown): string[] { + if (typeof raw === "string") { + return raw.trim().length > 0 ? [raw] : []; + } + if (Array.isArray(raw)) { + return raw + .map((item) => { + const direct = this.readString(item); + if (direct) { + return direct; + } + const record = this.asRecord(item); + if (!record) { + return undefined; + } + return ( + this.readString(record.reason) ?? + this.readString(record.text) ?? + this.readString(record.message) + ); + }) + .filter((item): item is string => Boolean(item)); + } + return []; + } + + private readBoolean(value: unknown): boolean { + if (typeof value === "boolean") { + return value; + } + if (typeof value === "number") { + return Number.isFinite(value) && value > 0; + } + if (typeof value !== "string") { + return false; + } + const normalized = value.trim().toLowerCase(); + return normalized === "true" || normalized === "1" || normalized === "yes"; + } + + private asRecord(value: unknown): Record | undefined { + if (!value || typeof value !== "object" || Array.isArray(value)) { + return undefined; + } + return value as Record; + } + + private readString(value: unknown): string | undefined { + if (typeof value !== "string") { + return undefined; + } + const normalized = value.trim(); + return normalized.length > 0 ? normalized : undefined; + } + + private readStringArray(value: unknown): string[] { + if (!Array.isArray(value)) { + return []; + } + return value + .map((item) => this.readString(item)) + .filter((item): item is string => Boolean(item)); + } + + private unique(values: string[]): string[] { + return Array.from(new Set(values)); } private buildSemanticInstructionBlock(contextPack?: RagContextPack): string { diff --git a/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts b/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts index 5087fc4..2159863 100644 --- a/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts +++ b/apps/backend/src/modules/conversation/delivery/delivery-contract.mapper.ts @@ -1,5 +1,6 @@ import { Injectable } from "@nestjs/common"; import type { + ClarificationDecisionEvidence, DeliveryContract, DeliveryEvidenceReplayLog, SqlRun @@ -97,9 +98,25 @@ interface TraceContextEvidence { conflictHint?: ContextConflictHint; } +interface SqlCoverageEvidenceSnapshot { + gateStatus: + | "passed" + | "failed" + | "skipped_no_evidence" + | "skipped_metadata_intent" + | "skipped_no_sql_objects"; + missingObjects: string[]; + triggerSource: "selected_context" | "semantic_context" | "explicit_pinning" | "none"; +} + +type ClarificationDecisionLayer = NonNullable< + NonNullable["clarificationDecision"] +>; + type DeliveryEvidenceWithContext = NonNullable & { effectiveContextSummary?: EffectiveContextSummary; conflictHint?: ContextConflictHint; + sqlCoverage?: SqlCoverageEvidenceSnapshot; }; interface SandboxPostProcessOutcome { @@ -120,6 +137,8 @@ export class DeliveryContractMapper { const fusedSnapshot = this.readRetrievalFusedSnapshot(replayIndex.retrievalFused); const semanticSnapshot = this.readSemanticSnapshot(input.run); const traceContextEvidence = this.readTraceContextEvidence(input.run); + const clarificationDecision = this.readClarificationDecisionEvidence(input.run); + const sqlCoverage = this.readSqlCoverageEvidence(input.run); const invalidInput = replayIndex.invalidPayload; const artifact = this.buildArtifact(input.run); const sandboxOutcome = this.applySandboxPostProcess({ @@ -134,6 +153,7 @@ export class DeliveryContractMapper { ...(traceContextEvidence.conflictHint?.hasConflict ? ["context_conflict_detected"] : []), + ...(sqlCoverage?.gateStatus === "failed" ? ["sql_coverage_gate_failed"] : []), ...sandboxOutcome.riskTags ]); @@ -189,7 +209,9 @@ export class DeliveryContractMapper { skillContextSummary: fusedSnapshot.skillContextSummary, evidenceStale: evidenceStale || undefined, effectiveContextSummary: traceContextEvidence.effectiveContextSummary, - conflictHint: traceContextEvidence.conflictHint + conflictHint: traceContextEvidence.conflictHint, + clarificationDecision, + sqlCoverage }; return { @@ -593,6 +615,224 @@ export class DeliveryContractMapper { }; } + private readClarificationDecisionEvidence( + run: SqlRun + ): ClarificationDecisionLayer | undefined { + const traceWithCompat = run.trace as SqlRun["trace"] & { + clarificationDecision?: unknown; + clarification_decision?: unknown; + }; + const traceDecision = this.readClarificationDecision( + traceWithCompat.clarificationDecision ?? traceWithCompat.clarification_decision + ); + if (traceDecision) { + return traceDecision; + } + + const stepDecision = this.readClarificationDecisionFromSteps(run.trace.steps); + if (stepDecision) { + return stepDecision; + } + + return this.readClarificationDecision(run.clarification); + } + + private readSqlCoverageEvidence(run: SqlRun): SqlCoverageEvidenceSnapshot | undefined { + const steps = run.trace.steps ?? []; + for (const step of [...steps].reverse()) { + if (step.node !== "generate-sql") { + continue; + } + const output = this.parseSummaryObject(step.outputSummary); + if (!output) { + continue; + } + const directCoverage = this.readSqlCoverageFromRecord( + output.coverage ?? output.sqlCoverage ?? output.sql_coverage + ); + if (directCoverage) { + return directCoverage; + } + const errorCode = this.readString(output.errorCode ?? output.error_code); + if (errorCode !== "LLM_SQL_EVIDENCE_COVERAGE_FAILED") { + continue; + } + const errorDetails = this.readRecord(output.errorDetails ?? output.error_details); + const detailsCoverage = this.readSqlCoverageFromRecord( + errorDetails?.coverage ?? errorDetails?.sqlCoverage ?? errorDetails?.sql_coverage + ); + if (detailsCoverage) { + return detailsCoverage; + } + } + return undefined; + } + + private readSqlCoverageFromRecord( + value: unknown + ): SqlCoverageEvidenceSnapshot | undefined { + if (!this.isRecord(value)) { + return undefined; + } + const gateStatusRaw = this.readString(value.gateStatus ?? value.gate_status); + const gateStatus = + gateStatusRaw === "passed" || + gateStatusRaw === "failed" || + gateStatusRaw === "skipped_no_evidence" || + gateStatusRaw === "skipped_metadata_intent" || + gateStatusRaw === "skipped_no_sql_objects" + ? gateStatusRaw + : undefined; + const triggerSourceRaw = this.readString( + value.triggerSource ?? value.trigger_source + ); + const triggerSource = + triggerSourceRaw === "selected_context" || + triggerSourceRaw === "semantic_context" || + triggerSourceRaw === "explicit_pinning" || + triggerSourceRaw === "none" + ? triggerSourceRaw + : undefined; + if (!gateStatus || !triggerSource) { + return undefined; + } + return { + gateStatus, + missingObjects: this.readStringArray( + value.missingObjects ?? value.missing_objects + ), + triggerSource + }; + } + + private readClarificationDecisionFromSteps( + steps: SqlRun["trace"]["steps"] | undefined + ): ClarificationDecisionLayer | undefined { + if (!steps || steps.length === 0) { + return undefined; + } + + for (const step of [...steps].reverse()) { + if (step.node !== "clarify") { + continue; + } + const output = this.parseSummaryObject(step.outputSummary); + if (!output) { + continue; + } + const question = this.readString( + output.clarificationQuestion ?? output.clarification_question + ); + const stepDecision = this.readClarificationDecision( + output.clarificationDecision ?? output.clarification_decision ?? output, + { + question, + reason: this.readString(step.detail) + } + ); + if (stepDecision) { + return stepDecision; + } + } + + return undefined; + } + + private readClarificationDecision( + value: unknown, + fallback?: { + question?: string; + reason?: string; + } + ): ClarificationDecisionLayer | undefined { + if (!this.isRecord(value)) { + return undefined; + } + + const decisionRaw = this.readString(value.decision)?.toLowerCase(); + const decision = + decisionRaw === "continue" || decisionRaw === "clarify" + ? (decisionRaw as ClarificationDecisionEvidence["decision"]) + : undefined; + const triggerPathRaw = this.readString( + value.triggerPath ?? + value.trigger_path ?? + value.triggerSource ?? + value.trigger_source + )?.toLowerCase(); + const triggerPath = + triggerPathRaw === "rule" || + triggerPathRaw === "semantic" || + triggerPathRaw === "hybrid" + ? (triggerPathRaw as ClarificationDecisionEvidence["triggerPath"]) + : undefined; + const decisionSource = this.readString( + value.decisionSource ?? value.decision_source ?? value.source + ); + const bypassed = this.readBoolean(value.bypassed ?? value.isBypassed ?? value.is_bypassed); + const bypassReasonCode = this.readString( + value.bypassReasonCode ?? value.bypass_reason_code + ); + const confidenceLevelRaw = this.readString( + value.confidenceLevel ?? value.confidence_level ?? value.confidence + )?.toLowerCase(); + const confidenceLevel = + confidenceLevelRaw === "high" || + confidenceLevelRaw === "medium" || + confidenceLevelRaw === "low" + ? (confidenceLevelRaw as ClarificationDecisionEvidence["confidenceLevel"]) + : undefined; + const missingCriticalSlots = this.readStringArray( + value.missingCriticalSlots ?? + value.missing_critical_slots ?? + value.missingSlots ?? + value.missing_slots + ); + const conflictDetected = this.readBoolean( + value.conflictDetected ?? + value.conflict_detected ?? + value.hasConflict ?? + value.has_conflict + ); + const reasonCodes = this.readStringArray(value.reasonCodes ?? value.reason_codes); + const question = this.readString( + value.question ?? + value.clarificationQuestion ?? + value.clarification_question ?? + fallback?.question + ); + const reason = this.readString(value.reason ?? fallback?.reason); + const hasStructuredPayload = Boolean( + decision || + triggerPath || + decisionSource || + bypassed !== undefined || + bypassReasonCode || + confidenceLevel || + missingCriticalSlots.length > 0 || + conflictDetected !== undefined || + reasonCodes.length > 0 + ); + + if (!hasStructuredPayload) { + return undefined; + } + + return { + ...(decision ? { decision } : {}), + ...(triggerPath ? { triggerPath } : {}), + ...(decisionSource ? { decisionSource } : {}), + ...(bypassed !== undefined ? { bypassed } : {}), + ...(bypassReasonCode ? { bypassReasonCode } : {}), + ...(confidenceLevel ? { confidenceLevel } : {}), + ...(missingCriticalSlots.length > 0 ? { missingCriticalSlots } : {}), + ...(conflictDetected !== undefined ? { conflictDetected } : {}), + ...(reasonCodes.length > 0 ? { reasonCodes } : {}), + ...(question ? { question } : {}), + ...(reason ? { reason } : {}) + }; + } + private readEffectiveContextSummary(value: unknown): EffectiveContextSummary | undefined { if (!this.isRecord(value)) { return undefined; @@ -767,6 +1007,22 @@ export class DeliveryContractMapper { return normalized ? normalized : undefined; } + private readBoolean(value: unknown): boolean | undefined { + if (typeof value === "boolean") { + return value; + } + if (typeof value === "string") { + const normalized = value.trim().toLowerCase(); + if (normalized === "true") { + return true; + } + if (normalized === "false") { + return false; + } + } + return undefined; + } + private readNonNegativeInt(value: unknown): number { if (typeof value === "number" && Number.isFinite(value) && value >= 0) { return Math.floor(value); diff --git a/apps/backend/src/modules/governance/workspace/workspace-modeling.service.ts b/apps/backend/src/modules/governance/workspace/workspace-modeling.service.ts index 86f0466..fbf5225 100644 --- a/apps/backend/src/modules/governance/workspace/workspace-modeling.service.ts +++ b/apps/backend/src/modules/governance/workspace/workspace-modeling.service.ts @@ -50,6 +50,8 @@ type ModelingRelationshipRecommendation = { confidence: number; reason: "foreign_key_constraint" | "fk_naming_suffix"; isAutoGenerated: true; + type: "many-to-one" | "one-to-many" | "one-to-one"; + cardinality: "many-to-one" | "one-to-many" | "one-to-one"; bridge: { left: { dataset: string; @@ -2088,11 +2090,19 @@ export class WorkspaceModelingService { .map((relationship) => { const source: ModelingGraphPayload["relationships"][number]["source"] = relationship.reason === "foreign_key_constraint" ? "fk" : "inferred"; + const relationshipType = + relationship.type === "many-to-one" || + relationship.type === "one-to-many" || + relationship.type === "one-to-one" + ? relationship.type + : "many-to-one"; return { id: relationship.id, name: relationship.name, source, confidence: Number(relationship.confidence.toFixed(4)), + type: relationshipType, + cardinality: relationshipType, bridge: { left: { dataset: relationship.bridge.left.dataset, @@ -2472,6 +2482,8 @@ ORDER BY kcu.table_name, kcu.column_name, ccu.table_name, ccu.column_name confidence: normalizedConfidence, reason: input.reason, isAutoGenerated: true, + type: "many-to-one", + cardinality: "many-to-one", bridge: { left: { dataset: input.datasourceId, diff --git a/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts b/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts index d17254d..ff58b3b 100644 --- a/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts +++ b/apps/backend/src/modules/knowledge/rag/rerank/rag-rerank.service.ts @@ -430,6 +430,7 @@ export class RagRerankService { rerankedCount: bundle.reranked?.length ?? 0, selectedContextCount: bundle.selected_context?.length ?? 0, riskTags: bundle.risk_tags ?? [], + columnPruning: this.readColumnPruningEvidence(bundle), contextPack: bundle.context_pack ? { status: bundle.context_pack.status, @@ -474,6 +475,21 @@ export class RagRerankService { }); } + private readColumnPruningEvidence( + bundle: RagRetrievalBundle + ): Record | undefined { + const record = bundle as RagRetrievalBundle & { + column_pruning?: unknown; + columnPruning?: unknown; + }; + const candidate = record.column_pruning ?? record.columnPruning; + return this.isRecord(candidate) ? candidate : undefined; + } + + private isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); + } + private normalizePositiveInt(value: number | undefined, fallback: number): number { if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) { return fallback; diff --git a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts index a828d73..e1471c8 100644 --- a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts +++ b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.service.ts @@ -15,6 +15,7 @@ import { ModelingGraphRepository } from "../../../platform/data/persistence/mode import { fuseWithRrf } from "../../../rag/retrieval/fusion/rrf-fusion"; import { RAG_RETRIEVAL_LANES, + type RagPriorSqlLaneEvidence, type RagRetrievalCandidate, type RagRetrievalChunkMetadata, type RagRetrievalChunkPayload, @@ -54,6 +55,55 @@ type LaneExecutionOutput = degradeReason?: string; }; +interface PriorSqlSelectionResult { + lane: RagPriorSqlLaneEvidence; + selectedCandidates: RagRetrievalCandidate[]; + blockedChunkIds: Set; +} + +interface ColumnPruningTableDecision { + table_name: string; + mode: "none" | "light" | "conservative"; + evidence_strength: "weak" | "medium" | "strong"; + total_columns: number; + kept_columns: string[]; + pruned_columns: string[]; + reason_codes: string[]; +} + +interface ColumnPruningEvidence { + strategy: "table_first_field_second_conservative"; + status: "applied" | "skipped"; + query_tokens: string[]; + affected_candidate_count: number; + tables: ColumnPruningTableDecision[]; + reason_codes: string[]; +} + +interface ColumnPruningResult { + candidates: RagRetrievalCandidate[]; + evidence?: ColumnPruningEvidence; +} + +interface WideTableProfile { + tableName: string; + normalizedTableName: string; + columns: string[]; +} + +interface TablePruningPlan { + tableName: string; + normalizedTableName: string; + keepColumns: Set; + decision: ColumnPruningTableDecision; +} + +const WIDE_TABLE_COLUMN_THRESHOLD = 8; +const LIGHT_PRUNING_KEEP_RATIO = 0.7; +const CONSERVATIVE_PRUNING_MAX_KEEP = 6; +const MIN_COLUMN_KEEP_COUNT = 3; +const COLUMN_HINT_PREFIX = "column:"; + @Injectable() export class RagRetrievalService { constructor( @@ -72,6 +122,8 @@ export class RagRetrievalService { const query = input.query.trim(); const datasourceId = input.datasourceId.trim(); const runId = input.runId.trim(); + const workspaceId = input.workspaceId?.trim() || undefined; + const allowedTables = this.normalizeAllowedTables(input.allowedTables); const requestedPerLaneLimit = this.normalizeLimit(input.perLaneLimit, DEFAULT_PER_LANE_LIMIT); const requestedFinalCandidateLimit = this.normalizeLimit( input.finalCandidateLimit, @@ -91,7 +143,7 @@ export class RagRetrievalService { lane_results: this.createEmptyLaneResults(laneTimeoutMs, "invalid_retrieval_input"), candidates: [], context_pack: await this.buildContextPack({ - workspaceId: input.workspaceId, + workspaceId, datasourceId, status: "degraded", degradeReasons @@ -116,7 +168,7 @@ export class RagRetrievalService { lane_results: this.createEmptyLaneResults(laneTimeoutMs, "no_active_index"), candidates: [], context_pack: await this.buildContextPack({ - workspaceId: input.workspaceId, + workspaceId, datasourceId, status: "degraded", degradeReasons @@ -135,7 +187,11 @@ export class RagRetrievalService { }); const perLaneLimit = budgetDecision.perLaneLimit; const finalCandidateLimit = budgetDecision.finalCandidateLimit; - const budgetLaneProfile = budgetDecision.enabledLanes.join("+"); + const budgetLaneProfile = this.buildBudgetLaneProfile({ + enabledLanes: budgetDecision.enabledLanes, + workspaceId, + allowedTables + }); await this.writeBudgetReplay({ runId, datasourceId, @@ -161,7 +217,7 @@ export class RagRetrievalService { cachedBundle: cacheRead.value, query, datasourceId, - workspaceId: input.workspaceId, + workspaceId, runId, decisionReasons: budgetDecision.decisionReasons }); @@ -193,12 +249,31 @@ export class RagRetrievalService { graph: laneResults.graph.hits }; const fused = fuseWithRrf({ laneHits }); - const coveredCandidates = this.applyDomainCoverage( + const priorSqlSelection = this.selectTrustedPriorSqlCandidates({ + candidates: fused, + datasourceId, + workspaceId, + allowedTables + }); + const priorSqlFiltered = this.filterBlockedPriorSqlCandidates( fused, + priorSqlSelection.blockedChunkIds + ); + const fusedWithPrior = this.injectTrustedPriorSqlCandidates( + priorSqlFiltered, + priorSqlSelection.selectedCandidates + ); + const coveredCandidates = this.applyDomainCoverage( + fusedWithPrior, finalCandidateLimit, REQUIRED_DOMAIN_COVERAGE ); - const candidates = this.decorateSemanticCandidates(coveredCandidates); + const decoratedCandidates = this.decorateSemanticCandidates(coveredCandidates); + const columnPruning = this.applyConservativeColumnPruning({ + query, + candidates: decoratedCandidates + }); + const candidates = columnPruning.candidates; const skillContext = await this.resolveSkillContext(query, candidates); const degradeReasons = this.collectDegradeReasons(laneResults); @@ -225,12 +300,15 @@ export class RagRetrievalService { lane_results: laneResults, candidates, skill_context: skillContext, + prior_sql_lane: priorSqlSelection.lane, + priorSqlLane: priorSqlSelection.lane, decision_reasons: budgetDecision.decisionReasons } }; + this.attachColumnPruningEvidence(response.retrieval_bundle, columnPruning.evidence); response.retrieval_bundle.context_pack = await this.buildContextPack({ bundle: response.retrieval_bundle, - workspaceId: input.workspaceId, + workspaceId, datasourceId, status: response.retrieval_bundle.status, degradeReasons: uniqueDegradeReasons @@ -262,11 +340,15 @@ export class RagRetrievalService { runId: string; decisionReasons: string[]; }): Promise { + const priorSqlLane = this.readPriorSqlLaneEvidence(input.cachedBundle); + const columnPruning = this.readColumnPruningEvidence(input.cachedBundle); const hydratedBundle: RagRetrievalResponse["retrieval_bundle"] = { ...input.cachedBundle, query: input.query, datasource_id: input.datasourceId, run_id: input.runId, + prior_sql_lane: priorSqlLane, + priorSqlLane: priorSqlLane, decision_reasons: this.unique([ ...(input.cachedBundle.decision_reasons ?? []), ...input.decisionReasons, @@ -277,6 +359,7 @@ export class RagRetrievalService { ...input.decisionReasons ]) }; + this.attachColumnPruningEvidence(hydratedBundle, columnPruning); hydratedBundle.context_pack = await this.buildContextPack({ bundle: hydratedBundle, workspaceId: input.workspaceId, @@ -601,6 +684,204 @@ export class RagRetrievalService { }; } + private buildBudgetLaneProfile(input: { + enabledLanes: RagRetrievalLane[]; + workspaceId?: string; + allowedTables: string[]; + }): string { + const workspacePart = input.workspaceId?.trim().toLowerCase() || "none"; + const tableDigest = createHash("sha256") + .update(input.allowedTables.join("|")) + .digest("hex") + .slice(0, 12); + return `${input.enabledLanes.join("+")}::ws:${workspacePart}::tables:${tableDigest}`; + } + + private normalizeAllowedTables(raw: string[] | undefined): string[] { + if (!Array.isArray(raw) || raw.length === 0) { + return []; + } + return this.unique( + raw + .map((tableName) => this.readString(tableName)) + .filter((tableName): tableName is string => Boolean(tableName)) + .map((tableName) => tableName.toLowerCase()) + ).sort(); + } + + private selectTrustedPriorSqlCandidates(input: { + candidates: RagRetrievalCandidate[]; + datasourceId: string; + workspaceId?: string; + allowedTables: string[]; + }): PriorSqlSelectionResult { + const trustedSqlExampleCandidates = input.candidates.filter( + (candidate) => + candidate.chunk.metadata.domain === "sql_example" && + this.isTrustedPriorSqlCandidate(candidate) + ); + const selectedCandidates: RagRetrievalCandidate[] = []; + const blockedChunkIds = new Set(); + const degradeReasons: string[] = []; + const allowedTables = new Set(input.allowedTables); + const workspaceId = input.workspaceId?.trim(); + + for (const candidate of trustedSqlExampleCandidates) { + const filterReasons = this.collectPriorSqlFilterReasons({ + candidate, + datasourceId: input.datasourceId, + workspaceId, + allowedTables + }); + if (filterReasons.length > 0) { + blockedChunkIds.add(candidate.chunk_id); + degradeReasons.push(...filterReasons); + continue; + } + selectedCandidates.push({ + ...candidate, + evidence: this.unique([...candidate.evidence, "prior_sql:trusted"]) + }); + } + + if (selectedCandidates.length > 0) { + const lane: RagPriorSqlLaneEvidence = { + status: "hit", + matched_count: trustedSqlExampleCandidates.length, + selected_count: selectedCandidates.length, + filtered_count: blockedChunkIds.size, + ...(degradeReasons.length > 0 + ? { + degrade_reasons: this.unique(degradeReasons) + } + : {}) + }; + return { + lane: this.withPriorSqlLaneCompatFields(lane), + selectedCandidates, + blockedChunkIds + }; + } + + const lane: RagPriorSqlLaneEvidence = { + status: trustedSqlExampleCandidates.length > 0 ? "filtered" : "miss", + matched_count: trustedSqlExampleCandidates.length, + selected_count: 0, + filtered_count: blockedChunkIds.size, + degrade_reasons: + trustedSqlExampleCandidates.length > 0 + ? this.unique(degradeReasons) + : ["prior_sql_no_trusted_match"] + }; + return { + lane: this.withPriorSqlLaneCompatFields(lane), + selectedCandidates: [], + blockedChunkIds + }; + } + + private isTrustedPriorSqlCandidate(candidate: RagRetrievalCandidate): boolean { + const sourceMetadata = candidate.chunk.metadata.sourceMetadata; + if (!this.isRecord(sourceMetadata)) { + return false; + } + return ( + this.readBooleanFlag(sourceMetadata.trusted) || + this.readBooleanFlag(sourceMetadata.verified) || + this.readBooleanFlag(sourceMetadata.priorSql) || + this.readBooleanFlag(sourceMetadata.prior_sql) + ); + } + + private readBooleanFlag(value: unknown): boolean { + if (typeof value === "boolean") { + return value; + } + if (typeof value === "number") { + return Number.isFinite(value) && value > 0; + } + if (typeof value !== "string") { + return false; + } + const normalized = value.trim().toLowerCase(); + return normalized === "true" || normalized === "1" || normalized === "yes"; + } + + private collectPriorSqlFilterReasons(input: { + candidate: RagRetrievalCandidate; + datasourceId: string; + workspaceId?: string; + allowedTables: Set; + }): string[] { + const reasons: string[] = []; + const normalizedDatasourceId = input.datasourceId.trim().toLowerCase(); + if (input.candidate.chunk.metadata.datasourceId.trim().toLowerCase() !== normalizedDatasourceId) { + reasons.push("prior_sql_filtered_datasource_mismatch"); + } + + const sourceMetadata = input.candidate.chunk.metadata.sourceMetadata; + const metadataWorkspaceId = this.readWorkspaceIdFromSourceMetadata(sourceMetadata); + if (metadataWorkspaceId && metadataWorkspaceId !== (input.workspaceId?.trim() || "")) { + reasons.push("prior_sql_filtered_workspace_mismatch"); + } + + if (input.allowedTables.size > 0) { + const tableNames = input.candidate.chunk.metadata.tableNames.map((tableName) => + tableName.trim().toLowerCase() + ); + const isAllowedSubset = tableNames.every((tableName) => input.allowedTables.has(tableName)); + if (!isAllowedSubset) { + reasons.push("prior_sql_filtered_not_in_allowed_tables"); + } + } + return reasons; + } + + private readWorkspaceIdFromSourceMetadata(sourceMetadata: unknown): string | undefined { + if (!this.isRecord(sourceMetadata)) { + return undefined; + } + return this.readString(sourceMetadata.workspaceId) ?? this.readString(sourceMetadata.workspace_id); + } + + private filterBlockedPriorSqlCandidates( + candidates: RagRetrievalCandidate[], + blockedChunkIds: Set + ): RagRetrievalCandidate[] { + if (blockedChunkIds.size === 0) { + return candidates; + } + return candidates.filter((candidate) => !blockedChunkIds.has(candidate.chunk_id)); + } + + private injectTrustedPriorSqlCandidates( + candidates: RagRetrievalCandidate[], + trustedCandidates: RagRetrievalCandidate[] + ): RagRetrievalCandidate[] { + if (trustedCandidates.length === 0) { + return candidates; + } + const trustedByChunkId = new Map( + trustedCandidates.map((candidate) => [candidate.chunk_id, candidate]) + ); + const promoted: RagRetrievalCandidate[] = []; + const seen = new Set(); + for (const candidate of trustedCandidates) { + if (!seen.has(candidate.chunk_id)) { + promoted.push(candidate); + seen.add(candidate.chunk_id); + } + } + for (const candidate of candidates) { + if (seen.has(candidate.chunk_id)) { + continue; + } + promoted.push(trustedByChunkId.get(candidate.chunk_id) ?? candidate); + seen.add(candidate.chunk_id); + } + return promoted; + } + private applyDomainCoverage( candidates: RagRetrievalCandidate[], limit: number, @@ -722,6 +1003,335 @@ export class RagRetrievalService { }); } + private applyConservativeColumnPruning(input: { + query: string; + candidates: RagRetrievalCandidate[]; + }): ColumnPruningResult { + if (input.candidates.length === 0) { + return { + candidates: input.candidates + }; + } + const queryTokens = this.extractTokens(input.query); + const wideTableProfiles = this.collectWideTableProfiles(input.candidates); + if (wideTableProfiles.length === 0) { + return { + candidates: input.candidates + }; + } + + const tablePlans = wideTableProfiles.map((profile) => + this.buildTablePruningPlan({ + profile, + queryTokens, + candidates: input.candidates + }) + ); + const planByTable = new Map( + tablePlans.map((plan) => [plan.normalizedTableName, plan]) + ); + let affectedCandidateCount = 0; + const candidates = input.candidates.map((candidate) => { + const candidateTableNames = candidate.chunk.metadata.tableNames + .map((tableName) => tableName.trim().toLowerCase()) + .filter((tableName) => tableName.length > 0); + const keepColumns = new Set(); + for (const tableName of candidateTableNames) { + const plan = planByTable.get(tableName); + if (!plan) { + continue; + } + for (const columnName of plan.keepColumns) { + keepColumns.add(columnName); + } + } + if (keepColumns.size === 0) { + return candidate; + } + + const originalColumns = candidate.chunk.metadata.columnNames; + if (originalColumns.length === 0) { + return candidate; + } + const nextColumns = originalColumns.filter((columnName) => + keepColumns.has(columnName.trim().toLowerCase()) + ); + if (nextColumns.length === 0 || nextColumns.length === originalColumns.length) { + return candidate; + } + affectedCandidateCount += 1; + return { + ...candidate, + chunk: { + ...candidate.chunk, + metadata: { + ...candidate.chunk.metadata, + columnNames: nextColumns + } + } + }; + }); + + const reasonCodes = this.unique( + tablePlans.flatMap((plan) => plan.decision.reason_codes) + ); + const evidence: ColumnPruningEvidence = { + strategy: "table_first_field_second_conservative", + status: affectedCandidateCount > 0 ? "applied" : "skipped", + query_tokens: queryTokens, + affected_candidate_count: affectedCandidateCount, + tables: tablePlans.map((plan) => plan.decision), + reason_codes: reasonCodes + }; + return { + candidates, + evidence + }; + } + + private collectWideTableProfiles(candidates: RagRetrievalCandidate[]): WideTableProfile[] { + const profileByTable = new Map< + string, + { + tableName: string; + columnOrder: string[]; + seenColumns: Set; + } + >(); + for (const candidate of candidates) { + const tableNames = candidate.chunk.metadata.tableNames; + const columns = candidate.chunk.metadata.columnNames; + if (tableNames.length === 0 || columns.length === 0) { + continue; + } + for (const tableName of tableNames) { + const normalizedTableName = tableName.trim().toLowerCase(); + if (!normalizedTableName) { + continue; + } + const existing = profileByTable.get(normalizedTableName) ?? { + tableName, + columnOrder: [], + seenColumns: new Set() + }; + for (const columnName of columns) { + const normalizedColumnName = columnName.trim().toLowerCase(); + if (!normalizedColumnName || existing.seenColumns.has(normalizedColumnName)) { + continue; + } + existing.seenColumns.add(normalizedColumnName); + existing.columnOrder.push(columnName); + } + profileByTable.set(normalizedTableName, existing); + } + } + return Array.from(profileByTable.entries()) + .filter(([, profile]) => profile.columnOrder.length >= WIDE_TABLE_COLUMN_THRESHOLD) + .map(([normalizedTableName, profile]) => ({ + tableName: profile.tableName, + normalizedTableName, + columns: profile.columnOrder + })); + } + + private buildTablePruningPlan(input: { + profile: WideTableProfile; + queryTokens: string[]; + candidates: RagRetrievalCandidate[]; + }): TablePruningPlan { + const fieldIntentTokens = this.buildFieldIntentTokens( + input.queryTokens, + input.profile.normalizedTableName + ); + const scoreByColumn = new Map(); + const evidenceByColumn = new Map(); + const columnHints = this.collectColumnHintsForTable(input.candidates, input.profile.normalizedTableName); + for (const columnName of input.profile.columns) { + const normalizedColumnName = columnName.trim().toLowerCase(); + const signalReasons: string[] = []; + const tokenScore = this.scoreColumnByQueryTokens(normalizedColumnName, fieldIntentTokens); + if (tokenScore >= 2) { + signalReasons.push("query_token_strong_match"); + } else if (tokenScore > 0) { + signalReasons.push("query_token_partial_match"); + } + const columnHintMatched = columnHints.has(normalizedColumnName) && tokenScore > 0; + if (columnHintMatched) { + signalReasons.push("lane_column_evidence"); + } + const score = tokenScore + (columnHintMatched ? 1 : 0); + scoreByColumn.set(normalizedColumnName, score); + evidenceByColumn.set(normalizedColumnName, signalReasons); + } + + const rankedColumns = [...input.profile.columns].sort((left, right) => { + const leftScore = scoreByColumn.get(left.trim().toLowerCase()) ?? 0; + const rightScore = scoreByColumn.get(right.trim().toLowerCase()) ?? 0; + if (rightScore !== leftScore) { + return rightScore - leftScore; + } + return left.localeCompare(right); + }); + const strongMatches = rankedColumns.filter( + (columnName) => (scoreByColumn.get(columnName.trim().toLowerCase()) ?? 0) >= 2 + ); + const weakMatches = rankedColumns.filter( + (columnName) => + (scoreByColumn.get(columnName.trim().toLowerCase()) ?? 0) === 1 + ); + const anchorColumns = input.profile.columns.filter((columnName) => + this.isAnchorColumn(columnName.trim().toLowerCase()) + ); + const totalColumns = input.profile.columns.length; + + let mode: ColumnPruningTableDecision["mode"] = "none"; + let evidenceStrength: ColumnPruningTableDecision["evidence_strength"] = "weak"; + let reasonCodes: string[] = ["column_pruning_weak_evidence"]; + let keepTarget = totalColumns; + if (strongMatches.length > 0) { + mode = "conservative"; + evidenceStrength = "strong"; + reasonCodes = ["column_pruning_strong_field_evidence"]; + keepTarget = Math.max( + MIN_COLUMN_KEEP_COUNT, + Math.min( + totalColumns, + Math.max(strongMatches.length + anchorColumns.length, CONSERVATIVE_PRUNING_MAX_KEEP) + ) + ); + } else if (weakMatches.length > 0) { + mode = "light"; + evidenceStrength = "medium"; + reasonCodes = ["column_pruning_uncertain_field_evidence"]; + keepTarget = Math.max( + MIN_COLUMN_KEEP_COUNT, + Math.min(totalColumns, Math.ceil(totalColumns * LIGHT_PRUNING_KEEP_RATIO)) + ); + } + + const keepColumns = new Set(); + for (const columnName of [...strongMatches, ...weakMatches, ...anchorColumns, ...rankedColumns]) { + if (keepColumns.size >= keepTarget) { + break; + } + keepColumns.add(columnName.trim().toLowerCase()); + } + if (keepColumns.size === 0) { + for (const columnName of input.profile.columns) { + keepColumns.add(columnName.trim().toLowerCase()); + } + } + + const keptColumns = input.profile.columns.filter((columnName) => + keepColumns.has(columnName.trim().toLowerCase()) + ); + const prunedColumns = + mode === "none" + ? [] + : input.profile.columns.filter( + (columnName) => !keepColumns.has(columnName.trim().toLowerCase()) + ); + const signalReasons = this.unique( + keptColumns.flatMap((columnName) => + evidenceByColumn.get(columnName.trim().toLowerCase()) ?? [] + ) + ); + if (mode !== "none" && signalReasons.length > 0) { + reasonCodes = this.unique([...reasonCodes, ...signalReasons]); + } + + return { + tableName: input.profile.tableName, + normalizedTableName: input.profile.normalizedTableName, + keepColumns, + decision: { + table_name: input.profile.tableName, + mode, + evidence_strength: evidenceStrength, + total_columns: totalColumns, + kept_columns: keptColumns, + pruned_columns: prunedColumns, + reason_codes: reasonCodes + } + }; + } + + private buildFieldIntentTokens( + queryTokens: string[], + normalizedTableName: string + ): string[] { + const tableTokens = normalizedTableName + .split(/[_\s]+/g) + .map((part) => part.trim()) + .filter((part) => part.length > 0); + const blocked = new Set([normalizedTableName, ...tableTokens]); + return queryTokens.filter((token) => !blocked.has(token)); + } + + private collectColumnHintsForTable( + candidates: RagRetrievalCandidate[], + normalizedTableName: string + ): Set { + const hints = new Set(); + for (const candidate of candidates) { + const belongsToTable = candidate.chunk.metadata.tableNames.some( + (tableName) => tableName.trim().toLowerCase() === normalizedTableName + ); + if (!belongsToTable) { + continue; + } + for (const evidence of candidate.evidence) { + if (!evidence.startsWith(COLUMN_HINT_PREFIX)) { + continue; + } + const rawColumnName = evidence.slice(COLUMN_HINT_PREFIX.length).trim().toLowerCase(); + if (rawColumnName) { + hints.add(rawColumnName); + } + } + } + return hints; + } + + private scoreColumnByQueryTokens( + normalizedColumnName: string, + queryTokens: string[] + ): number { + if (queryTokens.length === 0) { + return 0; + } + const columnParts = normalizedColumnName.split(/[_\s]+/g).filter((part) => part.length > 0); + const substantialParts = columnParts.filter((part) => part.length >= 3); + let score = 0; + for (const token of queryTokens) { + if (token === normalizedColumnName || columnParts.includes(token)) { + score += 2; + continue; + } + if ( + token.length >= 3 && + (normalizedColumnName.includes(token) || + substantialParts.some((part) => part.includes(token) || token.includes(part))) + ) { + score += 1; + } + } + return score; + } + + private isAnchorColumn(normalizedColumnName: string): boolean { + if ( + normalizedColumnName === "id" || + normalizedColumnName.endsWith("_id") || + normalizedColumnName === "created_at" || + normalizedColumnName === "updated_at" || + normalizedColumnName === "deleted_at" + ) { + return true; + } + return normalizedColumnName.endsWith("_at"); + } + private resolveLaneTimeoutMs( input: RagRetrievalRequest ): Record { @@ -826,6 +1436,60 @@ export class RagRetrievalService { return Array.from(new Set(values)); } + private attachColumnPruningEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"], + evidence: ColumnPruningEvidence | undefined + ): void { + const target = bundle as RagRetrievalResponse["retrieval_bundle"] & { + column_pruning?: ColumnPruningEvidence; + columnPruning?: ColumnPruningEvidence; + }; + if (!evidence) { + delete target.column_pruning; + delete target.columnPruning; + return; + } + target.column_pruning = evidence; + target.columnPruning = evidence; + } + + private readColumnPruningEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"] + ): ColumnPruningEvidence | undefined { + const source = bundle as RagRetrievalResponse["retrieval_bundle"] & { + column_pruning?: ColumnPruningEvidence; + columnPruning?: ColumnPruningEvidence; + }; + return source.column_pruning ?? source.columnPruning; + } + + private readPriorSqlLaneEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"] + ): RagPriorSqlLaneEvidence | undefined { + const priorSqlLane = bundle.prior_sql_lane ?? bundle.priorSqlLane; + if (!priorSqlLane) { + return undefined; + } + return this.withPriorSqlLaneCompatFields(priorSqlLane); + } + + private withPriorSqlLaneCompatFields( + lane: RagPriorSqlLaneEvidence + ): RagPriorSqlLaneEvidence { + const degradeReasons = lane.degrade_reasons ?? lane.degradeReasons; + return { + ...lane, + matched_count: lane.matched_count, + selected_count: lane.selected_count, + filtered_count: lane.filtered_count, + ...(degradeReasons ? { degrade_reasons: degradeReasons } : {}), + matchedCount: lane.matched_count, + selectedCount: lane.selected_count, + filteredCount: lane.filtered_count, + ...(degradeReasons ? { degradeReasons } : {}) + }; + } + private async buildContextPack(input: { bundle?: RagRetrievalResponse["retrieval_bundle"]; workspaceId?: string; @@ -1076,6 +1740,8 @@ export class RagRetrievalService { degradeReasons: bundle.degrade_reasons, decisionReasons: bundle.decision_reasons ?? [], candidateCount: bundle.candidates.length, + priorSqlLane: this.readPriorSqlLaneEvidence(bundle), + columnPruning: this.readColumnPruningEvidence(bundle), skillContext: bundle.skill_context, candidates: bundle.candidates.map((candidate) => ({ chunkId: candidate.chunk_id, diff --git a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts index 2f05276..372c7de 100644 --- a/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts +++ b/apps/backend/src/modules/knowledge/rag/retrieval/rag-retrieval.types.ts @@ -14,6 +14,7 @@ export interface RagRetrievalRequest { datasourceId: string; runId: string; workspaceId?: string; + allowedTables?: string[]; activeIndexVersionId?: string; perLaneLimit?: number; finalCandidateLimit?: number; @@ -140,6 +141,18 @@ export interface RagContextPack { riskTags?: string[]; } +export interface RagPriorSqlLaneEvidence { + status: "hit" | "miss" | "filtered"; + matched_count: number; + selected_count: number; + filtered_count: number; + degrade_reasons?: string[]; + matchedCount?: number; + selectedCount?: number; + filteredCount?: number; + degradeReasons?: string[]; +} + export interface RagRetrievalBundle { query: string; run_id: string; @@ -154,6 +167,8 @@ export interface RagRetrievalBundle { risk_tags?: string[]; skill_context?: RagSkillContext; context_pack?: RagContextPack; + prior_sql_lane?: RagPriorSqlLaneEvidence; + priorSqlLane?: RagPriorSqlLaneEvidence; decision_reasons?: string[]; } diff --git a/apps/backend/src/modules/rag/quality/rag-quality.service.ts b/apps/backend/src/modules/rag/quality/rag-quality.service.ts index 44724fc..989dbd0 100644 --- a/apps/backend/src/modules/rag/quality/rag-quality.service.ts +++ b/apps/backend/src/modules/rag/quality/rag-quality.service.ts @@ -19,11 +19,27 @@ export interface RagQualityEvaluationInput { mrrAt10: number; retrievalRerankP95Ms: number; degradeRate: number; + priorSqlLane?: RagPriorSqlLaneMetricsInput; recordedAt?: string; } +export interface RagPriorSqlLaneMetricsInput { + totalCount: number; + hitCount: number; + filteredCount?: number; + staleCount?: number; +} + interface RagQualityEvaluationRecord extends RagQualityEvaluationInput { recordedAt: string; + priorSqlLane?: RagPriorSqlLaneMetricsRecord; +} + +interface RagPriorSqlLaneMetricsRecord { + totalCount: number; + hitCount: number; + filteredCount: number; + staleCount: number; } export interface RagDatasourceOrchestrationSample { @@ -108,6 +124,7 @@ export interface RagQualityGateReport { glossarySelectedContext: GlossarySelectedContextGateReport; datasourceOrchestration: RagDatasourceOrchestrationReport; cacheBudget: RagCacheBudgetReport; + priorSqlLane: RagPriorSqlLaneGateReport; latest?: { runId: string; datasourceId: string; @@ -125,6 +142,13 @@ export interface RagQualityGateReport { generatedAt: string; } +export interface RagPriorSqlLaneGateReport { + sampleSize: number; + priorSqlHitRate: number; + priorSqlFilteredCount: number; + priorSqlStaleRate: number; +} + interface GlossarySelectedContextSampleRow { sampleId: string; query: string; @@ -305,6 +329,7 @@ export class RagQualityService { mrrAt10: this.normalizeRatio(input.mrrAt10), retrievalRerankP95Ms: Math.max(0, input.retrievalRerankP95Ms), degradeRate: this.normalizeRatio(input.degradeRate), + priorSqlLane: this.normalizePriorSqlLaneMetrics(input.priorSqlLane), recordedAt: this.normalizeIsoTimestamp(input.recordedAt) }); } @@ -378,6 +403,7 @@ export class RagQualityService { glossarySelectedContext, datasourceOrchestration: this.snapshotDatasourceOrchestration(), cacheBudget: this.snapshotCacheBudget(), + priorSqlLane: this.snapshotPriorSqlLane(), latest: latest ? { runId: latest.runId, @@ -596,6 +622,31 @@ export class RagQualityService { return Math.max(0, Math.min(1, Number(input.toFixed(6)))); } + private normalizeCount(input: number | undefined): number { + if (input === undefined || !Number.isFinite(input)) { + return 0; + } + return Math.max(0, Math.floor(input)); + } + + private normalizePriorSqlLaneMetrics( + input: RagPriorSqlLaneMetricsInput | undefined + ): RagPriorSqlLaneMetricsRecord | undefined { + if (!input) { + return undefined; + } + const totalCount = this.normalizeCount(input.totalCount); + const hitCount = Math.min(totalCount, this.normalizeCount(input.hitCount)); + const filteredCount = this.normalizeCount(input.filteredCount); + const staleCount = Math.min(totalCount, this.normalizeCount(input.staleCount)); + return { + totalCount, + hitCount, + filteredCount, + staleCount + }; + } + private snapshotDatasourceOrchestration(): RagDatasourceOrchestrationReport { const now = Date.now(); const last24h = this.datasourceRecords.filter( @@ -630,6 +681,40 @@ export class RagQualityService { }; } + private snapshotPriorSqlLane(): RagPriorSqlLaneGateReport { + const summary = this.records.reduce( + (accumulator, record) => { + if (!record.priorSqlLane) { + return accumulator; + } + return { + totalCount: accumulator.totalCount + record.priorSqlLane.totalCount, + hitCount: accumulator.hitCount + record.priorSqlLane.hitCount, + filteredCount: accumulator.filteredCount + record.priorSqlLane.filteredCount, + staleCount: accumulator.staleCount + record.priorSqlLane.staleCount + }; + }, + { + totalCount: 0, + hitCount: 0, + filteredCount: 0, + staleCount: 0 + } + ); + return { + sampleSize: summary.totalCount, + priorSqlHitRate: + summary.totalCount === 0 + ? 0 + : Number((summary.hitCount / summary.totalCount).toFixed(6)), + priorSqlFilteredCount: summary.filteredCount, + priorSqlStaleRate: + summary.totalCount === 0 + ? 0 + : Number((summary.staleCount / summary.totalCount).toFixed(6)) + }; + } + private percentile(values: number[], quantile: number): number { if (values.length === 0) { return 0; diff --git a/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts b/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts index ea8002d..5724204 100644 --- a/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts +++ b/apps/backend/src/modules/rag/rerank/rag-rerank.service.ts @@ -429,6 +429,7 @@ export class RagRerankService { rerankedCount: bundle.reranked?.length ?? 0, selectedContextCount: bundle.selected_context?.length ?? 0, riskTags: bundle.risk_tags ?? [], + columnPruning: this.readColumnPruningEvidence(bundle), contextPack: bundle.context_pack ? { status: bundle.context_pack.status, @@ -473,6 +474,21 @@ export class RagRerankService { }); } + private readColumnPruningEvidence( + bundle: RagRetrievalBundle + ): Record | undefined { + const record = bundle as RagRetrievalBundle & { + column_pruning?: unknown; + columnPruning?: unknown; + }; + const candidate = record.column_pruning ?? record.columnPruning; + return this.isRecord(candidate) ? candidate : undefined; + } + + private isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); + } + private normalizePositiveInt(value: number | undefined, fallback: number): number { if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) { return fallback; diff --git a/apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts b/apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts index d8385ee..4aa5119 100644 --- a/apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts +++ b/apps/backend/src/modules/rag/retrieval/rag-retrieval.service.ts @@ -15,6 +15,7 @@ import { ModelingGraphRepository } from "../../platform/data/persistence/modelin import { fuseWithRrf } from "./fusion/rrf-fusion"; import { RAG_RETRIEVAL_LANES, + type RagPriorSqlLaneEvidence, type RagRetrievalCandidate, type RagRetrievalChunkMetadata, type RagRetrievalChunkPayload, @@ -53,6 +54,55 @@ type LaneExecutionOutput = degradeReason?: string; }; +interface PriorSqlSelectionResult { + lane: RagPriorSqlLaneEvidence; + selectedCandidates: RagRetrievalCandidate[]; + blockedChunkIds: Set; +} + +interface ColumnPruningTableDecision { + table_name: string; + mode: "none" | "light" | "conservative"; + evidence_strength: "weak" | "medium" | "strong"; + total_columns: number; + kept_columns: string[]; + pruned_columns: string[]; + reason_codes: string[]; +} + +interface ColumnPruningEvidence { + strategy: "table_first_field_second_conservative"; + status: "applied" | "skipped"; + query_tokens: string[]; + affected_candidate_count: number; + tables: ColumnPruningTableDecision[]; + reason_codes: string[]; +} + +interface ColumnPruningResult { + candidates: RagRetrievalCandidate[]; + evidence?: ColumnPruningEvidence; +} + +interface WideTableProfile { + tableName: string; + normalizedTableName: string; + columns: string[]; +} + +interface TablePruningPlan { + tableName: string; + normalizedTableName: string; + keepColumns: Set; + decision: ColumnPruningTableDecision; +} + +const WIDE_TABLE_COLUMN_THRESHOLD = 8; +const LIGHT_PRUNING_KEEP_RATIO = 0.7; +const CONSERVATIVE_PRUNING_MAX_KEEP = 6; +const MIN_COLUMN_KEEP_COUNT = 3; +const COLUMN_HINT_PREFIX = "column:"; + @Injectable() export class RagRetrievalService { constructor( @@ -71,6 +121,8 @@ export class RagRetrievalService { const query = input.query.trim(); const datasourceId = input.datasourceId.trim(); const runId = input.runId.trim(); + const workspaceId = input.workspaceId?.trim() || undefined; + const allowedTables = this.normalizeAllowedTables(input.allowedTables); const requestedPerLaneLimit = this.normalizeLimit(input.perLaneLimit, DEFAULT_PER_LANE_LIMIT); const requestedFinalCandidateLimit = this.normalizeLimit( input.finalCandidateLimit, @@ -90,7 +142,7 @@ export class RagRetrievalService { lane_results: this.createEmptyLaneResults(laneTimeoutMs, "invalid_retrieval_input"), candidates: [], context_pack: await this.buildContextPack({ - workspaceId: input.workspaceId, + workspaceId, datasourceId, status: "degraded", degradeReasons @@ -115,7 +167,7 @@ export class RagRetrievalService { lane_results: this.createEmptyLaneResults(laneTimeoutMs, "no_active_index"), candidates: [], context_pack: await this.buildContextPack({ - workspaceId: input.workspaceId, + workspaceId, datasourceId, status: "degraded", degradeReasons @@ -134,7 +186,11 @@ export class RagRetrievalService { }); const perLaneLimit = budgetDecision.perLaneLimit; const finalCandidateLimit = budgetDecision.finalCandidateLimit; - const budgetLaneProfile = budgetDecision.enabledLanes.join("+"); + const budgetLaneProfile = this.buildBudgetLaneProfile({ + enabledLanes: budgetDecision.enabledLanes, + workspaceId, + allowedTables + }); await this.writeBudgetReplay({ runId, datasourceId, @@ -160,7 +216,7 @@ export class RagRetrievalService { cachedBundle: cacheRead.value, query, datasourceId, - workspaceId: input.workspaceId, + workspaceId, runId, decisionReasons: budgetDecision.decisionReasons }); @@ -192,12 +248,31 @@ export class RagRetrievalService { graph: laneResults.graph.hits }; const fused = fuseWithRrf({ laneHits }); - const coveredCandidates = this.applyDomainCoverage( + const priorSqlSelection = this.selectTrustedPriorSqlCandidates({ + candidates: fused, + datasourceId, + workspaceId, + allowedTables + }); + const priorSqlFiltered = this.filterBlockedPriorSqlCandidates( fused, + priorSqlSelection.blockedChunkIds + ); + const fusedWithPrior = this.injectTrustedPriorSqlCandidates( + priorSqlFiltered, + priorSqlSelection.selectedCandidates + ); + const coveredCandidates = this.applyDomainCoverage( + fusedWithPrior, finalCandidateLimit, REQUIRED_DOMAIN_COVERAGE ); - const candidates = this.decorateSemanticCandidates(coveredCandidates); + const decoratedCandidates = this.decorateSemanticCandidates(coveredCandidates); + const columnPruning = this.applyConservativeColumnPruning({ + query, + candidates: decoratedCandidates + }); + const candidates = columnPruning.candidates; const skillContext = await this.resolveSkillContext(query, candidates); const degradeReasons = this.collectDegradeReasons(laneResults); @@ -224,12 +299,15 @@ export class RagRetrievalService { lane_results: laneResults, candidates, skill_context: skillContext, + prior_sql_lane: priorSqlSelection.lane, + priorSqlLane: priorSqlSelection.lane, decision_reasons: budgetDecision.decisionReasons } }; + this.attachColumnPruningEvidence(response.retrieval_bundle, columnPruning.evidence); response.retrieval_bundle.context_pack = await this.buildContextPack({ bundle: response.retrieval_bundle, - workspaceId: input.workspaceId, + workspaceId, datasourceId, status: response.retrieval_bundle.status, degradeReasons: uniqueDegradeReasons @@ -261,11 +339,15 @@ export class RagRetrievalService { runId: string; decisionReasons: string[]; }): Promise { + const priorSqlLane = this.readPriorSqlLaneEvidence(input.cachedBundle); + const columnPruning = this.readColumnPruningEvidence(input.cachedBundle); const hydratedBundle: RagRetrievalResponse["retrieval_bundle"] = { ...input.cachedBundle, query: input.query, datasource_id: input.datasourceId, run_id: input.runId, + prior_sql_lane: priorSqlLane, + priorSqlLane: priorSqlLane, decision_reasons: this.unique([ ...(input.cachedBundle.decision_reasons ?? []), ...input.decisionReasons, @@ -276,6 +358,7 @@ export class RagRetrievalService { ...input.decisionReasons ]) }; + this.attachColumnPruningEvidence(hydratedBundle, columnPruning); hydratedBundle.context_pack = await this.buildContextPack({ bundle: hydratedBundle, workspaceId: input.workspaceId, @@ -600,6 +683,204 @@ export class RagRetrievalService { }; } + private buildBudgetLaneProfile(input: { + enabledLanes: RagRetrievalLane[]; + workspaceId?: string; + allowedTables: string[]; + }): string { + const workspacePart = input.workspaceId?.trim().toLowerCase() || "none"; + const tableDigest = createHash("sha256") + .update(input.allowedTables.join("|")) + .digest("hex") + .slice(0, 12); + return `${input.enabledLanes.join("+")}::ws:${workspacePart}::tables:${tableDigest}`; + } + + private normalizeAllowedTables(raw: string[] | undefined): string[] { + if (!Array.isArray(raw) || raw.length === 0) { + return []; + } + return this.unique( + raw + .map((tableName) => this.readString(tableName)) + .filter((tableName): tableName is string => Boolean(tableName)) + .map((tableName) => tableName.toLowerCase()) + ).sort(); + } + + private selectTrustedPriorSqlCandidates(input: { + candidates: RagRetrievalCandidate[]; + datasourceId: string; + workspaceId?: string; + allowedTables: string[]; + }): PriorSqlSelectionResult { + const trustedSqlExampleCandidates = input.candidates.filter( + (candidate) => + candidate.chunk.metadata.domain === "sql_example" && + this.isTrustedPriorSqlCandidate(candidate) + ); + const selectedCandidates: RagRetrievalCandidate[] = []; + const blockedChunkIds = new Set(); + const degradeReasons: string[] = []; + const allowedTables = new Set(input.allowedTables); + const workspaceId = input.workspaceId?.trim(); + + for (const candidate of trustedSqlExampleCandidates) { + const filterReasons = this.collectPriorSqlFilterReasons({ + candidate, + datasourceId: input.datasourceId, + workspaceId, + allowedTables + }); + if (filterReasons.length > 0) { + blockedChunkIds.add(candidate.chunk_id); + degradeReasons.push(...filterReasons); + continue; + } + selectedCandidates.push({ + ...candidate, + evidence: this.unique([...candidate.evidence, "prior_sql:trusted"]) + }); + } + + if (selectedCandidates.length > 0) { + const lane: RagPriorSqlLaneEvidence = { + status: "hit", + matched_count: trustedSqlExampleCandidates.length, + selected_count: selectedCandidates.length, + filtered_count: blockedChunkIds.size, + ...(degradeReasons.length > 0 + ? { + degrade_reasons: this.unique(degradeReasons) + } + : {}) + }; + return { + lane: this.withPriorSqlLaneCompatFields(lane), + selectedCandidates, + blockedChunkIds + }; + } + + const lane: RagPriorSqlLaneEvidence = { + status: trustedSqlExampleCandidates.length > 0 ? "filtered" : "miss", + matched_count: trustedSqlExampleCandidates.length, + selected_count: 0, + filtered_count: blockedChunkIds.size, + degrade_reasons: + trustedSqlExampleCandidates.length > 0 + ? this.unique(degradeReasons) + : ["prior_sql_no_trusted_match"] + }; + return { + lane: this.withPriorSqlLaneCompatFields(lane), + selectedCandidates: [], + blockedChunkIds + }; + } + + private isTrustedPriorSqlCandidate(candidate: RagRetrievalCandidate): boolean { + const sourceMetadata = candidate.chunk.metadata.sourceMetadata; + if (!this.isRecord(sourceMetadata)) { + return false; + } + return ( + this.readBooleanFlag(sourceMetadata.trusted) || + this.readBooleanFlag(sourceMetadata.verified) || + this.readBooleanFlag(sourceMetadata.priorSql) || + this.readBooleanFlag(sourceMetadata.prior_sql) + ); + } + + private readBooleanFlag(value: unknown): boolean { + if (typeof value === "boolean") { + return value; + } + if (typeof value === "number") { + return Number.isFinite(value) && value > 0; + } + if (typeof value !== "string") { + return false; + } + const normalized = value.trim().toLowerCase(); + return normalized === "true" || normalized === "1" || normalized === "yes"; + } + + private collectPriorSqlFilterReasons(input: { + candidate: RagRetrievalCandidate; + datasourceId: string; + workspaceId?: string; + allowedTables: Set; + }): string[] { + const reasons: string[] = []; + const normalizedDatasourceId = input.datasourceId.trim().toLowerCase(); + if (input.candidate.chunk.metadata.datasourceId.trim().toLowerCase() !== normalizedDatasourceId) { + reasons.push("prior_sql_filtered_datasource_mismatch"); + } + + const sourceMetadata = input.candidate.chunk.metadata.sourceMetadata; + const metadataWorkspaceId = this.readWorkspaceIdFromSourceMetadata(sourceMetadata); + if (metadataWorkspaceId && metadataWorkspaceId !== (input.workspaceId?.trim() || "")) { + reasons.push("prior_sql_filtered_workspace_mismatch"); + } + + if (input.allowedTables.size > 0) { + const tableNames = input.candidate.chunk.metadata.tableNames.map((tableName) => + tableName.trim().toLowerCase() + ); + const isAllowedSubset = tableNames.every((tableName) => input.allowedTables.has(tableName)); + if (!isAllowedSubset) { + reasons.push("prior_sql_filtered_not_in_allowed_tables"); + } + } + return reasons; + } + + private readWorkspaceIdFromSourceMetadata(sourceMetadata: unknown): string | undefined { + if (!this.isRecord(sourceMetadata)) { + return undefined; + } + return this.readString(sourceMetadata.workspaceId) ?? this.readString(sourceMetadata.workspace_id); + } + + private filterBlockedPriorSqlCandidates( + candidates: RagRetrievalCandidate[], + blockedChunkIds: Set + ): RagRetrievalCandidate[] { + if (blockedChunkIds.size === 0) { + return candidates; + } + return candidates.filter((candidate) => !blockedChunkIds.has(candidate.chunk_id)); + } + + private injectTrustedPriorSqlCandidates( + candidates: RagRetrievalCandidate[], + trustedCandidates: RagRetrievalCandidate[] + ): RagRetrievalCandidate[] { + if (trustedCandidates.length === 0) { + return candidates; + } + const trustedByChunkId = new Map( + trustedCandidates.map((candidate) => [candidate.chunk_id, candidate]) + ); + const promoted: RagRetrievalCandidate[] = []; + const seen = new Set(); + for (const candidate of trustedCandidates) { + if (!seen.has(candidate.chunk_id)) { + promoted.push(candidate); + seen.add(candidate.chunk_id); + } + } + for (const candidate of candidates) { + if (seen.has(candidate.chunk_id)) { + continue; + } + promoted.push(trustedByChunkId.get(candidate.chunk_id) ?? candidate); + seen.add(candidate.chunk_id); + } + return promoted; + } + private applyDomainCoverage( candidates: RagRetrievalCandidate[], limit: number, @@ -721,6 +1002,335 @@ export class RagRetrievalService { }); } + private applyConservativeColumnPruning(input: { + query: string; + candidates: RagRetrievalCandidate[]; + }): ColumnPruningResult { + if (input.candidates.length === 0) { + return { + candidates: input.candidates + }; + } + const queryTokens = this.extractTokens(input.query); + const wideTableProfiles = this.collectWideTableProfiles(input.candidates); + if (wideTableProfiles.length === 0) { + return { + candidates: input.candidates + }; + } + + const tablePlans = wideTableProfiles.map((profile) => + this.buildTablePruningPlan({ + profile, + queryTokens, + candidates: input.candidates + }) + ); + const planByTable = new Map( + tablePlans.map((plan) => [plan.normalizedTableName, plan]) + ); + let affectedCandidateCount = 0; + const candidates = input.candidates.map((candidate) => { + const candidateTableNames = candidate.chunk.metadata.tableNames + .map((tableName) => tableName.trim().toLowerCase()) + .filter((tableName) => tableName.length > 0); + const keepColumns = new Set(); + for (const tableName of candidateTableNames) { + const plan = planByTable.get(tableName); + if (!plan) { + continue; + } + for (const columnName of plan.keepColumns) { + keepColumns.add(columnName); + } + } + if (keepColumns.size === 0) { + return candidate; + } + + const originalColumns = candidate.chunk.metadata.columnNames; + if (originalColumns.length === 0) { + return candidate; + } + const nextColumns = originalColumns.filter((columnName) => + keepColumns.has(columnName.trim().toLowerCase()) + ); + if (nextColumns.length === 0 || nextColumns.length === originalColumns.length) { + return candidate; + } + affectedCandidateCount += 1; + return { + ...candidate, + chunk: { + ...candidate.chunk, + metadata: { + ...candidate.chunk.metadata, + columnNames: nextColumns + } + } + }; + }); + + const reasonCodes = this.unique( + tablePlans.flatMap((plan) => plan.decision.reason_codes) + ); + const evidence: ColumnPruningEvidence = { + strategy: "table_first_field_second_conservative", + status: affectedCandidateCount > 0 ? "applied" : "skipped", + query_tokens: queryTokens, + affected_candidate_count: affectedCandidateCount, + tables: tablePlans.map((plan) => plan.decision), + reason_codes: reasonCodes + }; + return { + candidates, + evidence + }; + } + + private collectWideTableProfiles(candidates: RagRetrievalCandidate[]): WideTableProfile[] { + const profileByTable = new Map< + string, + { + tableName: string; + columnOrder: string[]; + seenColumns: Set; + } + >(); + for (const candidate of candidates) { + const tableNames = candidate.chunk.metadata.tableNames; + const columns = candidate.chunk.metadata.columnNames; + if (tableNames.length === 0 || columns.length === 0) { + continue; + } + for (const tableName of tableNames) { + const normalizedTableName = tableName.trim().toLowerCase(); + if (!normalizedTableName) { + continue; + } + const existing = profileByTable.get(normalizedTableName) ?? { + tableName, + columnOrder: [], + seenColumns: new Set() + }; + for (const columnName of columns) { + const normalizedColumnName = columnName.trim().toLowerCase(); + if (!normalizedColumnName || existing.seenColumns.has(normalizedColumnName)) { + continue; + } + existing.seenColumns.add(normalizedColumnName); + existing.columnOrder.push(columnName); + } + profileByTable.set(normalizedTableName, existing); + } + } + return Array.from(profileByTable.entries()) + .filter(([, profile]) => profile.columnOrder.length >= WIDE_TABLE_COLUMN_THRESHOLD) + .map(([normalizedTableName, profile]) => ({ + tableName: profile.tableName, + normalizedTableName, + columns: profile.columnOrder + })); + } + + private buildTablePruningPlan(input: { + profile: WideTableProfile; + queryTokens: string[]; + candidates: RagRetrievalCandidate[]; + }): TablePruningPlan { + const fieldIntentTokens = this.buildFieldIntentTokens( + input.queryTokens, + input.profile.normalizedTableName + ); + const scoreByColumn = new Map(); + const evidenceByColumn = new Map(); + const columnHints = this.collectColumnHintsForTable(input.candidates, input.profile.normalizedTableName); + for (const columnName of input.profile.columns) { + const normalizedColumnName = columnName.trim().toLowerCase(); + const signalReasons: string[] = []; + const tokenScore = this.scoreColumnByQueryTokens(normalizedColumnName, fieldIntentTokens); + if (tokenScore >= 2) { + signalReasons.push("query_token_strong_match"); + } else if (tokenScore > 0) { + signalReasons.push("query_token_partial_match"); + } + const columnHintMatched = columnHints.has(normalizedColumnName) && tokenScore > 0; + if (columnHintMatched) { + signalReasons.push("lane_column_evidence"); + } + const score = tokenScore + (columnHintMatched ? 1 : 0); + scoreByColumn.set(normalizedColumnName, score); + evidenceByColumn.set(normalizedColumnName, signalReasons); + } + + const rankedColumns = [...input.profile.columns].sort((left, right) => { + const leftScore = scoreByColumn.get(left.trim().toLowerCase()) ?? 0; + const rightScore = scoreByColumn.get(right.trim().toLowerCase()) ?? 0; + if (rightScore !== leftScore) { + return rightScore - leftScore; + } + return left.localeCompare(right); + }); + const strongMatches = rankedColumns.filter( + (columnName) => (scoreByColumn.get(columnName.trim().toLowerCase()) ?? 0) >= 2 + ); + const weakMatches = rankedColumns.filter( + (columnName) => + (scoreByColumn.get(columnName.trim().toLowerCase()) ?? 0) === 1 + ); + const anchorColumns = input.profile.columns.filter((columnName) => + this.isAnchorColumn(columnName.trim().toLowerCase()) + ); + const totalColumns = input.profile.columns.length; + + let mode: ColumnPruningTableDecision["mode"] = "none"; + let evidenceStrength: ColumnPruningTableDecision["evidence_strength"] = "weak"; + let reasonCodes: string[] = ["column_pruning_weak_evidence"]; + let keepTarget = totalColumns; + if (strongMatches.length > 0) { + mode = "conservative"; + evidenceStrength = "strong"; + reasonCodes = ["column_pruning_strong_field_evidence"]; + keepTarget = Math.max( + MIN_COLUMN_KEEP_COUNT, + Math.min( + totalColumns, + Math.max(strongMatches.length + anchorColumns.length, CONSERVATIVE_PRUNING_MAX_KEEP) + ) + ); + } else if (weakMatches.length > 0) { + mode = "light"; + evidenceStrength = "medium"; + reasonCodes = ["column_pruning_uncertain_field_evidence"]; + keepTarget = Math.max( + MIN_COLUMN_KEEP_COUNT, + Math.min(totalColumns, Math.ceil(totalColumns * LIGHT_PRUNING_KEEP_RATIO)) + ); + } + + const keepColumns = new Set(); + for (const columnName of [...strongMatches, ...weakMatches, ...anchorColumns, ...rankedColumns]) { + if (keepColumns.size >= keepTarget) { + break; + } + keepColumns.add(columnName.trim().toLowerCase()); + } + if (keepColumns.size === 0) { + for (const columnName of input.profile.columns) { + keepColumns.add(columnName.trim().toLowerCase()); + } + } + + const keptColumns = input.profile.columns.filter((columnName) => + keepColumns.has(columnName.trim().toLowerCase()) + ); + const prunedColumns = + mode === "none" + ? [] + : input.profile.columns.filter( + (columnName) => !keepColumns.has(columnName.trim().toLowerCase()) + ); + const signalReasons = this.unique( + keptColumns.flatMap((columnName) => + evidenceByColumn.get(columnName.trim().toLowerCase()) ?? [] + ) + ); + if (mode !== "none" && signalReasons.length > 0) { + reasonCodes = this.unique([...reasonCodes, ...signalReasons]); + } + + return { + tableName: input.profile.tableName, + normalizedTableName: input.profile.normalizedTableName, + keepColumns, + decision: { + table_name: input.profile.tableName, + mode, + evidence_strength: evidenceStrength, + total_columns: totalColumns, + kept_columns: keptColumns, + pruned_columns: prunedColumns, + reason_codes: reasonCodes + } + }; + } + + private buildFieldIntentTokens( + queryTokens: string[], + normalizedTableName: string + ): string[] { + const tableTokens = normalizedTableName + .split(/[_\s]+/g) + .map((part) => part.trim()) + .filter((part) => part.length > 0); + const blocked = new Set([normalizedTableName, ...tableTokens]); + return queryTokens.filter((token) => !blocked.has(token)); + } + + private collectColumnHintsForTable( + candidates: RagRetrievalCandidate[], + normalizedTableName: string + ): Set { + const hints = new Set(); + for (const candidate of candidates) { + const belongsToTable = candidate.chunk.metadata.tableNames.some( + (tableName) => tableName.trim().toLowerCase() === normalizedTableName + ); + if (!belongsToTable) { + continue; + } + for (const evidence of candidate.evidence) { + if (!evidence.startsWith(COLUMN_HINT_PREFIX)) { + continue; + } + const rawColumnName = evidence.slice(COLUMN_HINT_PREFIX.length).trim().toLowerCase(); + if (rawColumnName) { + hints.add(rawColumnName); + } + } + } + return hints; + } + + private scoreColumnByQueryTokens( + normalizedColumnName: string, + queryTokens: string[] + ): number { + if (queryTokens.length === 0) { + return 0; + } + const columnParts = normalizedColumnName.split(/[_\s]+/g).filter((part) => part.length > 0); + const substantialParts = columnParts.filter((part) => part.length >= 3); + let score = 0; + for (const token of queryTokens) { + if (token === normalizedColumnName || columnParts.includes(token)) { + score += 2; + continue; + } + if ( + token.length >= 3 && + (normalizedColumnName.includes(token) || + substantialParts.some((part) => part.includes(token) || token.includes(part))) + ) { + score += 1; + } + } + return score; + } + + private isAnchorColumn(normalizedColumnName: string): boolean { + if ( + normalizedColumnName === "id" || + normalizedColumnName.endsWith("_id") || + normalizedColumnName === "created_at" || + normalizedColumnName === "updated_at" || + normalizedColumnName === "deleted_at" + ) { + return true; + } + return normalizedColumnName.endsWith("_at"); + } + private resolveLaneTimeoutMs( input: RagRetrievalRequest ): Record { @@ -825,6 +1435,60 @@ export class RagRetrievalService { return Array.from(new Set(values)); } + private attachColumnPruningEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"], + evidence: ColumnPruningEvidence | undefined + ): void { + const target = bundle as RagRetrievalResponse["retrieval_bundle"] & { + column_pruning?: ColumnPruningEvidence; + columnPruning?: ColumnPruningEvidence; + }; + if (!evidence) { + delete target.column_pruning; + delete target.columnPruning; + return; + } + target.column_pruning = evidence; + target.columnPruning = evidence; + } + + private readColumnPruningEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"] + ): ColumnPruningEvidence | undefined { + const source = bundle as RagRetrievalResponse["retrieval_bundle"] & { + column_pruning?: ColumnPruningEvidence; + columnPruning?: ColumnPruningEvidence; + }; + return source.column_pruning ?? source.columnPruning; + } + + private readPriorSqlLaneEvidence( + bundle: RagRetrievalResponse["retrieval_bundle"] + ): RagPriorSqlLaneEvidence | undefined { + const priorSqlLane = bundle.prior_sql_lane ?? bundle.priorSqlLane; + if (!priorSqlLane) { + return undefined; + } + return this.withPriorSqlLaneCompatFields(priorSqlLane); + } + + private withPriorSqlLaneCompatFields( + lane: RagPriorSqlLaneEvidence + ): RagPriorSqlLaneEvidence { + const degradeReasons = lane.degrade_reasons ?? lane.degradeReasons; + return { + ...lane, + matched_count: lane.matched_count, + selected_count: lane.selected_count, + filtered_count: lane.filtered_count, + ...(degradeReasons ? { degrade_reasons: degradeReasons } : {}), + matchedCount: lane.matched_count, + selectedCount: lane.selected_count, + filteredCount: lane.filtered_count, + ...(degradeReasons ? { degradeReasons } : {}) + }; + } + private async buildContextPack(input: { bundle?: RagRetrievalResponse["retrieval_bundle"]; workspaceId?: string; @@ -991,6 +1655,8 @@ export class RagRetrievalService { degradeReasons: bundle.degrade_reasons, decisionReasons: bundle.decision_reasons ?? [], candidateCount: bundle.candidates.length, + priorSqlLane: this.readPriorSqlLaneEvidence(bundle), + columnPruning: this.readColumnPruningEvidence(bundle), skillContext: bundle.skill_context, candidates: bundle.candidates.map((candidate) => ({ chunkId: candidate.chunk_id, diff --git a/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts b/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts index dcef1b4..6e5f68b 100644 --- a/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts +++ b/apps/backend/src/modules/rag/retrieval/rag-retrieval.types.ts @@ -14,6 +14,7 @@ export interface RagRetrievalRequest { datasourceId: string; runId: string; workspaceId?: string; + allowedTables?: string[]; activeIndexVersionId?: string; perLaneLimit?: number; finalCandidateLimit?: number; @@ -140,6 +141,18 @@ export interface RagContextPack { riskTags?: string[]; } +export interface RagPriorSqlLaneEvidence { + status: "hit" | "miss" | "filtered"; + matched_count: number; + selected_count: number; + filtered_count: number; + degrade_reasons?: string[]; + matchedCount?: number; + selectedCount?: number; + filteredCount?: number; + degradeReasons?: string[]; +} + export interface RagRetrievalBundle { query: string; run_id: string; @@ -154,6 +167,8 @@ export interface RagRetrievalBundle { risk_tags?: string[]; skill_context?: RagSkillContext; context_pack?: RagContextPack; + prior_sql_lane?: RagPriorSqlLaneEvidence; + priorSqlLane?: RagPriorSqlLaneEvidence; decision_reasons?: string[]; } diff --git a/apps/backend/test/e2e/clarification-hybrid-balance.acceptance.spec.ts b/apps/backend/test/e2e/clarification-hybrid-balance.acceptance.spec.ts new file mode 100644 index 0000000..11405eb --- /dev/null +++ b/apps/backend/test/e2e/clarification-hybrid-balance.acceptance.spec.ts @@ -0,0 +1,87 @@ +import { mkdtemp, readFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { collectClarificationBalanceGate } from "../../scripts/collect-clarification-balance-gate"; + +describe("clarification hybrid balance acceptance", () => { + let tempDir = ""; + let outputPath = ""; + const envBackup = { + CLARIFICATION_BALANCE_GATE_MIN_SAMPLES: process.env.CLARIFICATION_BALANCE_GATE_MIN_SAMPLES, + CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE: + process.env.CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE, + CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE: + process.env.CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE, + CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE: + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE, + CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE: + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE, + CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE: + process.env.CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE, + CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS: + process.env.CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS + }; + + beforeAll(async () => { + tempDir = await mkdtemp(join(tmpdir(), "clarification-hybrid-balance-")); + outputPath = join(tempDir, "gate-summary.json"); + process.env.CLARIFICATION_BALANCE_GATE_MIN_SAMPLES = "4"; + process.env.CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE = "0.25"; + process.env.CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE = "0.75"; + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE = "0"; + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE = "0"; + process.env.CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE = "1"; + process.env.CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS = "1"; + }); + + afterAll(async () => { + process.env.CLARIFICATION_BALANCE_GATE_MIN_SAMPLES = + envBackup.CLARIFICATION_BALANCE_GATE_MIN_SAMPLES; + process.env.CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE = + envBackup.CLARIFICATION_BALANCE_GATE_MIN_TRIGGER_RATE; + process.env.CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE = + envBackup.CLARIFICATION_BALANCE_GATE_MAX_TRIGGER_RATE; + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE = + envBackup.CLARIFICATION_BALANCE_GATE_MAX_FALSE_POSITIVE_RATE; + process.env.CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE = + envBackup.CLARIFICATION_BALANCE_GATE_MAX_FALSE_NEGATIVE_RATE; + process.env.CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE = + envBackup.CLARIFICATION_BALANCE_GATE_MIN_POST_CLARIFY_SEMANTIC_PASS_RATE; + process.env.CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS = + envBackup.CLARIFICATION_BALANCE_GATE_MAX_AVG_CLARIFICATION_ROUNDS; + if (tempDir) { + await rm(tempDir, { recursive: true, force: true }); + } + }); + + it("collects machine-readable metrics for hybrid clarification balance and passes strict mode", async () => { + const report = await collectClarificationBalanceGate({ + fixturePath: resolve(process.cwd(), "test/fixtures/clarification-balance-cases.json"), + outputPath, + strictMode: true + }); + + expect(report.fixture.sampleSize).toBe(4); + expect(report.sampleReady).toBe(true); + expect(report.gatePass).toBe(true); + expect(report.metrics.triggerRate).toBeCloseTo(0.5, 6); + expect(report.metrics.falsePositiveRate).toBe(0); + expect(report.metrics.falseNegativeRate).toBe(0); + expect(report.metrics.postClarifySemanticPassRate).toBe(1); + expect(report.metrics.averageClarificationRounds).toBe(1); + expect(report.diagnostics.metadataBypassMismatchCaseIds).toEqual([]); + expect(report.diagnostics.strictSemanticPathMismatchCaseIds).toEqual([]); + expect(report.diagnostics.caseErrors).toEqual([]); + expect(report.diagnostics.falseNegativeCaseIds).toEqual([]); + expect(report.diagnostics.falsePositiveCaseIds).toEqual([]); + + const persisted = JSON.parse(await readFile(outputPath, "utf8")) as { + gatePass: boolean; + metrics: { + triggerRate: number; + }; + }; + expect(persisted.gatePass).toBe(true); + expect(persisted.metrics.triggerRate).toBeCloseTo(0.5, 6); + }); +}); diff --git a/apps/backend/test/e2e/stage1-acceptance.spec.ts b/apps/backend/test/e2e/stage1-acceptance.spec.ts index 14013f1..2f7a6d1 100644 --- a/apps/backend/test/e2e/stage1-acceptance.spec.ts +++ b/apps/backend/test/e2e/stage1-acceptance.spec.ts @@ -1,5 +1,6 @@ import { access, readFile, writeFile } from "node:fs/promises"; import { resolve } from "node:path"; +import { randomUUID } from "node:crypto"; import yaml from "js-yaml"; import { Test } from "@nestjs/testing"; import { AppModule } from "../../src/app.module"; @@ -70,7 +71,7 @@ describe("stage1 acceptance", () => { for (const item of parsed.cases) { const run = await graph.run({ - runId: `stage1-${item.id}`, + runId: randomUUID(), sessionId: "stage1", question: item.question, datasourceId: "sqlite_main", diff --git a/apps/backend/test/fixtures/clarification-balance-cases.json b/apps/backend/test/fixtures/clarification-balance-cases.json new file mode 100644 index 0000000..4efdf12 --- /dev/null +++ b/apps/backend/test/fixtures/clarification-balance-cases.json @@ -0,0 +1,52 @@ +{ + "version": "2026-04-25-clarification-hybrid-balance-v1", + "generatedAt": "2026-04-25T08:00:00.000Z", + "cases": [ + { + "id": "clarify-to-aggregate", + "scenario": "clarify_to_aggregate", + "initialQuestion": "帮我看下订单", + "followUpQuestion": "近30天订单总数按支付方式分组", + "expected": { + "shouldClarify": true, + "metadataBypass": false, + "strictSemanticPath": true, + "maxClarificationRounds": 1 + } + }, + { + "id": "clarify-to-compare", + "scenario": "clarify_to_compare", + "initialQuestion": "帮我比较一下订单", + "followUpQuestion": "近30天订单总数和退款金额对比", + "expected": { + "shouldClarify": true, + "metadataBypass": false, + "strictSemanticPath": true, + "maxClarificationRounds": 1 + } + }, + { + "id": "skip-but-strict", + "scenario": "skip_but_strict", + "initialQuestion": "近30天订单总数按支付方式分组", + "expected": { + "shouldClarify": false, + "metadataBypass": false, + "strictSemanticPath": true, + "maxClarificationRounds": 0 + } + }, + { + "id": "metadata-bypass", + "scenario": "metadata_bypass", + "initialQuestion": "show tables", + "expected": { + "shouldClarify": false, + "metadataBypass": true, + "strictSemanticPath": true, + "maxClarificationRounds": 0 + } + } + ] +} diff --git a/apps/backend/test/integration/agent-main-flow.spec.ts b/apps/backend/test/integration/agent-main-flow.spec.ts index 3a4df0a..25fec54 100644 --- a/apps/backend/test/integration/agent-main-flow.spec.ts +++ b/apps/backend/test/integration/agent-main-flow.spec.ts @@ -32,6 +32,8 @@ describe("agent main flow", () => { expect(run.trace.steps[0]?.sequence).toBe(1); expect(run.trace.steps[0]?.stepId).toContain("run-1:"); expect(run.trace.steps[0]?.lifecycle).toBeTruthy(); + expect(run.trace.clarificationDecision).toBeDefined(); + expect(["continue", "clarify"]).toContain(run.trace.clarificationDecision?.decision); }); it("should reject non-readonly sql intent", async () => { @@ -47,5 +49,7 @@ describe("agent main flow", () => { datasourceType: "sqlite" }); expect(run.status).toBe("rejected"); + expect(run.trace.clarificationDecision?.decisionSource).toBe("sql-write-intent"); + expect(run.trace.clarificationDecision?.bypassed).toBe(true); }); }); diff --git a/apps/backend/test/integration/agent-rag-main-flow.spec.ts b/apps/backend/test/integration/agent-rag-main-flow.spec.ts index 3278d1e..fe83199 100644 --- a/apps/backend/test/integration/agent-rag-main-flow.spec.ts +++ b/apps/backend/test/integration/agent-rag-main-flow.spec.ts @@ -2,6 +2,8 @@ import { resolve } from "node:path"; import { Test } from "@nestjs/testing"; import { AppModule } from "../../src/app.module"; import { GraphBuilderService } from "../../src/modules/conversation/agent/graph/graph.builder"; +import { BuildIntentPlanNode } from "../../src/modules/conversation/agent/nodes/build-intent-plan.node"; +import { RetrieveKnowledgeNode } from "../../src/modules/conversation/agent/nodes/retrieve-knowledge.node"; import { RagEventConsumerService } from "../../src/modules/rag/events/rag-event-consumer.service"; import { RagIndexBuilderService } from "../../src/modules/rag/index/rag-index-builder.service"; import { RagIndexRepository } from "../../src/modules/rag/index/rag-index.repository"; @@ -25,6 +27,8 @@ describe("agent rag main flow integration", () => { }).compile(); const graph = moduleRef.get(GraphBuilderService); + const buildIntentPlanNode = moduleRef.get(BuildIntentPlanNode); + const retrieveKnowledgeNode = moduleRef.get(RetrieveKnowledgeNode); const repository = moduleRef.get(RagIndexRepository); const builder = moduleRef.get(RagIndexBuilderService); const eventConsumer = moduleRef.get(RagEventConsumerService); @@ -85,6 +89,42 @@ describe("agent rag main flow integration", () => { const safetyStep = run.trace.steps.find((step) => step.node === "safety-check"); expect(safetyStep?.outputSummary).toContain("riskTags"); + expect(run.trace.clarificationDecision).toBeDefined(); + + const unpinnedKnowledge = await retrieveKnowledgeNode.run({ + question: "统计订单 GMV", + datasourceId: "sqlite_main", + runId: "run-agent-rag-main-retrieve-unpinned" + }); + const pinnedKnowledge = await retrieveKnowledgeNode.run({ + question: "统计订单 GMV", + datasourceId: "sqlite_main", + runId: "run-agent-rag-main-retrieve-pinned", + pinnedTables: ["refunds"], + pinnedColumns: ["refund_amount"] + }); + expect(unpinnedKnowledge.pinning?.status).toBe("inactive"); + expect(pinnedKnowledge.pinning?.enabled).toBe(true); + expect(pinnedKnowledge.pinning?.status).toBe("applied"); + expect(pinnedKnowledge.summary).toContain("pinning[candidates="); + expect( + (pinnedKnowledge.retrievalBundle?.selected_context?.length ?? 0) <= + (unpinnedKnowledge.retrievalBundle?.selected_context?.length ?? 0) + ).toBe(true); + expect( + pinnedKnowledge.retrievalBundle?.candidates.every((candidate) => + candidate.chunk.metadata.tableNames + .map((tableName) => tableName.toLowerCase()) + .includes("refunds") + ) ?? true + ).toBe(true); + const pinnedIntentPlan = await buildIntentPlanNode.run( + "统计订单 GMV", + pinnedKnowledge + ); + expect(pinnedIntentPlan.constraints).toEqual( + expect.arrayContaining(["require_pinned_table_alignment"]) + ); await eventConsumer.consumeEvent({ eventId: "evt-agent-rag-main-linkage-degraded-1", diff --git a/apps/backend/test/integration/chat-context-envelope-accuracy.spec.ts b/apps/backend/test/integration/chat-context-envelope-accuracy.spec.ts index 012f303..72305ba 100644 --- a/apps/backend/test/integration/chat-context-envelope-accuracy.spec.ts +++ b/apps/backend/test/integration/chat-context-envelope-accuracy.spec.ts @@ -99,6 +99,28 @@ describe("chat context envelope accuracy integration", () => { expect.arrayContaining(["context_conflict_detected"]) ); + const pinnedContextRun = await graphBuilder.run({ + runId: "run-context-accuracy-pinning", + sessionId: "session-context-accuracy-pinning", + question: "数据库有哪些表", + datasourceId: "sqlite_main", + datasourceType: "sqlite", + contextEnvelope: { + pinnedTables: ["orders"], + pinnedColumns: ["amount"] + }, + planningScaffoldEnabled: true + }); + expect( + pinnedContextRun.trace.effectiveContextSummary?.userEnvelope.pinnedTableCount + ).toBe(1); + expect( + pinnedContextRun.trace.effectiveContextSummary?.userEnvelope.pinnedColumnCount + ).toBe(1); + expect( + pinnedContextRun.trace.effectiveContextSummary?.retrievalContext?.pinning?.status + ).toBeTruthy(); + await moduleRef.close(); }); }); diff --git a/apps/backend/test/integration/rag-quality.spec.ts b/apps/backend/test/integration/rag-quality.spec.ts index 4e8953d..b8c152c 100644 --- a/apps/backend/test/integration/rag-quality.spec.ts +++ b/apps/backend/test/integration/rag-quality.spec.ts @@ -38,6 +38,12 @@ describe("rag quality gate integration", () => { expect(snapshot.sampleReady).toBe(true); expect(snapshot.gatePass).toBe(true); expect(snapshot.reasons).toEqual([]); + expect(snapshot.priorSqlLane).toEqual({ + sampleSize: 0, + priorSqlHitRate: 0, + priorSqlFilteredCount: 0, + priorSqlStaleRate: 0 + }); await moduleRef.close(); }); @@ -67,6 +73,56 @@ describe("rag quality gate integration", () => { await moduleRef.close(); }); + it("aggregates prior SQL lane metrics when data is present", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + const quality = moduleRef.get(RagQualityService); + quality.reset(); + + quality.recordEvaluation({ + runId: "run-rag-quality-prior-sql-v1", + datasourceId: "ds-rag-quality-prior-sql", + sampleSize: 32, + recallAt20: 0.84, + mrrAt10: 0.7, + retrievalRerankP95Ms: 620, + degradeRate: 0.03, + priorSqlLane: { + totalCount: 20, + hitCount: 15, + filteredCount: 2, + staleCount: 1 + } + }); + quality.recordEvaluation({ + runId: "run-rag-quality-prior-sql-v2", + datasourceId: "ds-rag-quality-prior-sql", + sampleSize: 36, + recallAt20: 0.86, + mrrAt10: 0.72, + retrievalRerankP95Ms: 600, + degradeRate: 0.02, + priorSqlLane: { + totalCount: 10, + hitCount: 5, + filteredCount: 1, + staleCount: 2 + } + }); + + const snapshot = quality.snapshot(); + expect(snapshot.priorSqlLane).toEqual({ + sampleSize: 30, + priorSqlHitRate: 0.666667, + priorSqlFilteredCount: 3, + priorSqlStaleRate: 0.1 + }); + expect(snapshot.gatePass).toBe(true); + + await moduleRef.close(); + }); + it("exposes rag quality gate summary on /health", async () => { const moduleRef = await Test.createTestingModule({ imports: [AppModule] diff --git a/apps/backend/test/integration/rag-retrieval.service.spec.ts b/apps/backend/test/integration/rag-retrieval.service.spec.ts index 1cbb757..be1f29b 100644 --- a/apps/backend/test/integration/rag-retrieval.service.spec.ts +++ b/apps/backend/test/integration/rag-retrieval.service.spec.ts @@ -180,6 +180,295 @@ describe("rag retrieval service integration", () => { await moduleRef.close(); }); + it("marks trusted prior SQL as hit and promotes it into retrieval candidates", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + + const indexRepository = moduleRef.get(RagIndexRepository); + const indexBuilder = moduleRef.get(RagIndexBuilderService); + const retrievalService = moduleRef.get(RagRetrievalService); + const replayRepository = moduleRef.get(RagReplayRepository); + + const datasourceId = "ds-rag-retrieval-prior-sql-hit"; + const workspaceId = "ws-rag-prior-hit"; + indexRepository.seedChunksForDatasource(datasourceId, [ + { + id: "chunk-prior-hit-schema", + datasourceId, + domain: "schema", + content: "table orders(id, amount, status)", + metadata: JSON.stringify({ + tableNames: ["orders"], + columnNames: ["id", "amount", "status"] + }) + }, + { + id: "chunk-prior-hit-sql", + datasourceId, + domain: "sql_example", + content: "SELECT SUM(amount) FROM orders WHERE status = 'paid'", + metadata: JSON.stringify({ + trusted: true, + priorSql: true, + workspaceId, + tableNames: ["orders"], + columnNames: ["amount", "status"] + }) + }, + { + id: "chunk-prior-hit-semantic", + datasourceId, + domain: "semantic_term", + content: "GMV maps to SUM(amount) for paid orders.", + metadata: JSON.stringify({ + tableNames: ["orders"], + columnNames: ["amount"] + }) + } + ]); + await indexBuilder.buildAndActivate({ + datasourceId, + sourceVersion: "source-rag-retrieval-prior-sql-hit-v1", + createdByRunId: "run-rag-retrieval-prior-sql-hit-build-v1", + activatedByRunId: "run-rag-retrieval-prior-sql-hit-build-v1" + }); + + const response = await retrievalService.retrieve({ + query: "orders paid gmv", + datasourceId, + workspaceId, + allowedTables: ["orders"], + runId: "run-rag-retrieval-prior-sql-hit-v1" + }); + + const priorSqlLane = response.retrieval_bundle.prior_sql_lane; + expect(priorSqlLane).toBeDefined(); + expect(priorSqlLane?.status).toBe("hit"); + expect(priorSqlLane?.matched_count).toBe(1); + expect(priorSqlLane?.selected_count).toBe(1); + expect(response.retrieval_bundle.candidates[0]?.chunk_id).toBe("chunk-prior-hit-sql"); + expect(response.retrieval_bundle.candidates[0]?.evidence).toEqual( + expect.arrayContaining(["prior_sql:trusted"]) + ); + + const replayEvents = await replayRepository.listByRunId("run-rag-retrieval-prior-sql-hit-v1"); + const fusedReplay = replayEvents.find((item) => item.replayKey === "retrieval:fused"); + expect(fusedReplay).toBeDefined(); + const fusedPayload = JSON.parse(fusedReplay?.payload ?? "{}") as { + priorSqlLane?: { status?: string; selectedCount?: number }; + }; + expect(fusedPayload.priorSqlLane?.status).toBe("hit"); + expect(fusedPayload.priorSqlLane?.selectedCount).toBe(1); + + await moduleRef.close(); + }); + + it("filters trusted prior SQL by workspace and allowed tables while preserving fallback candidates", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + + const indexRepository = moduleRef.get(RagIndexRepository); + const indexBuilder = moduleRef.get(RagIndexBuilderService); + const retrievalService = moduleRef.get(RagRetrievalService); + + const datasourceId = "ds-rag-retrieval-prior-sql-filtered"; + indexRepository.seedChunksForDatasource(datasourceId, [ + { + id: "chunk-prior-filter-schema", + datasourceId, + domain: "schema", + content: "table orders(id, amount, status)", + metadata: JSON.stringify({ + tableNames: ["orders"], + columnNames: ["id", "amount", "status"] + }) + }, + { + id: "chunk-prior-filter-workspace", + datasourceId, + domain: "sql_example", + content: "SELECT SUM(amount) FROM orders WHERE status = 'paid'", + metadata: JSON.stringify({ + trusted: true, + priorSql: true, + workspaceId: "workspace-other", + tableNames: ["orders"], + columnNames: ["amount", "status"] + }) + }, + { + id: "chunk-prior-filter-tables", + datasourceId, + domain: "sql_example", + content: "SELECT COUNT(*) FROM secret_orders", + metadata: JSON.stringify({ + trusted: true, + verified: true, + workspaceId: "workspace-current", + tableNames: ["secret_orders"], + columnNames: ["id"] + }) + }, + { + id: "chunk-prior-filter-semantic", + datasourceId, + domain: "semantic_term", + content: "GMV maps to order amount.", + metadata: JSON.stringify({ + tableNames: ["orders"], + columnNames: ["amount"] + }) + } + ]); + await indexBuilder.buildAndActivate({ + datasourceId, + sourceVersion: "source-rag-retrieval-prior-sql-filtered-v1", + createdByRunId: "run-rag-retrieval-prior-sql-filtered-build-v1", + activatedByRunId: "run-rag-retrieval-prior-sql-filtered-build-v1" + }); + + const response = await retrievalService.retrieve({ + query: "orders paid gmv", + datasourceId, + workspaceId: "workspace-current", + allowedTables: ["orders"], + runId: "run-rag-retrieval-prior-sql-filtered-v1" + }); + + const priorSqlLane = response.retrieval_bundle.prior_sql_lane; + expect(priorSqlLane).toBeDefined(); + expect(priorSqlLane?.status).toBe("filtered"); + expect(priorSqlLane?.matched_count).toBe(2); + expect(priorSqlLane?.selected_count).toBe(0); + expect(priorSqlLane?.filtered_count).toBe(2); + expect(priorSqlLane?.degrade_reasons).toEqual( + expect.arrayContaining([ + "prior_sql_filtered_workspace_mismatch", + "prior_sql_filtered_not_in_allowed_tables" + ]) + ); + expect( + response.retrieval_bundle.candidates.some( + (item) => item.chunk_id === "chunk-prior-filter-workspace" + ) + ).toBe(false); + expect( + response.retrieval_bundle.candidates.some( + (item) => item.chunk_id === "chunk-prior-filter-tables" + ) + ).toBe(false); + expect(response.retrieval_bundle.candidates.length).toBeGreaterThan(0); + expect(response.retrieval_bundle.lane_results.lexical.status).toBe("ok"); + expect(response.retrieval_bundle.lane_results.dense.status).toBe("ok"); + expect(response.retrieval_bundle.lane_results.graph.status).toBe("ok"); + + await moduleRef.close(); + }); + + it("uses conservative table-first/field-second pruning on wide tables and safely degrades when evidence is weak", async () => { + const moduleRef = await Test.createTestingModule({ + imports: [AppModule] + }).compile(); + + const indexRepository = moduleRef.get(RagIndexRepository); + const indexBuilder = moduleRef.get(RagIndexBuilderService); + const retrievalService = moduleRef.get(RagRetrievalService); + const replayRepository = moduleRef.get(RagReplayRepository); + + const datasourceId = "ds-rag-retrieval-column-pruning"; + const wideColumns = [ + "id", + "customer_id", + "amount", + "status", + "order_date", + "shipping_city", + "sales_region", + "discount_rate", + "internal_note", + "legacy_flag", + "created_at", + "updated_at" + ]; + indexRepository.seedChunksForDatasource(datasourceId, [ + { + id: "chunk-wide-schema-orders", + datasourceId, + domain: "schema", + content: `table orders_wide(${wideColumns.join(", ")})`, + metadata: JSON.stringify({ + chunkProfile: "schema_table", + tableNames: ["orders_wide"], + columnNames: wideColumns + }) + } + ]); + await indexBuilder.buildAndActivate({ + datasourceId, + sourceVersion: "source-rag-retrieval-column-pruning-v1", + createdByRunId: "run-rag-retrieval-column-pruning-build-v1", + activatedByRunId: "run-rag-retrieval-column-pruning-build-v1" + }); + + const strongSignal = await retrievalService.retrieve({ + query: "orders_wide amount status", + datasourceId, + runId: "run-rag-retrieval-column-pruning-strong-v1" + }); + const strongColumns = + strongSignal.retrieval_bundle.candidates.find( + (candidate) => candidate.chunk_id === "chunk-wide-schema-orders" + )?.chunk.metadata.columnNames ?? []; + expect(strongColumns.length).toBeGreaterThan(0); + expect(strongColumns.length).toBeLessThan(wideColumns.length); + expect(strongColumns).toEqual(expect.arrayContaining(["amount", "status"])); + expect(strongColumns).not.toContain("internal_note"); + + const uncertainSignal = await retrievalService.retrieve({ + query: "orders_wide customer profile", + datasourceId, + runId: "run-rag-retrieval-column-pruning-uncertain-v1" + }); + const uncertainColumns = + uncertainSignal.retrieval_bundle.candidates.find( + (candidate) => candidate.chunk_id === "chunk-wide-schema-orders" + )?.chunk.metadata.columnNames ?? []; + expect(uncertainColumns.length).toBeGreaterThan(1); + expect(uncertainColumns.length).toBeLessThanOrEqual(wideColumns.length); + + const weakSignal = await retrievalService.retrieve({ + query: "orders_wide overview", + datasourceId, + runId: "run-rag-retrieval-column-pruning-weak-v1" + }); + const weakColumns = + weakSignal.retrieval_bundle.candidates.find( + (candidate) => candidate.chunk_id === "chunk-wide-schema-orders" + )?.chunk.metadata.columnNames ?? []; + expect(weakColumns).toHaveLength(wideColumns.length); + + const replayEvents = await replayRepository.listByRunId( + "run-rag-retrieval-column-pruning-strong-v1" + ); + const fusedReplay = replayEvents.find((item) => item.replayKey === "retrieval:fused"); + const fusedPayload = JSON.parse(fusedReplay?.payload ?? "{}") as { + columnPruning?: { + status?: string; + tables?: Array<{ + table_name?: string; + mode?: string; + }>; + }; + }; + expect(fusedPayload.columnPruning?.status).toBe("applied"); + expect(fusedPayload.columnPruning?.tables?.[0]?.table_name).toBe("orders_wide"); + expect(fusedPayload.columnPruning?.tables?.[0]?.mode).toBe("conservative"); + + await moduleRef.close(); + }); + it("returns degraded bundle when datasource has no active index", async () => { const moduleRef = await Test.createTestingModule({ imports: [AppModule] diff --git a/apps/backend/test/integration/workspace-modeling-setup.spec.ts b/apps/backend/test/integration/workspace-modeling-setup.spec.ts index 7bdcfa4..d52b97e 100644 --- a/apps/backend/test/integration/workspace-modeling-setup.spec.ts +++ b/apps/backend/test/integration/workspace-modeling-setup.spec.ts @@ -130,6 +130,8 @@ describe("workspace modeling setup integration", () => { expect(preview.relationshipRecommendations[0]?.sourceTable).toBe("orders"); expect(preview.relationshipRecommendations[0]?.targetTable).toBe("customers"); expect(preview.relationshipRecommendations[0]?.reason).toBe("foreign_key_constraint"); + expect(preview.relationshipRecommendations[0]?.type).toBe("many-to-one"); + expect(preview.relationshipRecommendations[0]?.cardinality).toBe("many-to-one"); const committed = await service.commitSetup( actor, @@ -166,7 +168,9 @@ describe("workspace modeling setup integration", () => { relationships: [ expect.objectContaining({ id: preview.relationshipRecommendations[0]!.id, - source: "fk" + source: "fk", + type: "many-to-one", + cardinality: "many-to-one" }) ] }) diff --git a/apps/backend/test/unit/build-intent-plan.node.spec.ts b/apps/backend/test/unit/build-intent-plan.node.spec.ts new file mode 100644 index 0000000..4f8e166 --- /dev/null +++ b/apps/backend/test/unit/build-intent-plan.node.spec.ts @@ -0,0 +1,144 @@ +import { + BuildIntentPlanNode +} from "../../src/modules/conversation/agent/nodes/build-intent-plan.node"; +import type { ClarificationFusionPolicy } from "../../src/modules/conversation/agent/nodes/clarification-fusion.policy"; +import type { ClarificationSemanticEvaluatorService } from "../../src/modules/conversation/agent/nodes/clarification-semantic-evaluator.service"; +import type { RetrievedKnowledge } from "../../src/modules/conversation/agent/nodes/retrieve-knowledge.node"; + +describe("BuildIntentPlanNode", () => { + const createNode = (overrides?: { + shouldArbitrate?: boolean; + fusedDecision?: { + decision?: "continue" | "clarify"; + decisionSource?: string; + bypassed?: boolean; + confidenceLevel?: "high" | "medium" | "low"; + reasonCodes?: string[]; + conflictDetected?: boolean; + missingCriticalSlots?: string[]; + reason?: string; + question?: string; + }; + }) => { + const semanticEvaluator = { + evaluate: jest.fn().mockResolvedValue({ + status: "success", + decision: { + decision: "clarify", + triggerPath: "semantic", + confidenceLevel: "medium", + reasonCodes: ["semantic_needs_clarification"], + missingCriticalSlots: ["metric"] + }, + reasonCodes: ["semantic_needs_clarification"], + metadata: { + timeoutMs: 1000, + elapsedMs: 10, + fallbackApplied: false + } + }) + } as unknown as ClarificationSemanticEvaluatorService; + + const fusionPolicy = { + shouldArbitrate: jest.fn().mockReturnValue(overrides?.shouldArbitrate ?? false), + fuse: jest.fn().mockReturnValue({ + decision: { + decision: overrides?.fusedDecision?.decision ?? "continue", + triggerPath: "rule", + decisionSource: overrides?.fusedDecision?.decisionSource ?? "rule", + bypassed: overrides?.fusedDecision?.bypassed ?? false, + confidenceLevel: overrides?.fusedDecision?.confidenceLevel ?? "high", + reasonCodes: overrides?.fusedDecision?.reasonCodes ?? ["rule_slots_sufficient"], + conflictDetected: overrides?.fusedDecision?.conflictDetected ?? false, + missingCriticalSlots: overrides?.fusedDecision?.missingCriticalSlots ?? [], + reason: overrides?.fusedDecision?.reason ?? "问题信息充足", + question: overrides?.fusedDecision?.question ?? "" + }, + metadata: { + mode: "hybrid", + semanticRequested: overrides?.shouldArbitrate ?? false, + semanticUsed: overrides?.shouldArbitrate ?? false, + fallbackApplied: false, + strictModeReasons: + overrides?.fusedDecision?.confidenceLevel === "low" + ? ["clarification_low_confidence"] + : [], + stageRiskTags: { + rule: ["rule:rule_slots_sufficient"], + semantic: [], + fusion: [] + } + } + }) + } as unknown as ClarificationFusionPolicy; + + return { + node: new BuildIntentPlanNode(semanticEvaluator, fusionPolicy), + semanticEvaluator, + fusionPolicy + }; + }; + + const readyKnowledge = { + status: "ready" as const, + snippets: ["stub"], + summary: "retrieval ready", + retrievalBundle: { + selected_context: [] + } + } as unknown as RetrievedKnowledge; + + it("returns aggregate intent with low uncertainty for clear query", async () => { + const { node } = createNode(); + + const plan = await node.run("统计近30天订单总数", readyKnowledge); + + expect(plan).toMatchObject({ + status: "ready", + intent: "aggregate" + }); + expect(plan.uncertaintySignal).toMatchObject({ + level: "low", + needsStrictSemanticPath: false + }); + expect(plan.riskTags).toEqual(expect.arrayContaining(["intent:rule:rule_slots_sufficient"])); + }); + + it("marks bypass decisions to skip business semantic assertions", async () => { + const { node } = createNode({ + fusedDecision: { + decision: "continue", + decisionSource: "metadata-intent", + bypassed: true, + reasonCodes: ["bypass_metadata_intent"] + } + }); + + const plan = await node.run("show tables", readyKnowledge); + + expect(plan.constraints).toContain("skip_business_semantic_assertions"); + expect(plan.uncertaintySignal).toMatchObject({ + level: "low", + needsStrictSemanticPath: false + }); + }); + + it("promotes uncertainty when fused decision is low confidence", async () => { + const { node } = createNode({ + shouldArbitrate: true, + fusedDecision: { + decision: "continue", + confidenceLevel: "low", + reasonCodes: ["rule_confidence_low"] + } + }); + + const plan = await node.run("统计订单", readyKnowledge); + + expect(plan.uncertaintySignal).toMatchObject({ + level: "medium", + needsStrictSemanticPath: true + }); + expect(plan.planningWarnings?.join(" ")).toContain("strict semantic path"); + }); +}); diff --git a/apps/backend/test/unit/build-semantic-query.node.spec.ts b/apps/backend/test/unit/build-semantic-query.node.spec.ts new file mode 100644 index 0000000..12db642 --- /dev/null +++ b/apps/backend/test/unit/build-semantic-query.node.spec.ts @@ -0,0 +1,108 @@ +import { + BuildSemanticQueryNode +} from "../../src/modules/conversation/agent/nodes/build-semantic-query.node"; +import type { PlannerVersionLockService } from "../../src/modules/conversation/agent/planner/planner-version-lock.service"; + +describe("BuildSemanticQueryNode", () => { + const createNode = (lockStatus: "locked" | "fallback" | "degraded" = "locked") => { + const plannerVersionLock = { + resolve: jest.fn().mockResolvedValue({ + lockStatus, + semanticVersion: 3, + fallbackApplied: lockStatus === "fallback", + degradeReason: lockStatus === "degraded" ? "semantic_unavailable" : undefined, + riskTags: lockStatus === "degraded" ? ["semantic_registry_degraded"] : [], + domain: "semantic_term", + term: "orders" + }) + } as unknown as PlannerVersionLockService; + + return { + node: new BuildSemanticQueryNode(plannerVersionLock), + plannerVersionLock + }; + }; + + const baseIntentPlan = { + status: "ready" as const, + intent: "aggregate" as const, + constraints: ["must_use_selected_context"], + summary: "intent ready", + uncertaintySignal: { + level: "high" as const, + needsStrictSemanticPath: true, + reasonCodes: ["clarification_low_confidence"] + }, + clarificationDecision: { + decision: "continue" as const, + triggerPath: "hybrid" as const, + confidenceLevel: "low" as const, + reasonCodes: ["rule_confidence_low"] + }, + riskTags: ["intent:rule:rule_confidence_low"], + planningWarnings: ["strict semantic path required"] + }; + + it("enables strict semantic hints when intent uncertainty requires it", async () => { + const { node } = createNode("locked"); + + const plan = await node.run({ + intentPlan: baseIntentPlan, + question: "统计订单", + retrievalBundle: { + context_pack: { + status: "ready", + instruction_sets: { + model_bindings: [], + relationship_bindings: [], + metric_bindings: [], + calculated_field_bindings: [] + } + } + } as never + }); + + expect(plan.status).toBe("ready"); + expect(plan.strictMode).toBe(true); + expect(plan.semanticHints).toEqual( + expect.arrayContaining(["enforce_strict_clarification_guards", "must_consume_selected_context"]) + ); + expect(plan.riskTags).toEqual( + expect.arrayContaining(["intent:rule:rule_confidence_low", "semantic:strict_mode_enabled"]) + ); + }); + + it("keeps strict mode disabled for bypass constraints", async () => { + const { node } = createNode("locked"); + + const plan = await node.run({ + intentPlan: { + ...baseIntentPlan, + constraints: ["skip_business_semantic_assertions"], + uncertaintySignal: { + level: "low", + needsStrictSemanticPath: false, + reasonCodes: ["intent_bypass_business_semantics"] + } + }, + question: "show tables" + }); + + expect(plan.strictMode).toBe(false); + expect(plan.semanticHints).toContain("preserve_clarification_bypass_reason"); + }); + + it("returns degraded status when version lock is degraded", async () => { + const { node } = createNode("degraded"); + + const plan = await node.run({ + intentPlan: baseIntentPlan, + question: "统计订单" + }); + + expect(plan.status).toBe("degraded"); + expect(plan.lockStatus).toBe("degraded"); + expect(plan.degradeReason).toBe("semantic_unavailable"); + expect(plan.planningWarnings?.semantic.join(" ")).toContain("语义版本锁降级"); + }); +}); diff --git a/apps/backend/test/unit/clarification-fusion.policy.spec.ts b/apps/backend/test/unit/clarification-fusion.policy.spec.ts new file mode 100644 index 0000000..53441b9 --- /dev/null +++ b/apps/backend/test/unit/clarification-fusion.policy.spec.ts @@ -0,0 +1,112 @@ +import type { AppConfigService } from "../../src/modules/config/app-config.service"; +import { + ClarificationFusionPolicy, + mapRuleDecisionToEvidence +} from "../../src/modules/conversation/agent/nodes/clarification-fusion.policy"; + +describe("ClarificationFusionPolicy", () => { + const createPolicy = (options?: { + hybridEnabled?: boolean; + rulesOnlyKillSwitch?: boolean; + }) => { + const config = { + clarificationHybridEnabled: options?.hybridEnabled ?? true, + clarificationRulesOnlyKillSwitch: options?.rulesOnlyKillSwitch ?? false + } as Pick; + return new ClarificationFusionPolicy(config as AppConfigService); + }; + + it("does not arbitrate bypassed decisions", () => { + const policy = createPolicy(); + expect( + policy.shouldArbitrate({ + decision: "continue", + decisionSource: "metadata-intent", + bypassed: true, + confidenceLevel: "high" + }) + ).toBe(false); + }); + + it("forces rules-only when kill switch is enabled", () => { + const policy = createPolicy({ rulesOnlyKillSwitch: true }); + const result = policy.fuse({ + ruleDecision: { + decision: "continue", + triggerPath: "rule", + confidenceLevel: "high", + reasonCodes: ["rule_slots_sufficient"] + } + }); + + expect(result.metadata.mode).toBe("rules-only"); + expect(result.metadata.fallbackApplied).toBe(true); + expect(result.metadata.fallbackReason).toBe("kill-switch"); + expect(result.decision.triggerPath).toBe("rule"); + }); + + it("uses semantic decision when rule confidence is low and semantic is stronger", () => { + const policy = createPolicy(); + const result = policy.fuse({ + ruleDecision: { + decision: "continue", + triggerPath: "rule", + confidenceLevel: "low", + reasonCodes: ["rule_confidence_low"] + }, + semanticEvaluation: { + status: "success", + decision: { + decision: "clarify", + triggerPath: "semantic", + confidenceLevel: "high", + missingCriticalSlots: ["metric"], + reasonCodes: ["semantic_needs_clarification"], + question: "请补充指标口径", + reason: "语义评估认为指标缺失" + }, + reasonCodes: ["semantic_needs_clarification"], + metadata: { + timeoutMs: 1000, + elapsedMs: 12, + fallbackApplied: false + } + } + }); + + expect(result.metadata.mode).toBe("hybrid"); + expect(result.metadata.semanticUsed).toBe(true); + expect(result.decision).toMatchObject({ + decision: "clarify", + triggerPath: "hybrid", + conflictDetected: true + }); + }); + + it("maps rule decision to evidence with bypass metadata", () => { + const evidence = mapRuleDecisionToEvidence({ + decision: "continue", + action: "proceed", + source: "metadata-intent", + decisionSource: "metadata-intent", + triggerPath: "rule", + bypassed: true, + bypassReasonCode: "bypass_metadata_intent", + confidence: "high", + confidenceLevel: "high", + missingSlots: [], + shouldClarify: false, + missingCriticalSlots: [], + reasonCodes: ["bypass_metadata_intent"], + reason: "元数据查询意图，无需业务槽位补全", + question: "" + }); + + expect(evidence).toMatchObject({ + decision: "continue", + decisionSource: "metadata-intent", + bypassed: true, + bypassReasonCode: "bypass_metadata_intent" + }); + }); +}); diff --git a/apps/backend/test/unit/clarification-semantic-evaluator.spec.ts b/apps/backend/test/unit/clarification-semantic-evaluator.spec.ts new file mode 100644 index 0000000..6689165 --- /dev/null +++ b/apps/backend/test/unit/clarification-semantic-evaluator.spec.ts @@ -0,0 +1,66 @@ +import type { AppConfigService } from "../../src/modules/config/app-config.service"; +import { + ClarificationSemanticEvaluatorService +} from "../../src/modules/conversation/agent/nodes/clarification-semantic-evaluator.service"; + +describe("ClarificationSemanticEvaluatorService", () => { + const createService = (timeoutMs = 20) => { + const config = { + clarificationSemanticTimeoutMs: timeoutMs + } as Pick; + return new ClarificationSemanticEvaluatorService(config as AppConfigService); + }; + + it("returns semantic clarify decision for missing slots", async () => { + const service = createService(); + const result = await service.evaluate({ + question: "帮我看下订单", + ruleDecision: { + decision: "clarify", + triggerPath: "rule", + confidenceLevel: "low", + missingCriticalSlots: ["metric"], + reasonCodes: ["missing_metric_slot"] + } + }); + + expect(result.status).toBe("success"); + expect(result.decision).toMatchObject({ + decision: "clarify", + triggerPath: "semantic" + }); + expect(result.reasonCodes).toEqual( + expect.arrayContaining(["semantic_classifier_v1", "semantic_needs_clarification"]) + ); + expect(result.metadata.fallbackApplied).toBe(false); + }); + + it("falls back with timeout when semantic evaluator times out", async () => { + class TimeoutEvaluator extends ClarificationSemanticEvaluatorService { + protected override async runSemanticArbitration(): Promise { + await new Promise((resolve) => setTimeout(resolve, 15)); + throw new Error("should timeout first"); + } + } + const config = { + clarificationSemanticTimeoutMs: 5 + } as Pick; + const service = new TimeoutEvaluator(config as AppConfigService); + + const result = await service.evaluate({ + question: "统计订单", + ruleDecision: { + decision: "clarify", + triggerPath: "rule", + confidenceLevel: "low", + missingCriticalSlots: ["metric"], + reasonCodes: ["missing_metric_slot"] + } + }); + + expect(result.status).toBe("timeout"); + expect(result.metadata.fallbackApplied).toBe(true); + expect(result.metadata.fallbackReason).toBe("timeout"); + expect(result.reasonCodes).toEqual(["semantic_timeout"]); + }); +}); diff --git a/apps/backend/test/unit/clarify.node.spec.ts b/apps/backend/test/unit/clarify.node.spec.ts index 8de045d..8ef1830 100644 --- a/apps/backend/test/unit/clarify.node.spec.ts +++ b/apps/backend/test/unit/clarify.node.spec.ts @@ -1,4 +1,5 @@ import { ClarifyNode } from "../../src/modules/conversation/agent/nodes/clarify.node"; +import * as slotFillingContext from "../../src/modules/conversation/agent/nodes/slot-filling-context"; describe("ClarifyNode", () => { const node = new ClarifyNode(); @@ -13,6 +14,44 @@ describe("ClarifyNode", () => { expect(clarification).toBeDefined(); expect(clarification?.reason).toContain("指标口径"); expect(clarification?.question).toContain("指标口径"); + expect(clarification).toMatchObject({ + decision: "clarify", + triggerPath: "rule", + decisionSource: "rule", + bypassed: false, + confidenceLevel: "medium", + missingCriticalSlots: ["metric"], + reasonCodes: ["missing_metric_slot"] + }); + }); + + it("exposes structured continue decision for sufficient input", () => { + const decision = node.evaluate("统计近30天订单总数"); + expect(decision).toMatchObject({ + decision: "continue", + action: "proceed", + source: "rule", + decisionSource: "rule", + bypassed: false, + confidence: "high", + confidenceLevel: "high", + missingSlots: [], + shouldClarify: false, + reasonCodes: ["rule_slots_sufficient"] + }); + }); + + it("exposes structured clarify decision when critical slots are missing", () => { + const decision = node.evaluate("帮我查一下华北大区订单"); + expect(decision).toMatchObject({ + decision: "clarify", + action: "ask_clarification", + source: "rule", + decisionSource: "rule", + bypassed: false + }); + expect(decision.missingSlots).toContain("metric"); + expect(decision.reasonCodes).toContain("missing_metric_slot"); }); it("uses context envelope to avoid repeated clarification", () => { @@ -38,14 +77,79 @@ describe("ClarifyNode", () => { expect(clarification?.reason).toContain("时间范围"); }); + it("skips clarification for metadata intent", () => { + const clarification = node.run("show tables"); + expect(clarification).toBeUndefined(); + }); + it("skips clarification for sql write intent input", () => { const clarification = node.run("DELETE orders where id = 1"); expect(clarification).toBeUndefined(); }); + it("records metadata intent bypass source and reason code in decision", () => { + const decision = node.evaluate("show tables"); + expect(decision).toMatchObject({ + decision: "continue", + source: "metadata-intent", + decisionSource: "metadata-intent", + bypassed: true, + bypassReasonCode: "bypass_metadata_intent", + reasonCodes: ["bypass_metadata_intent"] + }); + }); + + it("records sql write bypass source and reason code in decision", () => { + const decision = node.evaluate("DELETE FROM orders where id = 1"); + expect(decision).toMatchObject({ + decision: "continue", + source: "sql-write-intent", + decisionSource: "sql-write-intent", + bypassed: true, + bypassReasonCode: "bypass_sql_write_intent", + reasonCodes: ["bypass_sql_write_intent"] + }); + }); + it("keeps clarification behavior for non-sql delete wording", () => { const clarification = node.run("删除订单 where id = 1"); expect(clarification).toBeDefined(); expect(clarification?.reason).toContain("指标口径"); }); + + it("asks fallback clarification for very short input", () => { + const clarification = node.run("订单"); + expect(clarification).toBeDefined(); + expect(clarification?.reason).toContain("分析对象"); + expect(clarification?.reason).toContain("指标口径"); + expect(clarification?.reason).toContain("时间范围"); + }); + + it("returns fallback prompt when slot decision throws", () => { + const spy = jest + .spyOn(slotFillingContext, "decideSlotFilling") + .mockImplementation(() => { + throw new Error("boom"); + }); + try { + expect(node.run("统计近30天订单总数")).toMatchObject({ + reason: "槽位解析异常", + question: "请补充分析对象、指标口径和时间范围后重试。", + decision: "clarify", + triggerPath: "rule", + decisionSource: "exception-fallback", + bypassed: false, + confidenceLevel: "low", + missingCriticalSlots: ["subject", "metric", "time"], + reasonCodes: [ + "fallback_exception", + "missing_subject_slot", + "missing_metric_slot", + "missing_time_slot" + ] + }); + } finally { + spy.mockRestore(); + } + }); }); diff --git a/apps/backend/test/unit/delivery-contract.mapper.spec.ts b/apps/backend/test/unit/delivery-contract.mapper.spec.ts index e3f1f08..831e84d 100644 --- a/apps/backend/test/unit/delivery-contract.mapper.spec.ts +++ b/apps/backend/test/unit/delivery-contract.mapper.spec.ts @@ -198,6 +198,145 @@ describe("DeliveryContractMapper", () => { ); }); + it("maps clarification decision evidence from trace fields", () => { + const mapper = new DeliveryContractMapper(new SandboxRuntimeService()); + const run = createBaseRun({ + status: "clarification", + trace: { + runId: "run-delivery-unit", + provider: "volcengine", + retryCount: 0, + steps: [], + clarificationDecision: { + decision: "clarify", + triggerPath: "hybrid", + confidenceLevel: "low", + missingCriticalSlots: ["metric", "time"], + conflictDetected: true, + reasonCodes: ["rule_low_confidence", "context_conflict_detected"], + question: "请补充要统计的指标口径（例如订单数、退款金额、转化率）。", + reason: "关键槽位缺失：指标口径、时间范围" + } + } as SqlRun["trace"], + clarification: { + question: "请补充要统计的指标口径（例如订单数、退款金额、转化率）。", + reason: "关键槽位缺失：指标口径、时间范围" + } + }); + + const delivery = mapper.map({ + run, + replayRecords: [] + }); + + expect(delivery.evidence?.clarificationDecision).toEqual({ + decision: "clarify", + triggerPath: "hybrid", + confidenceLevel: "low", + missingCriticalSlots: ["metric", "time"], + conflictDetected: true, + reasonCodes: ["rule_low_confidence", "context_conflict_detected"], + question: "请补充要统计的指标口径（例如订单数、退款金额、转化率）。", + reason: "关键槽位缺失：指标口径、时间范围" + }); + }); + + it("reads snake_case clarification decision from clarify step summary", () => { + const mapper = new DeliveryContractMapper(new SandboxRuntimeService()); + const run = createBaseRun({ + status: "clarification", + trace: { + runId: "run-delivery-unit", + provider: "volcengine", + retryCount: 0, + steps: [ + { + node: "clarify", + status: "success", + at: "2026-04-18T00:00:00.500Z", + detail: "关键槽位缺失：时间范围", + outputSummary: JSON.stringify({ + clarification_question: "请补充时间范围（例如近30天、本季度或具体起止日期）。", + clarification_decision: { + decision: "clarify", + trigger_source: "rule", + confidence_level: "low", + missing_critical_slots: ["time"], + conflict_detected: "false", + reason_codes: ["missing_time_slot"] + } + }) + } + ] + } as SqlRun["trace"], + clarification: { + question: "请补充时间范围（例如近30天、本季度或具体起止日期）。", + reason: "关键槽位缺失：时间范围" + } + }); + + const delivery = mapper.map({ + run, + replayRecords: [] + }); + + expect(delivery.evidence?.clarificationDecision).toEqual({ + decision: "clarify", + triggerPath: "rule", + confidenceLevel: "low", + missingCriticalSlots: ["time"], + conflictDetected: false, + reasonCodes: ["missing_time_slot"], + question: "请补充时间范围（例如近30天、本季度或具体起止日期）。", + reason: "关键槽位缺失：时间范围" + }); + }); + + it("maps SQL coverage evidence from generate-sql step summary", () => { + const mapper = new DeliveryContractMapper(new SandboxRuntimeService()); + const run = createBaseRun({ + trace: { + runId: "run-delivery-unit", + provider: "volcengine", + retryCount: 0, + steps: [ + { + node: "generate-sql", + status: "success", + at: "2026-04-18T00:00:00.500Z", + outputSummary: JSON.stringify({ + coverage: { + gateStatus: "passed", + missingObjects: [], + triggerSource: "selected_context" + } + }) + } + ] + } as SqlRun["trace"] + }); + + const delivery = mapper.map({ + run, + replayRecords: [] + }); + + const evidenceWithCoverage = delivery.evidence as + | (NonNullable & { + sqlCoverage?: { + gateStatus: string; + missingObjects: string[]; + triggerSource: string; + }; + }) + | undefined; + expect(evidenceWithCoverage?.sqlCoverage).toEqual({ + gateStatus: "passed", + missingObjects: [], + triggerSource: "selected_context" + }); + }); + it("maps modelingRevision from trace into delivery evidence", () => { const mapper = new DeliveryContractMapper(new SandboxRuntimeService()); const run = createBaseRun({ @@ -441,6 +580,16 @@ describe("DeliveryContractMapper", () => { expect(delivery.answer.text).toBe("暂无可执行 SQL，已提供解释。"); expect(delivery.evidence?.runId).toBe(run.runId); + expect(delivery.evidence?.clarificationDecision).toBeUndefined(); + expect( + ( + delivery.evidence as + | (NonNullable & { + sqlCoverage?: unknown; + }) + | undefined + )?.sqlCoverage + ).toBeUndefined(); expect(delivery.artifact).toBeUndefined(); }); diff --git a/apps/backend/test/unit/langgraph-runtime.spec.ts b/apps/backend/test/unit/langgraph-runtime.spec.ts index c8c0f7d..51e744e 100644 --- a/apps/backend/test/unit/langgraph-runtime.spec.ts +++ b/apps/backend/test/unit/langgraph-runtime.spec.ts @@ -4,15 +4,69 @@ import { } from "../../src/modules/conversation/agent/graph/langgraph.state"; import { createLangGraphRuntime } from "../../src/modules/conversation/agent/graph/langgraph.runtime"; +const createMockIntentPlan = () => ({ + status: "ready" as const, + intent: "aggregate" as const, + constraints: [], + summary: "stub", + uncertaintySignal: { + level: "low" as const, + needsStrictSemanticPath: false, + reasonCodes: ["rule_slots_sufficient"] + }, + clarificationDecision: { + decision: "continue" as const, + triggerPath: "rule" as const, + confidenceLevel: "high" as const, + missingCriticalSlots: [], + conflictDetected: false, + reasonCodes: ["rule_slots_sufficient"], + question: "", + reason: "问题信息充足" + }, + riskTags: [], + planningWarnings: [] +}); + +const createMockSemanticPlan = () => ({ + status: "ready" as const, + semanticHints: [], + semanticVersion: 1, + lockStatus: "locked" as const, + fallbackApplied: false, + riskTags: [], + intentRiskTags: [], + semanticRiskTags: [], + planningWarnings: { + intent: [], + semantic: [] + }, + strictMode: false, + strictModeReasons: [], + summary: "stub" +}); + describe("langgraph runtime", () => { it("passes contextEnvelope into clarify node", async () => { const clarifyRun = jest.fn().mockReturnValue({ + decision: "clarify", + action: "ask_clarification", + source: "rule", + decisionSource: "rule", + triggerPath: "rule", + bypassed: false, + confidence: "low", + confidenceLevel: "low", + missingSlots: ["metric"], + shouldClarify: true, + missingCriticalSlots: ["metric"], + reasonCodes: ["missing_metric_slot"], reason: "need clarification", question: "请补充指标口径" }); const runtime = createLangGraphRuntime({ clarifyNode: { - run: clarifyRun + evaluate: clarifyRun }, retrieveKnowledgeNode: { run: async () => ({ @@ -22,23 +76,10 @@ describe("langgraph runtime", () => { }) }, buildIntentPlanNode: { - run: () => ({ - status: "ready", - intent: "aggregate", - constraints: [], - summary: "stub" - }) + run: async () => createMockIntentPlan() }, buildSemanticQueryNode: { - run: async () => ({ - status: "ready", - semanticHints: [], - semanticVersion: 1, - lockStatus: "locked" as const, - fallbackApplied: false, - riskTags: [], - summary: "stub" - }) + run: async () => createMockSemanticPlan() }, buildPhysicalPlanNode: { run: async () => ({ @@ -98,12 +139,28 @@ describe("langgraph runtime", () => { expect(clarifyRun).toHaveBeenCalledWith("这个趋势怎么样", contextEnvelope); expect(output.terminalStatus).toBe("clarification"); + expect(output.trace.clarificationDecision?.decision).toBe("clarify"); }); it("should run through executionResult path with expected node steps", async () => { const runtime = createLangGraphRuntime({ clarifyNode: { - run: () => undefined + evaluate: () => ({ + decision: "continue", + action: "proceed", + source: "rule", + decisionSource: "rule", + triggerPath: "rule", + bypassed: false, + confidence: "high", + confidenceLevel: "high", + missingSlots: [], + shouldClarify: false, + missingCriticalSlots: [], + reasonCodes: ["rule_slots_sufficient"], + reason: "问题信息充足", + question: "" + }) }, retrieveKnowledgeNode: { run: async () => ({ @@ -113,23 +170,10 @@ describe("langgraph runtime", () => { }) }, buildIntentPlanNode: { - run: () => ({ - status: "ready", - intent: "aggregate", - constraints: [], - summary: "stub" - }) + run: async () => createMockIntentPlan() }, buildSemanticQueryNode: { - run: async () => ({ - status: "ready", - semanticHints: [], - semanticVersion: 1, - lockStatus: "locked" as const, - fallbackApplied: false, - riskTags: [], - summary: "stub" - }) + run: async () => createMockSemanticPlan() }, buildPhysicalPlanNode: { run: async () => ({ @@ -203,6 +247,7 @@ describe("langgraph runtime", () => { expect(output.trace.steps[0]?.lifecycle).toBe("skipped"); expect(output.trace.steps[5]?.sequence).toBe(6); expect(output.trace.steps[5]?.lifecycle).toBe("completed"); + expect(output.trace.clarificationDecision?.decision).toBe("continue"); }); it("should normalize missing trace context", () => { @@ -215,7 +260,22 @@ describe("langgraph runtime", () => { it("should capture fatal llm error into state without throwing from runtime", async () => { const runtime = createLangGraphRuntime({ clarifyNode: { - run: () => undefined + evaluate: () => ({ + decision: "continue", + action: "proceed", + source: "rule", + decisionSource: "rule", + triggerPath: "rule", + bypassed: false, + confidence: "high", + confidenceLevel: "high", + missingSlots: [], + shouldClarify: false, + missingCriticalSlots: [], + reasonCodes: ["rule_slots_sufficient"], + reason: "问题信息充足", + question: "" + }) }, retrieveKnowledgeNode: { run: async () => ({ @@ -225,23 +285,10 @@ describe("langgraph runtime", () => { }) }, buildIntentPlanNode: { - run: () => ({ - status: "ready", - intent: "aggregate", - constraints: [], - summary: "stub" - }) + run: async () => createMockIntentPlan() }, buildSemanticQueryNode: { - run: async () => ({ - status: "ready", - semanticHints: [], - semanticVersion: 1, - lockStatus: "locked" as const, - fallbackApplied: false, - riskTags: [], - summary: "stub" - }) + run: async () => createMockSemanticPlan() }, buildPhysicalPlanNode: { run: async () => ({ diff --git a/apps/backend/test/unit/rag-rerank.service.spec.ts b/apps/backend/test/unit/rag-rerank.service.spec.ts index 2b51d6f..7a00c6e 100644 --- a/apps/backend/test/unit/rag-rerank.service.spec.ts +++ b/apps/backend/test/unit/rag-rerank.service.spec.ts @@ -176,4 +176,49 @@ describe("rag rerank service", () => { expect(response.retrieval_bundle.reranked?.[0]?.chunk_id).toBe("chunk-b"); expect(response.retrieval_bundle.reranked?.[0]?.secondary_score).toBeCloseTo(0.95); }); + + it("keeps retrieval column pruning evidence visible in rerank replay payload", async () => { + const adapter: Pick = { + rerank: jest.fn().mockResolvedValue([]) + }; + const replay: Pick = { + writeReplay: jest.fn().mockResolvedValue(undefined) + }; + const service = createService(adapter, replay); + const bundle = createBundle() as RagRetrievalBundle & { + column_pruning?: Record; + }; + bundle.column_pruning = { + strategy: "table_first_field_second_conservative", + status: "applied", + tables: [ + { + table_name: "orders", + mode: "conservative" + } + ] + }; + + await service.rerank({ + retrievalBundle: bundle, + secondaryMinCandidates: 10 + }); + + const finalReplayCall = (replay.writeReplay as jest.Mock).mock.calls + .map(([payload]) => payload) + .find((payload) => payload.replayKey === "rerank:final") as + | { + payload?: { + columnPruning?: { + status?: string; + tables?: Array<{ table_name?: string; mode?: string }>; + }; + }; + } + | undefined; + expect(finalReplayCall).toBeDefined(); + expect(finalReplayCall?.payload?.columnPruning?.status).toBe("applied"); + expect(finalReplayCall?.payload?.columnPruning?.tables?.[0]?.table_name).toBe("orders"); + expect(finalReplayCall?.payload?.columnPruning?.tables?.[0]?.mode).toBe("conservative"); + }); }); diff --git a/apps/backend/test/unit/slot-filling-context.spec.ts b/apps/backend/test/unit/slot-filling-context.spec.ts new file mode 100644 index 0000000..eb9b4ce --- /dev/null +++ b/apps/backend/test/unit/slot-filling-context.spec.ts @@ -0,0 +1,134 @@ +import { + buildFallbackSlotFillingDecision, + decideSlotFilling +} from "../../src/modules/conversation/agent/nodes/slot-filling-context"; + +describe("slot-filling-context", () => { + it("returns structured continue decision for complete question", () => { + const decision = decideSlotFilling("统计近30天订单总数"); + + expect(decision).toMatchObject({ + decision: "continue", + action: "proceed", + source: "rule", + decisionSource: "rule", + triggerPath: "rule", + bypassed: false, + confidence: "high", + confidenceLevel: "high", + missingSlots: [], + shouldClarify: false, + missingCriticalSlots: [], + reasonCodes: ["rule_slots_sufficient"] + }); + }); + + it("bypasses clarification for metadata intent", () => { + const decision = decideSlotFilling("show tables"); + + expect(decision).toMatchObject({ + decision: "continue", + action: "proceed", + source: "metadata-intent", + decisionSource: "metadata-intent", + bypassed: true, + bypassReasonCode: "bypass_metadata_intent", + confidence: "high", + shouldClarify: false, + missingSlots: [], + reasonCodes: ["bypass_metadata_intent"] + }); + }); + + it("bypasses clarification for sql write intent with reason code", () => { + const decision = decideSlotFilling("DELETE FROM orders where id = 1"); + + expect(decision).toMatchObject({ + decision: "continue", + action: "proceed", + source: "sql-write-intent", + decisionSource: "sql-write-intent", + bypassed: true, + bypassReasonCode: "bypass_sql_write_intent", + reasonCodes: ["bypass_sql_write_intent"] + }); + }); + + it("falls back to shortest clarification for very short input", () => { + const decision = decideSlotFilling("订单"); + + expect(decision).toMatchObject({ + decision: "clarify", + action: "ask_clarification", + source: "short-input-fallback", + decisionSource: "short-input-fallback", + bypassed: false, + confidence: "low", + confidenceLevel: "low", + shouldClarify: true, + missingSlots: ["subject", "metric", "time"], + reasonCodes: [ + "fallback_short_input", + "missing_subject_slot", + "missing_metric_slot", + "missing_time_slot" + ] + }); + expect(decision.reason).toContain("分析对象"); + expect(decision.reason).toContain("指标口径"); + expect(decision.reason).toContain("时间范围"); + }); + + it("asks for time when trend intent lacks time slot", () => { + const decision = decideSlotFilling("订单数量趋势怎么样"); + + expect(decision.shouldClarify).toBe(true); + expect(decision.missingSlots).toContain("time"); + expect(decision.source).toBe("rule"); + expect(decision.reasonCodes).toContain("missing_time_slot"); + }); + + it("uses context envelope to fill critical slots", () => { + const decision = decideSlotFilling("近30天趋势怎么样", { + metricDefinition: "订单总数", + entityMappings: [ + { + entity: "华北大区", + mappedTo: "region_north" + } + ], + timeRange: { + from: "2026-03-01", + to: "2026-03-31" + } + }); + + expect(decision).toMatchObject({ + decision: "continue", + action: "proceed", + source: "rule", + shouldClarify: false, + missingSlots: [], + reasonCodes: ["rule_slots_sufficient"] + }); + }); + + it("returns exception fallback decision shape", () => { + const decision = buildFallbackSlotFillingDecision(); + + expect(decision).toMatchObject({ + decision: "clarify", + action: "ask_clarification", + source: "exception-fallback", + confidence: "low", + shouldClarify: true, + missingSlots: ["subject", "metric", "time"], + reasonCodes: [ + "fallback_exception", + "missing_subject_slot", + "missing_metric_slot", + "missing_time_slot" + ] + }); + }); +}); diff --git a/apps/backend/test/unit/sql-generation.service.spec.ts b/apps/backend/test/unit/sql-generation.service.spec.ts index fdf099c..ed13ddf 100644 --- a/apps/backend/test/unit/sql-generation.service.spec.ts +++ b/apps/backend/test/unit/sql-generation.service.spec.ts @@ -167,4 +167,162 @@ describe("SqlGenerationService semantic guardrails", () => { expect(prompt.systemPrompt).toContain("Structured semantic instruction set"); expect(prompt.systemPrompt).toContain("Metric bindings: metric.gmv"); }); + + it("prefers selected-context pruning details and prints explainable source/rationale", async () => { + const providerRouter = { + generate: jest.fn().mockResolvedValue({ + provider: "mock-provider", + model: "mock-model", + rawText: "```sql\nSELECT SUM(amount) AS gmv FROM orders;\n```" + }), + stream: jest.fn() + }; + const service = createService(providerRouter); + + await service.generate("按 GMV 汇总订单", { + selectedContext: [ + { + chunk_id: "chunk-orders-pruned", + content: "orders schema snapshot with financial columns", + metadata: { + datasourceId: "ds-1", + indexVersionId: "idx-1", + chunkId: "chunk-orders-pruned", + domain: "schema_table", + tableNames: ["orders"], + columnNames: ["order_id", "amount", "status", "created_at"], + sourceMetadata: { + selected_context_pruning: { + source: "column_pruner_v2", + table: "orders", + selected_columns: [ + { + name: "order_id", + reason: "join key" + }, + { + name: "amount", + reason: "metric.gmv" + } + ], + reason: "semantic_metric_binding", + confidence: "low", + ambiguous: true + } + } + } + } + ] + }); + + const prompt = providerRouter.generate.mock.calls[0][0]; + expect(prompt.userPrompt).toContain( + "Selected-context pruning view (preferred when available):" + ); + expect(prompt.userPrompt).toContain("table=orders"); + expect(prompt.userPrompt).toContain("source=column_pruner_v2"); + expect(prompt.userPrompt).toContain("order_id (join key)"); + expect(prompt.userPrompt).toContain("amount (metric.gmv)"); + expect(prompt.userPrompt).toContain("rationale: semantic_metric_binding"); + expect(prompt.userPrompt).toContain("conservative_note:"); + expect(prompt.userPrompt).toContain("created_at"); + }); + + it("falls back to legacy context rendering when pruning payload is missing", async () => { + const providerRouter = { + generate: jest.fn().mockResolvedValue({ + provider: "mock-provider", + model: "mock-model", + rawText: "```sql\nSELECT order_id, amount FROM orders LIMIT 5;\n```" + }), + stream: jest.fn() + }; + const service = createService(providerRouter); + + await service.generate("查看订单金额", { + selectedContext: [ + { + chunk_id: "chunk-orders-legacy", + content: "orders schema: orders(order_id, amount, status)", + metadata: { + datasourceId: "ds-1", + indexVersionId: "idx-1", + chunkId: "chunk-orders-legacy", + domain: "schema_table", + tableNames: ["orders"], + columnNames: ["order_id", "amount", "status"], + sourceMetadata: {} + } + } + ] + }); + + const prompt = providerRouter.generate.mock.calls[0][0]; + expect(prompt.userPrompt).toContain("Retrieved context (trusted evidence):"); + expect(prompt.userPrompt).toContain( + "1. [schema_table] chunk-orders-legacy: orders schema: orders(order_id, amount, status)" + ); + expect(prompt.userPrompt).not.toContain( + "Selected-context pruning view (preferred when available):" + ); + }); + + it("passes evidence coverage gate when SQL tables/columns are covered by selected context", async () => { + const providerRouter = { + generate: jest.fn().mockResolvedValue({ + provider: "mock-provider", + model: "mock-model", + rawText: "```sql\nSELECT amount FROM orders;\n```" + }), + stream: jest.fn() + }; + const service = createService(providerRouter); + + const draft = await service.generate("查询订单金额", { + selectedContext: [ + { + chunk_id: "chunk-orders-coverage", + content: "orders(order_id, amount, created_at)", + metadata: { + datasourceId: "ds-1", + indexVersionId: "idx-1", + chunkId: "chunk-orders-coverage", + domain: "schema_table", + tableNames: ["orders"], + columnNames: ["order_id", "amount", "created_at"], + sourceMetadata: {} + } + } + ] + }); + + expect(draft.coverage?.gateStatus).toBe("passed"); + expect(draft.coverage?.missingObjects).toEqual([]); + expect(draft.coverage?.triggerSource).toBe("selected_context"); + }); + + it("fails evidence coverage gate when SQL references objects outside selected/semantic/pinning evidence", async () => { + const providerRouter = { + generate: jest.fn().mockResolvedValue({ + provider: "mock-provider", + model: "mock-model", + rawText: "```sql\nSELECT total_amount FROM invoices;\n```" + }), + stream: jest.fn() + }; + const service = createService(providerRouter); + + await expect( + service.generate("查询发票金额", { + selectedContext: [], + explicitPinning: { + source: "context_envelope", + tables: ["orders"], + columns: ["orders.amount"] + } + }) + ).rejects.toMatchObject>({ + code: "LLM_SQL_EVIDENCE_COVERAGE_FAILED" + }); +}); }); diff --git a/apps/frontend/src/app/data-sources/page.tsx b/apps/frontend/src/app/data-sources/page.tsx index 23ebc75..43441a0 100644 --- a/apps/frontend/src/app/data-sources/page.tsx +++ b/apps/frontend/src/app/data-sources/page.tsx @@ -596,6 +596,13 @@ function DataSourcesPageContent() { return; } const selectedTableNames = normalizeSelectedTableNames(setupWizard.selectedTableNames); + const previousSuggestionIds = normalizeSelectedIds(setupWizard.selectedSuggestionIds); + const previousSuggestionIdSet = new Set(previousSuggestionIds); + const previousAllSuggestionIdSet = new Set( + setupWizard.suggestions.map((item) => item.id) + ); + const hasExistingSuggestionSelectionState = + setupWizard.step === 2 || previousAllSuggestionIdSet.size > 0; if (selectedTableNames.length === 0) { setSetupWizard((previous) => ({ ...previous, @@ -637,13 +644,21 @@ function DataSourcesPageContent() { selectedTables: persistedTableNames } ); + const nextSelectedSuggestionIds = hasExistingSuggestionSelectionState + ? suggestions + .filter( + (item) => + previousSuggestionIdSet.has(item.id) || !previousAllSuggestionIdSet.has(item.id) + ) + .map((item) => item.id) + : suggestions.map((item) => item.id); setSetupWizard((previous) => ({ ...previous, step: 2, submitting: false, selectedTableNames: persistedTableNames, suggestions, - selectedSuggestionIds: suggestions.map((item) => item.id), + selectedSuggestionIds: nextSelectedSuggestionIds, failure: null })); } catch (error) { diff --git a/apps/frontend/src/app/settings/modeling/page.tsx b/apps/frontend/src/app/settings/modeling/page.tsx index 0ca7e18..cc4baf2 100644 --- a/apps/frontend/src/app/settings/modeling/page.tsx +++ b/apps/frontend/src/app/settings/modeling/page.tsx @@ -14,8 +14,10 @@ import { listWorkspaceDatasourceTablePermissions, listWorkspaces, precheckWorkspaceModelingDeploy, + recommendModelingSetupRelationships, resolveWorkspaceModelingSchemaChange, upsertWorkspaceModelingGraph, + type ModelingSetupRelationshipSuggestion, type PrecheckWorkspaceModelingDeployResult, type WorkspaceModelingGraphSnapshot } from "@/lib/admin-api-client"; @@ -222,6 +224,12 @@ type SaveModelingGraphOptions = { suppressSuccessMessage?: boolean; }; +type SyncRecommendedRelationshipsResult = { + nextPayload: ModelingGraphPayload; + addedCount: number; + skippedCount: number; +}; + type ModelingPositionPatch = { models: Record; views: Record; @@ -290,8 +298,221 @@ function mergePositionPatchIntoGraphPayload( }; } +function normalizeSignaturePart(value: string | undefined): string { + return value?.trim().toLowerCase() ?? ""; +} + +function normalizeRelationshipOperator(value: string | undefined): "eq" { + return normalizeSignaturePart(value) === "eq" ? "eq" : "eq"; +} + +function buildEndpointSignature(endpoint: { + dataset?: string; + table?: string; + column?: string; +}): string | null { + const dataset = normalizeSignaturePart(endpoint.dataset); + const table = normalizeSignaturePart(endpoint.table); + const column = normalizeSignaturePart(endpoint.column); + if (!table || !column) { + return null; + } + return `${dataset}|${table}|${column}`; +} + +function buildDirectedRelationshipSignature(input: { + left: { dataset?: string; table?: string; column?: string }; + right: { dataset?: string; table?: string; column?: string }; + operator?: string; +}): string | null { + const leftSignature = buildEndpointSignature(input.left); + const rightSignature = buildEndpointSignature(input.right); + if (!leftSignature || !rightSignature) { + return null; + } + const operator = normalizeSignaturePart(input.operator) || "eq"; + return `${leftSignature}->${rightSignature}|${operator}`; +} + +function buildUndirectedRelationshipSignature(input: { + left: { dataset?: string; table?: string; column?: string }; + right: { dataset?: string; table?: string; column?: string }; + operator?: string; +}): string | null { + const leftSignature = buildEndpointSignature(input.left); + const rightSignature = buildEndpointSignature(input.right); + if (!leftSignature || !rightSignature) { + return null; + } + const operator = normalizeSignaturePart(input.operator) || "eq"; + const [endpointA, endpointB] = [leftSignature, rightSignature].sort(); + return `${endpointA}<->${endpointB}|${operator}`; +} + +function resolveRecommendationRelationshipType( + recommendation: ModelingSetupRelationshipSuggestion +): ModelingGraphPayload["relationships"][number]["type"] { + const relationshipType = recommendation.type ?? recommendation.cardinality; + if ( + relationshipType === "many-to-one" || + relationshipType === "one-to-many" || + relationshipType === "one-to-one" + ) { + return relationshipType; + } + return "many-to-one"; +} + +function resolveRecommendationOperator( + recommendation: ModelingSetupRelationshipSuggestion +): "eq" { + const recommendationRecord = recommendation as ModelingSetupRelationshipSuggestion & { + operator?: string; + bridge?: { + operator?: string; + }; + }; + return normalizeRelationshipOperator( + recommendationRecord.bridge?.operator ?? recommendationRecord.operator + ); +} + +function mergeRecommendedRelationshipsIntoGraphPayload( + payload: ModelingGraphPayload, + recommendations: ModelingSetupRelationshipSuggestion[] +): SyncRecommendedRelationshipsResult { + const directedSignatureSet = new Set(); + const undirectedSignatureSet = new Set(); + + for (const relationship of payload.relationships) { + const directedSignature = buildDirectedRelationshipSignature({ + left: relationship.bridge.left, + right: relationship.bridge.right, + operator: relationship.bridge.operator + }); + if (directedSignature) { + directedSignatureSet.add(directedSignature); + } + const undirectedSignature = buildUndirectedRelationshipSignature({ + left: relationship.bridge.left, + right: relationship.bridge.right, + operator: relationship.bridge.operator + }); + if (undirectedSignature) { + undirectedSignatureSet.add(undirectedSignature); + } + } + + const additions: ModelingGraphPayload["relationships"] = []; + let skippedCount = 0; + + for (const recommendation of recommendations) { + const relationshipType = resolveRecommendationRelationshipType(recommendation); + const recommendationOperator = resolveRecommendationOperator(recommendation); + const nextRelationship: ModelingGraphPayload["relationships"][number] = { + id: recommendation.id, + name: recommendation.name, + source: recommendation.reason === "foreign_key_constraint" ? "fk" : "inferred", + confidence: Number(Math.max(0, Math.min(1, recommendation.confidence)).toFixed(4)), + type: relationshipType, + cardinality: relationshipType, + bridge: { + left: { + dataset: recommendation.left.dataset, + table: recommendation.left.table, + column: recommendation.left.column + }, + right: { + dataset: recommendation.right.dataset, + table: recommendation.right.table, + column: recommendation.right.column + }, + operator: recommendationOperator, + confidence: Number(Math.max(0, Math.min(1, recommendation.confidence)).toFixed(4)) + } + }; + const directedSignature = buildDirectedRelationshipSignature({ + left: nextRelationship.bridge.left, + right: nextRelationship.bridge.right, + operator: nextRelationship.bridge.operator + }); + const undirectedSignature = buildUndirectedRelationshipSignature({ + left: nextRelationship.bridge.left, + right: nextRelationship.bridge.right, + operator: nextRelationship.bridge.operator + }); + if (!directedSignature || !undirectedSignature) { + skippedCount += 1; + continue; + } + if ( + directedSignatureSet.has(directedSignature) || + undirectedSignatureSet.has(undirectedSignature) + ) { + skippedCount += 1; + continue; + } + directedSignatureSet.add(directedSignature); + undirectedSignatureSet.add(undirectedSignature); + additions.push(nextRelationship); + } + + if (additions.length === 0) { + return { + nextPayload: payload, + addedCount: 0, + skippedCount + }; + } + + return { + nextPayload: { + ...payload, + relationships: [...payload.relationships, ...additions] + }, + addedCount: additions.length, + skippedCount + }; +} + +function normalizeModelTableNamesForRecommendation(payload: ModelingGraphPayload): string[] { + const normalized = new Set(); + for (const model of payload.models) { + const tableName = model.tableName?.trim().toLowerCase(); + if (!tableName) { + continue; + } + normalized.add(tableName); + } + return Array.from(normalized).sort((left, right) => left.localeCompare(right)); +} + +function buildBootstrapAttemptDraftKey(input: { + workspaceId: string; + datasourceId: string; + snapshot: WorkspaceModelingGraphSnapshot | null; + payload: ModelingGraphPayload; +}): string | null { + const modelTableNames = normalizeModelTableNamesForRecommendation(input.payload); + if (modelTableNames.length === 0 || input.payload.relationships.length > 0) { + return null; + } + const draftRevision = input.snapshot?.draft?.revision ?? "none"; + const draftGraphHash = input.snapshot?.draft?.graphHash ?? "none"; + return [ + input.workspaceId, + input.datasourceId, + draftRevision, + draftGraphHash, + modelTableNames.join(",") + ].join("|"); +} + export default function ModelingWorkspacePage() { const showDetailsPanel = process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL === "true"; + const showSchemaAndDeployPanels = + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS === "true"; + const showContextDrawer = showDetailsPanel || showSchemaAndDeployPanels; const [workspaceId, setWorkspaceId] = useState(""); const [datasourceId, setDatasourceId] = useState(""); const [policyVersion, setPolicyVersion] = useState(0); @@ -311,6 +532,7 @@ export default function ModelingWorkspacePage() { const [hasPendingDraftChanges, setHasPendingDraftChanges] = useState(false); const [busy, setBusy] = useState(false); const [saving, setSaving] = useState(false); + const [syncingRecommendations, setSyncingRecommendations] = useState(false); const [deploying, setDeploying] = useState(false); const [schemaChangeItems, setSchemaChangeItems] = useState< Array<{ @@ -328,6 +550,8 @@ export default function ModelingWorkspacePage() { const latestSnapshotLoadTokenRef = useRef(0); const positionPatchChangedRef = useRef(false); const nextEditorIntentRequestIdRef = useRef(1); + const graphPayloadRef = useRef(createEmptyGraphPayload()); + const bootstrapAttemptedDraftKeysRef = useRef(new Set()); const scrollToLayoutPane = useCallback((paneId: string): void => { if (typeof window === "undefined") { @@ -505,6 +729,10 @@ export default function ModelingWorkspacePage() { setDeployPrecheck(null); }, [graphPayload]); + useEffect(() => { + graphPayloadRef.current = graphPayload; + }, [graphPayload]); + useEffect(() => { if (!message) { return; @@ -573,87 +801,179 @@ export default function ModelingWorkspacePage() { return undefined; }, [graphPayload.models, graphPayload.relationships, selectedNode]); - const saveModelingGraph = async ( - options: SaveModelingGraphOptions = {} - ): Promise => { - if (!workspaceId || !datasourceId) { - setError("请先选择工作空间与数据源。"); - return null; - } - const graphPayloadToSave = { - models: graphPayload.models, - relationships: graphPayload.relationships, - calculatedFields: graphPayload.calculatedFields, - views: graphPayload.views, - schemaChanges: graphPayload.schemaChanges - }; - const applySavedSnapshot = ( - nextSnapshot: WorkspaceModelingGraphSnapshot, - fallbackPolicyVersion: number - ): void => { - const nextGraphPayload = normalizeGraphPayload(nextSnapshot.draft?.graphPayload); - positionPatchChangedRef.current = false; - setSnapshot(nextSnapshot); - setGraphPayload(nextGraphPayload); - setPolicyVersion(nextSnapshot.draft?.policyVersion ?? fallbackPolicyVersion); - setSelectedNode((previous) => resolveNextSelectedNode(nextGraphPayload, previous)); - setDetailsDirty(false); - setHasPendingDraftChanges(false); - setDeployPrecheck(null); - }; - setSaving(true); - setError(""); - setMessage(""); - try { - const nextSnapshot = await upsertWorkspaceModelingGraph(workspaceId, datasourceId, { - policyVersion, - ...graphPayloadToSave - }); - applySavedSnapshot(nextSnapshot, policyVersion); - if (!options.suppressSuccessMessage) { - setMessage(`Modeling Draft 已保存（revision=${nextSnapshot.draft?.revision ?? "-"}）。`); + const persistModelingGraphPayload = useCallback( + async ( + graphPayloadToSave: ModelingGraphPayload, + options: SaveModelingGraphOptions = {} + ): Promise => { + if (!workspaceId || !datasourceId) { + setError("请先选择工作空间与数据源。"); + return null; } - return nextSnapshot; - } catch (submitError) { - if (isPolicyVersionConflictError(submitError)) { - try { - const latestPermissions = await listWorkspaceDatasourceTablePermissions( - workspaceId, - datasourceId - ); - const latestPolicyVersion = latestPermissions.policyVersion; - if (latestPolicyVersion !== policyVersion) { - const retriedSnapshot = await upsertWorkspaceModelingGraph( + const applySavedSnapshot = ( + nextSnapshot: WorkspaceModelingGraphSnapshot, + fallbackPolicyVersion: number + ): void => { + const nextGraphPayload = normalizeGraphPayload(nextSnapshot.draft?.graphPayload); + positionPatchChangedRef.current = false; + setSnapshot(nextSnapshot); + setGraphPayload(nextGraphPayload); + setPolicyVersion(nextSnapshot.draft?.policyVersion ?? fallbackPolicyVersion); + setSelectedNode((previous) => resolveNextSelectedNode(nextGraphPayload, previous)); + setDetailsDirty(false); + setHasPendingDraftChanges(false); + setDeployPrecheck(null); + }; + setSaving(true); + setError(""); + setMessage(""); + try { + const nextSnapshot = await upsertWorkspaceModelingGraph(workspaceId, datasourceId, { + policyVersion, + ...graphPayloadToSave + }); + applySavedSnapshot(nextSnapshot, policyVersion); + if (!options.suppressSuccessMessage) { + setMessage(`Modeling Draft 已保存（revision=${nextSnapshot.draft?.revision ?? "-"}）。`); + } + return nextSnapshot; + } catch (submitError) { + if (isPolicyVersionConflictError(submitError)) { + try { + const latestPermissions = await listWorkspaceDatasourceTablePermissions( workspaceId, - datasourceId, - { - policyVersion: latestPolicyVersion, - ...graphPayloadToSave - } + datasourceId ); - applySavedSnapshot(retriedSnapshot, latestPolicyVersion); - if (!options.suppressSuccessMessage) { - setMessage( - `policyVersion 已从 ${policyVersion} 更新为 ${latestPolicyVersion}，已自动重试并保存成功（revision=${retriedSnapshot.draft?.revision ?? "-"}）。` + const latestPolicyVersion = latestPermissions.policyVersion; + if (latestPolicyVersion !== policyVersion) { + const retriedSnapshot = await upsertWorkspaceModelingGraph( + workspaceId, + datasourceId, + { + policyVersion: latestPolicyVersion, + ...graphPayloadToSave + } ); + applySavedSnapshot(retriedSnapshot, latestPolicyVersion); + if (!options.suppressSuccessMessage) { + setMessage( + `policyVersion 已从 ${policyVersion} 更新为 ${latestPolicyVersion}，已自动重试并保存成功（revision=${retriedSnapshot.draft?.revision ?? "-"}）。` + ); + } + return retriedSnapshot; } - return retriedSnapshot; + } catch (retryError) { + setError( + retryError instanceof Error + ? `policyVersion 自动刷新重试失败：${retryError.message}` + : "policyVersion 自动刷新重试失败，请刷新后重试。" + ); + return null; } - } catch (retryError) { - setError( - retryError instanceof Error - ? `policyVersion 自动刷新重试失败：${retryError.message}` - : "policyVersion 自动刷新重试失败，请刷新后重试。" - ); - return null; } + setError(submitError instanceof Error ? submitError.message : "保存建模图失败"); + return null; + } finally { + setSaving(false); } - setError(submitError instanceof Error ? submitError.message : "保存建模图失败"); - return null; - } finally { - setSaving(false); + }, + [datasourceId, policyVersion, workspaceId] + ); + + const saveModelingGraph = useCallback( + async (options: SaveModelingGraphOptions = {}): Promise => + persistModelingGraphPayload( + { + models: graphPayload.models, + relationships: graphPayload.relationships, + calculatedFields: graphPayload.calculatedFields, + views: graphPayload.views, + schemaChanges: graphPayload.schemaChanges + }, + options + ), + [graphPayload, persistModelingGraphPayload] + ); + + const syncRecommendedRelationships = useCallback( + async (mode: "bootstrap" | "manual"): Promise => { + if (!workspaceId || !datasourceId) { + if (mode === "manual") { + setError("请先选择工作空间与数据源。"); + } + return; + } + const currentPayload = graphPayloadRef.current; + const selectedTableNames = normalizeModelTableNamesForRecommendation(currentPayload); + if (selectedTableNames.length === 0) { + if (mode === "manual") { + setMessage("当前草稿缺少可同步的模型表，请先创建或导入模型。"); + } + return; + } + setSyncingRecommendations(true); + setError(""); + if (mode === "manual") { + setMessage(""); + } + try { + const recommendations = await recommendModelingSetupRelationships( + workspaceId, + datasourceId, + { + selectedTables: selectedTableNames + } + ); + const mergeResult = mergeRecommendedRelationshipsIntoGraphPayload( + currentPayload, + recommendations + ); + if (mergeResult.addedCount === 0) { + if (mode === "manual") { + setMessage(`同步数据库完成：无新增关系（跳过 ${mergeResult.skippedCount} 条）。`); + } + return; + } + const savedSnapshot = await persistModelingGraphPayload(mergeResult.nextPayload, { + suppressSuccessMessage: true + }); + if (!savedSnapshot) { + return; + } + setMessage( + mode === "bootstrap" + ? `已自动补全 ${mergeResult.addedCount} 条关系（跳过 ${mergeResult.skippedCount} 条）。` + : `同步数据库完成：新增 ${mergeResult.addedCount} 条，跳过 ${mergeResult.skippedCount} 条。` + ); + } catch (syncError) { + const fallbackMessage = mode === "bootstrap" ? "自动补全关系失败" : "同步数据库失败"; + setError(syncError instanceof Error ? syncError.message : fallbackMessage); + } finally { + setSyncingRecommendations(false); + } + }, + [datasourceId, persistModelingGraphPayload, workspaceId] + ); + + useEffect(() => { + if (!workspaceId || !datasourceId) { + return; } - }; + const nextAttemptDraftKey = buildBootstrapAttemptDraftKey({ + workspaceId, + datasourceId, + snapshot, + payload: graphPayload + }); + if (!nextAttemptDraftKey) { + return; + } + if (bootstrapAttemptedDraftKeysRef.current.has(nextAttemptDraftKey)) { + return; + } + bootstrapAttemptedDraftKeysRef.current.add(nextAttemptDraftKey); + void syncRecommendedRelationships("bootstrap"); + }, [datasourceId, graphPayload, snapshot, syncRecommendedRelationships, workspaceId]); const publishModelingDraft = async (): Promise => { if (!workspaceId || !datasourceId) { @@ -823,7 +1143,8 @@ export default function ModelingWorkspacePage() { return true; } const shouldOpenContextDrawer = - showDetailsPanel && (nextNode.kind === "view" || nextNode.kind === "relationship"); + nextNode.kind === "relationship" || + (showDetailsPanel && nextNode.kind === "view"); const isSame = selectedNode?.kind === nextNode.kind && selectedNode?.id === nextNode.id; if (isSame) { @@ -1107,12 +1428,35 @@ export default function ModelingWorkspacePage() {
+ @@ -1218,7 +1562,7 @@ export default function ModelingWorkspacePage() { - { - if (open) { - setContextDrawerOpen(true); - return; - } - if (detailsDirty && typeof window !== "undefined") { - const confirmed = window.confirm("当前详情面板有未保存改动，确认关闭 Context Drawer 吗？"); - if (!confirmed) { + {showContextDrawer ? ( + { + if (open) { + setContextDrawerOpen(true); return; } - } - setContextDrawerOpen(false); - }} - > - {showDetailsPanel ? ( - { - setGraphPayload((previous) => { - if (node.kind === "model") { + if (detailsDirty && typeof window !== "undefined") { + const confirmed = window.confirm("当前详情面板有未保存改动，确认关闭 Context Drawer 吗？"); + if (!confirmed) { + return; + } + } + setContextDrawerOpen(false); + }} + > + {showDetailsPanel ? ( + { + setGraphPayload((previous) => { + if (node.kind === "model") { + return { + ...previous, + models: previous.models.map((item) => + item.id === node.id + ? { + ...item, + displayName: input.displayName, + description: input.description + } + : item + ) + }; + } return { ...previous, - models: previous.models.map((item) => + views: previous.views.map((item) => item.id === node.id ? { ...item, @@ -1285,66 +1643,58 @@ export default function ModelingWorkspacePage() { : item ) }; - } - return { + }); + setHasPendingDraftChanges(true); + setDeployPrecheck(null); + setMessage("Metadata 已更新，点击“保存 Modeling Draft”后提交。"); + }} + onCalculatedFieldsSave={async (fields) => { + setGraphPayload((previous) => ({ ...previous, - views: previous.views.map((item) => - item.id === node.id - ? { - ...item, - displayName: input.displayName, - description: input.description - } - : item - ) - }; - }); - setHasPendingDraftChanges(true); - setDeployPrecheck(null); - setMessage("Metadata 已更新，点击“保存 Modeling Draft”后提交。"); - }} - onCalculatedFieldsSave={async (fields) => { - setGraphPayload((previous) => ({ - ...previous, - calculatedFields: fields - })); - setHasPendingDraftChanges(true); - setDeployPrecheck(null); - setMessage("Calculated Fields 已更新，点击“保存 Modeling Draft”后提交。"); - }} - onRelationshipsSave={handleRelationshipsSave} - onLoadPreview={loadPreviewForDetails} - onDeleteTarget={async ({ targetKind, targetId }) => { - if (targetKind === "model") { - deleteModelFromSidebar(targetId); - return; - } - deleteViewFromSidebar(targetId); - }} - onSelectRelationship={handleSelectRelationship} - /> - ) : null} - - item.status === "detected").length} - onDetect={detectSchemaChanges} - onResolve={resolveSchemaChange} - /> + calculatedFields: fields + })); + setHasPendingDraftChanges(true); + setDeployPrecheck(null); + setMessage("Calculated Fields 已更新，点击“保存 Modeling Draft”后提交。"); + }} + onRelationshipsSave={handleRelationshipsSave} + onLoadPreview={loadPreviewForDetails} + onDeleteTarget={async ({ targetKind, targetId }) => { + if (targetKind === "model") { + deleteModelFromSidebar(targetId); + return; + } + deleteViewFromSidebar(targetId); + }} + onSelectRelationship={handleSelectRelationship} + /> + ) : null} - - + {showSchemaAndDeployPanels ? ( + <> + item.status === "detected").length} + onDetect={detectSchemaChanges} + onResolve={resolveSchemaChange} + /> + + + + ) : null} + + ) : null}
diff --git a/apps/frontend/src/components/data-sources/setup-relationships-step.tsx b/apps/frontend/src/components/data-sources/setup-relationships-step.tsx index f024526..0b132e0 100644 --- a/apps/frontend/src/components/data-sources/setup-relationships-step.tsx +++ b/apps/frontend/src/components/data-sources/setup-relationships-step.tsx @@ -17,6 +17,40 @@ function dedupe(value: string[]): string[] { return Array.from(new Set(value.map((item) => item.trim()).filter(Boolean))); } +function resolveCardinalityBadge( + cardinality: ModelingSetupRelationshipSuggestion["cardinality"] +): "1:N" | "1:1" { + if (cardinality === "one-to-one") { + return "1:1"; + } + return "1:N"; +} + +function resolveDirectionText( + cardinality: ModelingSetupRelationshipSuggestion["cardinality"] +): "N -> 1" | "1 -> N" | "1 -> 1" { + if (cardinality === "one-to-many") { + return "1 -> N"; + } + if (cardinality === "one-to-one") { + return "1 -> 1"; + } + return "N -> 1"; +} + +function resolveSourceLabel(reason?: string): string { + if (reason === "foreign_key_constraint") { + return "FK"; + } + if (reason === "fk_naming_suffix") { + return "命名推断"; + } + if (reason?.trim()) { + return reason.trim(); + } + return "推断"; +} + export function SetupRelationshipsStep({ suggestions, selectedSuggestionIds, @@ -89,14 +123,22 @@ export function SetupRelationshipsStep({ aria-label={`选择关系建议 ${item.name}`} />
-

{item.name}

+
+

{item.name}

+ + {resolveCardinalityBadge(item.cardinality)} + + + {resolveSourceLabel(item.reason)} + +

- {item.left.dataset}.{item.left.table}.{item.left.column} ={" "} + Bridge: {item.left.dataset}.{item.left.table}.{item.left.column} ={" "} {item.right.dataset}.{item.right.table}.{item.right.column}

- {item.reason ? ( -

依据：{item.reason}

- ) : null} +

+ 方向 {resolveDirectionText(item.cardinality)} +

置信度 {(item.confidence * 100).toFixed(0)}% @@ -108,4 +150,3 @@ export function SetupRelationshipsStep({ ); } - diff --git a/apps/frontend/src/components/settings/modeling/modeling-flow-canvas.tsx b/apps/frontend/src/components/settings/modeling/modeling-flow-canvas.tsx index 3e4c9e1..3da1aaf 100644 --- a/apps/frontend/src/components/settings/modeling/modeling-flow-canvas.tsx +++ b/apps/frontend/src/components/settings/modeling/modeling-flow-canvas.tsx @@ -81,6 +81,47 @@ export type ModelingFlowNodeActionEvent = { action: ModelingFlowNodeAction; }; +function areStringArraysEqual(left: string[], right: string[]): boolean { + if (left.length !== right.length) { + return false; + } + return left.every((value, index) => value === right[index]); +} + +function resolveHighlightedRelationshipIdsForNode( + nodeData: ModelingFlowNodeData, + selectedRelationshipId: string +): string[] { + const normalizedRelationshipId = selectedRelationshipId.trim(); + if (!normalizedRelationshipId || nodeData.kind !== "model") { + return []; + } + const hasRelationshipActionId = (nodeData.relationshipActionIds ?? []).some( + (relationshipActionId) => relationshipActionId === normalizedRelationshipId + ); + return hasRelationshipActionId ? [normalizedRelationshipId] : []; +} + +function decorateFlowEdgeWithSelectionState( + edge: Edge, + selectedRelationshipId: string, + selectedModelFlowNodeId: string +): Edge { + const isSelectedRelationship = edge.id === selectedRelationshipId; + const isIncidentToSelectedModel = Boolean(selectedModelFlowNodeId) && + (edge.source === selectedModelFlowNodeId || edge.target === selectedModelFlowNodeId); + const previousData = edge.data ?? ({} as ModelingFlowEdgeData); + return { + ...edge, + selected: isSelectedRelationship, + data: { + ...previousData, + highlightedBySelectedRelationship: isSelectedRelationship, + highlightedBySelectedModel: isIncidentToSelectedModel + } + }; +} + function resolveModelLabel(model: ModelingGraphPayload["models"][number]): string { return model.displayName?.trim() || model.modelName?.trim() || model.tableName; } @@ -960,6 +1001,7 @@ export function ModelingFlowCanvas(props: { selectedNode: ModelingSidebarNode | null; busy?: boolean; autoLayoutKey: string; + programmaticAutoLayoutRequestId?: number | null; onSelectNode: (node: ModelingSidebarNode | null) => void; onNodeAction?: (event: ModelingFlowNodeActionEvent) => void; onNodePositionsChange?: (patch: ModelingFlowPositionPatch) => void; @@ -969,6 +1011,7 @@ export function ModelingFlowCanvas(props: { selectedNode, busy, autoLayoutKey, + programmaticAutoLayoutRequestId, onSelectNode, onNodeAction, onNodePositionsChange @@ -983,8 +1026,11 @@ export function ModelingFlowCanvas(props: { const [layoutBusy, setLayoutBusy] = useState(false); const [layoutMessage, setLayoutMessage] = useState(""); const [layoutMessageVariant, setLayoutMessageVariant] = useState<"success" | "error">("success"); + const [pendingProgrammaticAutoLayoutRequestId, setPendingProgrammaticAutoLayoutRequestId] = + useState(null); const selectionFocusKeyRef = useRef(""); const latestLayoutRunIdRef = useRef(0); + const latestProgrammaticAutoLayoutRequestIdRef = useRef(null); const graph = useMemo(() => { try { @@ -1014,6 +1060,7 @@ export function ModelingFlowCanvas(props: { selectedNode?.kind === "model" || selectedNode?.kind === "view" ? toFlowNodeId(selectedNode) : ""; + const selectedModelFlowNodeId = selectedNode?.kind === "model" ? toFlowNodeId(selectedNode) : ""; const selectedRelationshipId = selectedNode?.kind === "relationship" ? selectedNode.id : ""; useEffect(() => { @@ -1024,54 +1071,90 @@ export function ModelingFlowCanvas(props: { const nextNodes = graph.nodes.map((node) => ({ ...node, selected: node.id === selectedFlowNodeId, + data: { + ...node.data, + highlightedRelationshipIds: resolveHighlightedRelationshipIdsForNode( + node.data, + selectedRelationshipId + ) + }, position: persistedPositionByNodeId.get(node.id) ?? previousPositionById.get(node.id) ?? node.position })); return nextNodes; }); setFlowEdges( - graph.edges.map((edge) => ({ - ...edge, - selected: edge.id === selectedRelationshipId - })) + graph.edges.map((edge) => + decorateFlowEdgeWithSelectionState(edge, selectedRelationshipId, selectedModelFlowNodeId) + ) ); - }, [graph, persistedPositionByNodeId, selectedFlowNodeId, selectedRelationshipId]); + }, [ + graph, + persistedPositionByNodeId, + selectedFlowNodeId, + selectedModelFlowNodeId, + selectedRelationshipId + ]); useEffect(() => { setFlowNodes((previousNodes) => previousNodes.map((node) => { const isSelected = node.id === selectedFlowNodeId; - if (node.selected === isSelected) { + const nextHighlightedRelationshipIds = resolveHighlightedRelationshipIdsForNode( + node.data, + selectedRelationshipId + ); + const previousHighlightedRelationshipIds = node.data.highlightedRelationshipIds ?? []; + const highlightedRelationshipChanged = !areStringArraysEqual( + previousHighlightedRelationshipIds, + nextHighlightedRelationshipIds + ); + if (node.selected === isSelected && !highlightedRelationshipChanged) { return node; } return { ...node, - selected: isSelected + selected: isSelected, + data: highlightedRelationshipChanged + ? { + ...node.data, + highlightedRelationshipIds: nextHighlightedRelationshipIds + } + : node.data }; }) ); setFlowEdges((previousEdges) => - previousEdges.map((edge) => { - const isSelected = edge.id === selectedRelationshipId; - if (edge.selected === isSelected) { - return edge; - } - return { - ...edge, - selected: isSelected - }; - }) + previousEdges.map((edge) => + decorateFlowEdgeWithSelectionState(edge, selectedRelationshipId, selectedModelFlowNodeId) + ) ); - }, [selectedFlowNodeId, selectedRelationshipId]); + }, [selectedFlowNodeId, selectedModelFlowNodeId, selectedRelationshipId]); useEffect(() => { setDidAutoFit(false); selectionFocusKeyRef.current = ""; latestLayoutRunIdRef.current += 1; + latestProgrammaticAutoLayoutRequestIdRef.current = null; + setPendingProgrammaticAutoLayoutRequestId(null); setLayoutBusy(false); setLayoutMessage(""); }, [autoLayoutKey]); + useEffect(() => { + if ( + typeof programmaticAutoLayoutRequestId !== "number" || + !Number.isFinite(programmaticAutoLayoutRequestId) + ) { + return; + } + if (latestProgrammaticAutoLayoutRequestIdRef.current === programmaticAutoLayoutRequestId) { + return; + } + latestProgrammaticAutoLayoutRequestIdRef.current = programmaticAutoLayoutRequestId; + setPendingProgrammaticAutoLayoutRequestId(programmaticAutoLayoutRequestId); + }, [programmaticAutoLayoutRequestId]); + useEffect(() => { if (!layoutMessage || layoutMessageVariant !== "success") { return; @@ -1209,6 +1292,22 @@ export function ModelingFlowCanvas(props: { } }, [busy, flowEdges, flowInstance, flowNodes, graph.handleLookup, layoutBusy, onNodePositionsChange]); + useEffect(() => { + if (pendingProgrammaticAutoLayoutRequestId === null || busy || layoutBusy || flowNodes.length < 2) { + return; + } + setPendingProgrammaticAutoLayoutRequestId((currentRequestId) => + currentRequestId === pendingProgrammaticAutoLayoutRequestId ? null : currentRequestId + ); + void handleAutoLayout(); + }, [ + busy, + flowNodes.length, + handleAutoLayout, + layoutBusy, + pendingProgrammaticAutoLayoutRequestId + ]); + const handleNodeClick = useCallback( (_event: unknown, node: Node) => { const nextNode = fromFlowNodeId(node.id); diff --git a/apps/frontend/src/components/settings/modeling/modeling-flow-edge.tsx b/apps/frontend/src/components/settings/modeling/modeling-flow-edge.tsx index 9e3942e..56ed87c 100644 --- a/apps/frontend/src/components/settings/modeling/modeling-flow-edge.tsx +++ b/apps/frontend/src/components/settings/modeling/modeling-flow-edge.tsx @@ -5,7 +5,7 @@ import { getSmoothStepPath, type EdgeProps } from "@xyflow/react"; -import { useMemo, useState } from "react"; +import { useState } from "react"; import type { ModelingGraphRelationshipType } from "@text2sql/shared-types"; import { cn } from "@/lib/utils"; @@ -29,8 +29,17 @@ export type ModelingFlowEdgeData = { }; description?: string; invalid?: boolean; + highlightedBySelectedRelationship?: boolean; + highlightedBySelectedModel?: boolean; }; +type ModelingFlowEdgeVisualState = + | "invalid" + | "selected-relationship" + | "selected-model-incident" + | "inferred-or-low-confidence" + | "normal"; + function resolveEdgeRelationshipType( edgeData?: ModelingFlowEdgeData ): ModelingGraphRelationshipType | "many-to-many" | null { @@ -82,6 +91,34 @@ function resolveEndpointMarkers( }; } +function resolveCardinalityBadgeText(value: "one" | "many" | "none"): string { + if (value === "one") { + return "1"; + } + if (value === "many") { + return "N"; + } + return ""; +} + +function resolveCardinalityBadgeOffset( + position: string | null | undefined +): { x: number; y: number } { + if (position === "left") { + return { x: -14, y: 0 }; + } + if (position === "right") { + return { x: 14, y: 0 }; + } + if (position === "top") { + return { x: 0, y: -14 }; + } + if (position === "bottom") { + return { x: 0, y: 14 }; + } + return { x: 0, y: 0 }; +} + function resolveRelationshipTypeLabel( relationshipType: ModelingGraphRelationshipType | "many-to-many" | null ): string { @@ -109,6 +146,29 @@ function formatEndpointPath( return `${endpoint.dataset}.${endpoint.table}.${endpoint.column}`; } +function resolveEdgeVisualState(params: { + invalid: boolean; + selectedRelationship: boolean; + selectedModelIncident: boolean; + inferred: boolean; + lowConfidence: boolean; +}): ModelingFlowEdgeVisualState { + const { invalid, selectedRelationship, selectedModelIncident, inferred, lowConfidence } = params; + if (invalid) { + return "invalid"; + } + if (selectedRelationship) { + return "selected-relationship"; + } + if (selectedModelIncident) { + return "selected-model-incident"; + } + if (inferred || lowConfidence) { + return "inferred-or-low-confidence"; + } + return "normal"; +} + export function ModelingFlowEdge({ id, sourceX, @@ -127,38 +187,40 @@ export function ModelingFlowEdge({ const markerType = resolveEndpointMarkers(relationshipType); const relationshipTypeLabel = resolveRelationshipTypeLabel(relationshipType); const isInvalid = Boolean(edgeData?.invalid); + const isInferred = edgeData?.source === "inferred"; const isLowConfidence = !isInvalid && confidence < 0.6; - const hasDash = !selected && (isInvalid || edgeData?.source === "inferred" || isLowConfidence); - const confidenceBand = isInvalid ? "invalid" : isLowConfidence ? "low" : "normal"; + const selectedRelationship = Boolean(selected || edgeData?.highlightedBySelectedRelationship); + const selectedModelIncident = Boolean(edgeData?.highlightedBySelectedModel) && !selectedRelationship; + const visualState = resolveEdgeVisualState({ + invalid: isInvalid, + selectedRelationship, + selectedModelIncident, + inferred: isInferred, + lowConfidence: isLowConfidence + }); + const hasDash = visualState === "invalid" || visualState === "inferred-or-low-confidence"; + const confidenceBand = isInvalid + ? "invalid" + : isLowConfidence + ? "low" + : isInferred + ? "inferred" + : "normal"; const fromPath = formatEndpointPath(edgeData?.from); const toPath = formatEndpointPath(edgeData?.to); - const markerColor = isInvalid - ? selected + const sourceBadgeText = resolveCardinalityBadgeText(markerType.source); + const targetBadgeText = resolveCardinalityBadgeText(markerType.target); + const sourceBadgeOffset = resolveCardinalityBadgeOffset(sourcePosition); + const targetBadgeOffset = resolveCardinalityBadgeOffset(targetPosition); + const badgeColor = + visualState === "invalid" ? "#dc2626" - : "#ef4444" - : isLowConfidence - ? "#d97706" - : selected + : visualState === "selected-relationship" ? "var(--action-primary)" - : "var(--border-strong)"; - const edgeMarkerIdPrefix = useMemo( - () => `modeling-flow-edge-${id.replace(/[^a-zA-Z0-9_-]/g, "_")}`, - [id] - ); - const oneMarkerId = `${edgeMarkerIdPrefix}-one`; - const manyMarkerId = `${edgeMarkerIdPrefix}-many`; - const markerStart = - markerType.source === "one" - ? `url(#${oneMarkerId})` - : markerType.source === "many" - ? `url(#${manyMarkerId})` - : undefined; - const markerEnd = - markerType.target === "one" - ? `url(#${oneMarkerId})` - : markerType.target === "many" - ? `url(#${manyMarkerId})` - : undefined; + : isLowConfidence + ? "#b45309" + : "#64748b"; + const badgeRadius = 10; const shouldShowDetailCard = Boolean(edgeData && (hovered || selected)); const [path, labelX, labelY] = getSmoothStepPath({ @@ -174,54 +236,23 @@ export function ModelingFlowEdge({ return ( <> - - - - - - - - { setHovered(true); @@ -230,28 +261,73 @@ export function ModelingFlowEdge({ setHovered(false); }} data-confidence-band={confidenceBand} - data-selected={selected ? "true" : "false"} + data-selected={selectedRelationship ? "true" : "false"} + data-visual-state={visualState} data-testid="modeling-flow-edge-path" /> + {sourceBadgeText ? ( + + + + {sourceBadgeText} + + + ) : null} + {targetBadgeText ? ( + + + + {targetBadgeText} + + + ) : null} {shouldShowDetailCard ? ( - -
+ +
Relationship
-
+

From

-

{fromPath}

+

+ {fromPath} +

To

-

{toPath}

+

+ {toPath} +

diff --git a/apps/frontend/src/components/settings/modeling/modeling-flow-node.tsx b/apps/frontend/src/components/settings/modeling/modeling-flow-node.tsx index 8c3524c..bec1ba6 100644 --- a/apps/frontend/src/components/settings/modeling/modeling-flow-node.tsx +++ b/apps/frontend/src/components/settings/modeling/modeling-flow-node.tsx @@ -60,6 +60,7 @@ export type ModelingFlowNodeData = { columnDisplayMeta?: ModelingFlowColumnDisplayMeta[]; relationshipDisplayMeta?: ModelingFlowRelationshipDisplayMeta[]; relationshipActionIds?: Array; + highlightedRelationshipIds?: string[]; actionsDisabled?: boolean; onNodeAction?: (action: ModelingFlowNodeAction) => void; }; @@ -165,6 +166,11 @@ export function ModelingFlowNode({ } nodeData.onNodeAction?.(action); }; + const highlightedRelationshipIdSet = new Set( + (nodeData.highlightedRelationshipIds ?? []) + .map((relationshipId) => relationshipId.trim()) + .filter((relationshipId) => relationshipId.length > 0) + ); return (
{hasRelationshipActionEntry ? ( <> diff --git a/apps/frontend/src/lib/admin-api-client.ts b/apps/frontend/src/lib/admin-api-client.ts index 99e084b..dbbe786 100644 --- a/apps/frontend/src/lib/admin-api-client.ts +++ b/apps/frontend/src/lib/admin-api-client.ts @@ -199,6 +199,8 @@ export interface ModelingSetupRelationshipSuggestion { confidence: number; left: RelationshipBridgeEndpoint; right: RelationshipBridgeEndpoint; + type: ModelingGraphRelationship["type"]; + cardinality: ModelingGraphRelationship["cardinality"]; reason?: string; } @@ -653,6 +655,16 @@ function normalizeModelingSetupRelationshipSuggestion( typeof value.name === "string" && value.name.trim() ? value.name.trim() : `${left.table}.${left.column} = ${right.table}.${right.column}`; + const typeRaw = + typeof value.type === "string" + ? value.type + : typeof value.cardinality === "string" + ? value.cardinality + : undefined; + const relationshipType: ModelingGraphRelationship["type"] = + typeRaw === "many-to-one" || typeRaw === "one-to-many" || typeRaw === "one-to-one" + ? typeRaw + : "many-to-one"; return { id, name, @@ -662,6 +674,8 @@ function normalizeModelingSetupRelationshipSuggestion( ), left, right, + type: relationshipType, + cardinality: relationshipType, reason: typeof value.reason === "string" ? value.reason diff --git a/apps/frontend/tests/unit/admin-api-client-modeling-contract.spec.ts b/apps/frontend/tests/unit/admin-api-client-modeling-contract.spec.ts index 47c895d..54810f7 100644 --- a/apps/frontend/tests/unit/admin-api-client-modeling-contract.spec.ts +++ b/apps/frontend/tests/unit/admin-api-client-modeling-contract.spec.ts @@ -1,7 +1,8 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { getWorkspaceModelingGraph, - precheckWorkspaceModelingDeploy + precheckWorkspaceModelingDeploy, + recommendModelingSetupRelationships } from "@/lib/admin-api-client"; describe("admin-api-client modeling contract parity", () => { @@ -153,4 +154,60 @@ describe("admin-api-client modeling contract parity", () => { deployState: "undeployed" }); }); + + it("preserves explicit recommendation cardinality and defaults missing to many-to-one", async () => { + global.fetch = vi.fn(async () => + new Response( + JSON.stringify({ + status: "ok", + data: { + suggestions: [ + { + id: "rel-explicit", + name: "orders_to_customers", + reason: "foreign_key_constraint", + type: "one-to-one", + bridge: { + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" }, + operator: "eq", + confidence: 0.98 + } + }, + { + id: "rel-defaulted", + name: "payments_to_orders", + reason: "fk_naming_suffix", + bridge: { + left: { dataset: "analytics", table: "payments", column: "order_id" }, + right: { dataset: "analytics", table: "orders", column: "id" }, + operator: "eq", + confidence: 0.86 + } + } + ] + } + }), + { + status: 200, + headers: { "content-type": "application/json" } + } + ) + ) as typeof fetch; + + const suggestions = await recommendModelingSetupRelationships("ws-1", "ds-1", { + selectedTables: ["orders", "customers", "payments"] + }); + expect(suggestions).toHaveLength(2); + expect(suggestions[0]).toMatchObject({ + id: "rel-explicit", + type: "one-to-one", + cardinality: "one-to-one" + }); + expect(suggestions[1]).toMatchObject({ + id: "rel-defaulted", + type: "many-to-one", + cardinality: "many-to-one" + }); + }); }); diff --git a/apps/frontend/tests/unit/data-sources-modeling-setup.spec.tsx b/apps/frontend/tests/unit/data-sources-modeling-setup.spec.tsx index 634378a..6d1dd2c 100644 --- a/apps/frontend/tests/unit/data-sources-modeling-setup.spec.tsx +++ b/apps/frontend/tests/unit/data-sources-modeling-setup.spec.tsx @@ -145,6 +145,9 @@ describe("DataSourcesPage modeling setup wizard", () => { id: "rel-orders-customers", name: "orders.customer_id = customers.id", confidence: 0.92, + type: "many-to-one", + cardinality: "many-to-one", + reason: "foreign_key_constraint", left: { dataset: "analytics", table: "orders", column: "customer_id" }, right: { dataset: "analytics", table: "customers", column: "id" } } @@ -256,4 +259,151 @@ describe("DataSourcesPage modeling setup wizard", () => { expect(mockListModelingSetupTables).toHaveBeenCalledWith("ws-existing", "mysql_main"); }); }); + + it("shows relationship card metadata with cardinality/source/bridge/confidence", async () => { + const user = userEvent.setup(); + await createDatasourceAndOpenSetupWizard(user); + + await user.click(screen.getByRole("button", { name: "下一步：确认关系" })); + + expect(await screen.findByText("orders.customer_id = customers.id")).toBeInTheDocument(); + expect(screen.getByText("1:N")).toBeInTheDocument(); + expect(screen.getByText("FK")).toBeInTheDocument(); + expect( + screen.getByText("Bridge: analytics.orders.customer_id = analytics.customers.id") + ).toBeInTheDocument(); + expect(screen.getByText("置信度 92%")).toBeInTheDocument(); + }); + + it("preserves existing suggestion selection and auto-selects newly added suggestions", async () => { + const user = userEvent.setup(); + mockRecommendModelingSetupRelationships + .mockResolvedValueOnce([ + { + id: "rel-orders-customers", + name: "orders.customer_id = customers.id", + confidence: 0.92, + type: "many-to-one", + cardinality: "many-to-one", + reason: "foreign_key_constraint", + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" } + }, + { + id: "rel-orders-users", + name: "orders.user_id = users.id", + confidence: 0.88, + type: "many-to-one", + cardinality: "many-to-one", + reason: "fk_naming_suffix", + left: { dataset: "analytics", table: "orders", column: "user_id" }, + right: { dataset: "analytics", table: "users", column: "id" } + } + ]) + .mockResolvedValueOnce([ + { + id: "rel-orders-customers", + name: "orders.customer_id = customers.id", + confidence: 0.92, + type: "many-to-one", + cardinality: "many-to-one", + reason: "foreign_key_constraint", + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" } + }, + { + id: "rel-orders-users", + name: "orders.user_id = users.id", + confidence: 0.88, + type: "many-to-one", + cardinality: "many-to-one", + reason: "fk_naming_suffix", + left: { dataset: "analytics", table: "orders", column: "user_id" }, + right: { dataset: "analytics", table: "users", column: "id" } + }, + { + id: "rel-orders-regions", + name: "orders.region_id = regions.id", + confidence: 0.81, + type: "many-to-one", + cardinality: "many-to-one", + reason: "fk_naming_suffix", + left: { dataset: "analytics", table: "orders", column: "region_id" }, + right: { dataset: "analytics", table: "regions", column: "id" } + } + ]); + + await createDatasourceAndOpenSetupWizard(user); + await user.click(screen.getByRole("button", { name: "下一步：确认关系" })); + + const preservedCheckbox = await screen.findByRole("checkbox", { + name: "选择关系建议 orders.user_id = users.id" + }); + expect(preservedCheckbox).toBeChecked(); + await user.click(preservedCheckbox); + expect(preservedCheckbox).not.toBeChecked(); + + await user.click(screen.getByRole("button", { name: "上一步" })); + await user.click(screen.getByRole("button", { name: "下一步：确认关系" })); + + const customersCheckbox = await screen.findByRole("checkbox", { + name: "选择关系建议 orders.customer_id = customers.id" + }); + const usersCheckbox = screen.getByRole("checkbox", { + name: "选择关系建议 orders.user_id = users.id" + }); + const regionsCheckbox = screen.getByRole("checkbox", { + name: "选择关系建议 orders.region_id = regions.id" + }); + + expect(customersCheckbox).toBeChecked(); + expect(usersCheckbox).not.toBeChecked(); + expect(regionsCheckbox).toBeChecked(); + }); + + it("keeps select-all and clear-all behavior on relationship step", async () => { + const user = userEvent.setup(); + mockRecommendModelingSetupRelationships.mockResolvedValueOnce([ + { + id: "rel-orders-customers", + name: "orders.customer_id = customers.id", + confidence: 0.92, + type: "many-to-one", + cardinality: "many-to-one", + reason: "foreign_key_constraint", + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" } + }, + { + id: "rel-orders-users", + name: "orders.user_id = users.id", + confidence: 0.88, + type: "many-to-one", + cardinality: "many-to-one", + reason: "fk_naming_suffix", + left: { dataset: "analytics", table: "orders", column: "user_id" }, + right: { dataset: "analytics", table: "users", column: "id" } + } + ]); + + await createDatasourceAndOpenSetupWizard(user); + await user.click(screen.getByRole("button", { name: "下一步：确认关系" })); + + const checkboxA = await screen.findByRole("checkbox", { + name: "选择关系建议 orders.customer_id = customers.id" + }); + const checkboxB = screen.getByRole("checkbox", { + name: "选择关系建议 orders.user_id = users.id" + }); + expect(checkboxA).toBeChecked(); + expect(checkboxB).toBeChecked(); + + await user.click(screen.getByRole("button", { name: "全不选" })); + expect(checkboxA).not.toBeChecked(); + expect(checkboxB).not.toBeChecked(); + + await user.click(screen.getByRole("button", { name: "全选" })); + expect(checkboxA).toBeChecked(); + expect(checkboxB).toBeChecked(); + }); }); diff --git a/apps/frontend/tests/unit/settings-modeling-auto-layout-resilience.spec.tsx b/apps/frontend/tests/unit/settings-modeling-auto-layout-resilience.spec.tsx index 83c7bf0..c9d52e9 100644 --- a/apps/frontend/tests/unit/settings-modeling-auto-layout-resilience.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-auto-layout-resilience.spec.tsx @@ -200,4 +200,58 @@ describe("ModelingFlowCanvas auto layout resilience", () => { expect(autoLayoutButton).toBeEnabled(); }); }); + + it("queues programmatic layout requests as one-shot triggers keyed by request id", async () => { + computeElkLayoutMock.mockResolvedValue({ + ok: true, + elapsedMs: 180, + positions: { + "model:model.a": { x: 120, y: 80 }, + "model:model.b": { x: 420, y: 80 } + } + }); + + const { rerender } = render( + + ); + + await waitFor(() => { + expect(computeElkLayoutMock).toHaveBeenCalledTimes(1); + expect(screen.getByTestId("resilience-first-position")).toHaveTextContent("120,80"); + }); + + rerender( + + ); + + await waitFor(() => { + expect(computeElkLayoutMock).toHaveBeenCalledTimes(1); + }); + + rerender( + + ); + + await waitFor(() => { + expect(computeElkLayoutMock).toHaveBeenCalledTimes(2); + }); + }); }); diff --git a/apps/frontend/tests/unit/settings-modeling-complete-parity.spec.tsx b/apps/frontend/tests/unit/settings-modeling-complete-parity.spec.tsx index 3c99239..381e6d9 100644 --- a/apps/frontend/tests/unit/settings-modeling-complete-parity.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-complete-parity.spec.tsx @@ -1,6 +1,6 @@ import { render, screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import ModelingWorkspacePage from "@/app/settings/modeling/page"; import { deployWorkspaceModeling, @@ -95,10 +95,13 @@ const mockGetWorkspaceModelingPreview = vi.mocked(getWorkspaceModelingPreview); const mockUpsertWorkspaceModelingGraph = vi.mocked(upsertWorkspaceModelingGraph); const mockPrecheckWorkspaceModelingDeploy = vi.mocked(precheckWorkspaceModelingDeploy); const mockDeployWorkspaceModeling = vi.mocked(deployWorkspaceModeling); +const originalShowSchemaDeployPanels = + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS; describe("settings modeling complete parity", () => { beforeEach(() => { vi.clearAllMocks(); + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = "true"; window.history.replaceState( {}, "", @@ -337,6 +340,10 @@ describe("settings modeling complete parity", () => { }); }); + afterEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = originalShowSchemaDeployPanels; + }); + it("keeps save-as-view context, supports view delete, and completes deploy gating flow", async () => { const user = userEvent.setup(); render(); diff --git a/apps/frontend/tests/unit/settings-modeling-deploy.spec.tsx b/apps/frontend/tests/unit/settings-modeling-deploy.spec.tsx index 6941460..4228f37 100644 --- a/apps/frontend/tests/unit/settings-modeling-deploy.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-deploy.spec.tsx @@ -1,6 +1,6 @@ import { render, screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import ModelingWorkspacePage from "@/app/settings/modeling/page"; import { deployWorkspaceModeling, @@ -8,7 +8,8 @@ import { listWorkspaceDatasourceBindings, listWorkspaceDatasourceTablePermissions, listWorkspaces, - precheckWorkspaceModelingDeploy + precheckWorkspaceModelingDeploy, + recommendModelingSetupRelationships } from "@/lib/admin-api-client"; vi.mock("@/components/settings/modeling/modeling-flow-canvas", () => ({ @@ -45,7 +46,8 @@ vi.mock("@/lib/admin-api-client", async (importOriginal) => { listWorkspaceDatasourceTablePermissions: vi.fn(), getWorkspaceModelingGraph: vi.fn(), precheckWorkspaceModelingDeploy: vi.fn(), - deployWorkspaceModeling: vi.fn() + deployWorkspaceModeling: vi.fn(), + recommendModelingSetupRelationships: vi.fn() }; }); @@ -57,10 +59,14 @@ const mockListWorkspaceDatasourceTablePermissions = vi.mocked( const mockGetWorkspaceModelingGraph = vi.mocked(getWorkspaceModelingGraph); const mockPrecheckWorkspaceModelingDeploy = vi.mocked(precheckWorkspaceModelingDeploy); const mockDeployWorkspaceModeling = vi.mocked(deployWorkspaceModeling); +const mockRecommendModelingSetupRelationships = vi.mocked(recommendModelingSetupRelationships); +const originalShowSchemaDeployPanels = + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS; describe("settings modeling deploy flow", () => { beforeEach(() => { vi.clearAllMocks(); + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = "true"; window.history.replaceState({}, "", "/modeling?workspaceId=ws-1&datasourceId=ds-1"); mockListWorkspaces.mockResolvedValue({ @@ -85,6 +91,11 @@ describe("settings modeling deploy flow", () => { tableNames: ["orders"], policyVersion: 11 }); + mockRecommendModelingSetupRelationships.mockResolvedValue([]); + }); + + afterEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = originalShowSchemaDeployPanels; }); it("shows policy_version_conflict guidance and dry-run failure details", async () => { @@ -305,10 +316,12 @@ describe("settings modeling deploy flow", () => { await screen.findByRole("button", { name: "选择 model Orders Model" }); expect(screen.getByTestId("modeling-top-status-bar")).toHaveTextContent(/Deploy State synced/); + await user.click(screen.getByRole("button", { name: "选择 model Orders Model" })); + await user.click(screen.getByRole("button", { name: "编辑 metadata" })); - await user.clear(screen.getByRole("textbox", { name: "显示名称" })); - await user.type(screen.getByRole("textbox", { name: "显示名称" }), "订单模型未保存"); - await user.click(screen.getByRole("button", { name: "保存 Metadata" })); + await user.clear(screen.getByRole("textbox", { name: "Model alias" })); + await user.type(screen.getByRole("textbox", { name: "Model alias" }), "订单模型未保存"); + await user.click(screen.getByRole("button", { name: "Submit" })); const topStatusBar = screen.getByTestId("modeling-top-status-bar"); expect(topStatusBar).toHaveTextContent(/Deploy State undeployed/); diff --git a/apps/frontend/tests/unit/settings-modeling-flow-canvas.spec.tsx b/apps/frontend/tests/unit/settings-modeling-flow-canvas.spec.tsx index f6eb4e9..6de2579 100644 --- a/apps/frontend/tests/unit/settings-modeling-flow-canvas.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-flow-canvas.spec.tsx @@ -91,6 +91,12 @@ vi.mock("@xyflow/react", () => { selected?: boolean; sourceHandle?: string; targetHandle?: string; + source?: string; + target?: string; + data?: { + highlightedBySelectedRelationship?: boolean; + highlightedBySelectedModel?: boolean; + }; }> | undefined) ?? []; const onNodeClick = props.onNodeClick as | ((event: unknown, node: { id: string }) => void) @@ -234,6 +240,26 @@ vi.mock("@xyflow/react", () => { .map((edge) => edge.id) .join(",")}

+

+ {nodes + .filter((node) => node.id.startsWith("model:")) + .map((node) => { + const highlightedRelationshipIds = ( + (node.data as { highlightedRelationshipIds?: string[] } | undefined) + ?.highlightedRelationshipIds ?? [] + ).join(","); + return `${node.id}:${highlightedRelationshipIds}`; + }) + .join("|")} +

+

+ {edges + .map( + (edge) => + `${edge.id}:${edge.data?.highlightedBySelectedRelationship ? 1 : 0},${edge.data?.highlightedBySelectedModel ? 1 : 0}` + ) + .join("|")} +

{edges[0] ? `${edges[0].sourceHandle ?? ""}->${edges[0].targetHandle ?? ""}` : "none"}

@@ -415,6 +441,56 @@ describe("ModelingFlowCanvas", () => { expect(onSelectNode).toHaveBeenCalledWith({ kind: "model", id: "model.customers" }); }); + it("runs programmatic auto-layout as a one-shot request by request id", async () => { + const onNodePositionsChange = vi.fn(); + const { rerender } = render( + + ); + + await waitFor(() => { + expect(computeElkLayoutMock).toHaveBeenCalledTimes(1); + expect(onNodePositionsChange).toHaveBeenCalledTimes(1); + }); + + rerender( + + ); + + await waitFor(() => { + expect(computeElkLayoutMock).toHaveBeenCalledTimes(1); + }); + + rerender( + + ); + + await waitFor(() => { + expect(computeElkLayoutMock).toHaveBeenCalledTimes(2); + expect(onNodePositionsChange).toHaveBeenCalledTimes(2); + }); + }); + it("prefers persisted payload positions over default grid fallback", async () => { render( { expect(screen.getByTestId("selected-node-ids")).toHaveTextContent("model:model.orders"); }); }); + + it("highlights selected relationship edge and both endpoint relationship rows", async () => { + render( + + ); + + await waitFor(() => { + expect(screen.getByTestId("selected-edge-ids")).toHaveTextContent("rel-orders-customers"); + expect(screen.getByTestId("edge-highlight-map")).toHaveTextContent( + "rel-orders-customers:1,0" + ); + expect(screen.getByTestId("node-highlight-map")).toHaveTextContent( + "model:model.customers:rel-orders-customers" + ); + expect(screen.getByTestId("node-highlight-map")).toHaveTextContent( + "model:model.orders:rel-orders-customers" + ); + }); + }); + + it("softly marks incident edges when a model node is selected", async () => { + render( + + ); + + await waitFor(() => { + expect(screen.getByTestId("selected-edge-ids")).not.toHaveTextContent(/\S/); + expect(screen.getByTestId("edge-highlight-map")).toHaveTextContent( + "rel-orders-customers:0,1" + ); + expect(screen.getByTestId("node-highlight-map")).toHaveTextContent("model:model.orders:"); + expect(screen.getByTestId("node-highlight-map")).toHaveTextContent("model:model.customers:"); + }); + }); }); diff --git a/apps/frontend/tests/unit/settings-modeling-flow-node-edge.spec.tsx b/apps/frontend/tests/unit/settings-modeling-flow-node-edge.spec.tsx index ed99b75..842af08 100644 --- a/apps/frontend/tests/unit/settings-modeling-flow-node-edge.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-flow-node-edge.spec.tsx @@ -29,6 +29,7 @@ vi.mock("@xyflow/react", () => ({ onMouseLeave?: () => void; "data-confidence-band"?: string; "data-selected"?: string; + "data-visual-state"?: string; }) => (
({ data-marker-end={markerEnd ?? ""} data-confidence-band={props["data-confidence-band"]} data-selected={props["data-selected"]} + data-visual-state={props["data-visual-state"]} onMouseEnter={onMouseEnter} onMouseLeave={onMouseLeave} /> @@ -265,6 +267,40 @@ describe("ModelingFlowNode", () => { }); }); + it("highlights relationship rows when selected relationship ids are provided", () => { + const nodeProps = { + id: "model.orders", + data: { + kind: "model", + title: "Orders", + subtitle: "orders", + sections: { + columns: ["id"], + calculatedFields: [], + relationships: ["rel-orders-customers", "rel-orders-invoices"] + }, + relationshipDisplayMeta: [ + { + primaryText: "customers" + }, + { + primaryText: "invoices" + } + ], + relationshipActionIds: ["rel-orders-customers", "rel-orders-invoices"], + highlightedRelationshipIds: ["rel-orders-customers"] + }, + selected: false + } as unknown as Parameters[0]; + + render(); + + const customersRow = screen.getByText("customers").closest("li"); + const invoicesRow = screen.getByText("invoices").closest("li"); + expect(customersRow).toHaveAttribute("data-relationship-highlighted", "true"); + expect(invoicesRow).toHaveAttribute("data-relationship-highlighted", "false"); + }); + it("keeps action buttons disabled-safe and ignores keyboard triggers when actions are disabled", async () => { const user = userEvent.setup(); const onNodeAction = vi.fn(); @@ -361,6 +397,7 @@ describe("ModelingFlowEdge", () => { expect(edgeClass).toContain("[stroke-dasharray:6_4]"); expect(edge).toHaveAttribute("data-confidence-band", "low"); expect(edge).toHaveAttribute("data-selected", "false"); + expect(edge).toHaveAttribute("data-visual-state", "inferred-or-low-confidence"); expect(edge).toHaveAttribute("data-marker-start", ""); expect(edge).toHaveAttribute("data-marker-end", ""); expect(screen.queryByTestId("modeling-flow-edge-hover-card")).not.toBeInTheDocument(); @@ -397,12 +434,13 @@ describe("ModelingFlowEdge", () => { const edge = screen.getByTestId("flow-base-edge"); const edgeClass = edge.getAttribute("data-class") ?? ""; expect(edgeClass).toContain("stroke-[var(--action-primary)]"); - expect(edgeClass).toContain("stroke-[2.5]"); + expect(edgeClass).toContain("stroke-[2.7]"); expect(screen.getByTestId("modeling-flow-edge-hover-card")).toBeInTheDocument(); expect(screen.getByText("Description")).toBeInTheDocument(); expect(screen.getAllByText("-").length).toBeGreaterThan(0); expect(edgeClass).not.toContain("[stroke-dasharray:6_4]"); expect(edge).toHaveAttribute("data-selected", "true"); + expect(edge).toHaveAttribute("data-visual-state", "selected-relationship"); }); it("renders one/many markers based on relationship cardinality", async () => { @@ -431,8 +469,8 @@ describe("ModelingFlowEdge", () => { ); const edge = screen.getByTestId("flow-base-edge"); - expect(edge.getAttribute("data-marker-start")).toContain("-many"); - expect(edge.getAttribute("data-marker-end")).toContain("-one"); + expect(screen.getByTestId("modeling-flow-edge-source-cardinality")).toHaveTextContent("N"); + expect(screen.getByTestId("modeling-flow-edge-target-cardinality")).toHaveTextContent("1"); await user.hover(edge); expect(screen.getByTestId("modeling-flow-edge-hover-card")).toBeInTheDocument(); @@ -464,10 +502,10 @@ describe("ModelingFlowEdge", () => { ); - const edge = screen.getByTestId("flow-base-edge"); - expect(edge.getAttribute("data-marker-start")).toContain("-many"); - expect(edge.getAttribute("data-marker-end")).toContain("-many"); + expect(screen.getByTestId("modeling-flow-edge-source-cardinality")).toHaveTextContent("N"); + expect(screen.getByTestId("modeling-flow-edge-target-cardinality")).toHaveTextContent("N"); + const edge = screen.getByTestId("flow-base-edge"); await user.hover(edge); expect(screen.getByText("多对多 (Many-to-many)")).toBeInTheDocument(); }); @@ -528,13 +566,78 @@ describe("ModelingFlowEdge", () => { const edge = screen.getByTestId("flow-base-edge"); const edgeClass = edge.getAttribute("data-class") ?? ""; - expect(edgeClass).toContain("stroke-red-500"); - expect(edgeClass).toContain("stroke-[2.2]"); + expect(edgeClass).toContain("stroke-red-600"); + expect(edgeClass).toContain("stroke-[2.8]"); expect(edgeClass).toContain("[stroke-dasharray:6_4]"); expect(edge).toHaveAttribute("data-confidence-band", "invalid"); + expect(edge).toHaveAttribute("data-visual-state", "invalid"); await user.hover(edge); expect(screen.getByTestId("modeling-flow-edge-hover-card")).toBeInTheDocument(); expect(screen.getByText("Unknown")).toBeInTheDocument(); }); + + it("uses soft incident style when selected model highlights an edge", () => { + const edgeProps = { + id: "rel-selected-model-incident", + sourceX: 0, + sourceY: 0, + targetX: 100, + targetY: 50, + sourcePosition: "right", + targetPosition: "left", + selected: false, + data: { + label: "orders.customer_id = customers.id", + source: "manual", + confidence: 0.92, + highlightedBySelectedModel: true + } + } as unknown as Parameters[0]; + + render( + + + + ); + + const edge = screen.getByTestId("flow-base-edge"); + const edgeClass = edge.getAttribute("data-class") ?? ""; + expect(edgeClass).toContain("stroke-slate-400"); + expect(edgeClass).toContain("stroke-[2.1]"); + expect(edgeClass).toContain("[stroke-dasharray:6_4]"); + expect(edge).toHaveAttribute("data-visual-state", "selected-model-incident"); + }); + + it("keeps invalid edge priority above selected-relationship style", () => { + const edgeProps = { + id: "rel-invalid-selected", + sourceX: 0, + sourceY: 0, + targetX: 100, + targetY: 50, + sourcePosition: "right", + targetPosition: "left", + selected: true, + data: { + label: "orders.customer_id = missing.id", + source: "inferred", + confidence: 0.12, + invalid: true + } + } as unknown as Parameters[0]; + + render( + + + + ); + + const edge = screen.getByTestId("flow-base-edge"); + const edgeClass = edge.getAttribute("data-class") ?? ""; + expect(edgeClass).toContain("stroke-red-600"); + expect(edgeClass).toContain("stroke-[2.8]"); + expect(edge).toHaveAttribute("data-visual-state", "invalid"); + expect(edge).toHaveAttribute("data-selected", "true"); + }); }); diff --git a/apps/frontend/tests/unit/settings-modeling-layout-parity.spec.tsx b/apps/frontend/tests/unit/settings-modeling-layout-parity.spec.tsx index f188277..8838290 100644 --- a/apps/frontend/tests/unit/settings-modeling-layout-parity.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-layout-parity.spec.tsx @@ -1,7 +1,7 @@ import React from "react"; import { render, screen, waitFor, within } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import ModelingWorkspacePage from "@/app/settings/modeling/page"; import { getWorkspaceModelingGraph, @@ -143,11 +143,16 @@ const mockListWorkspaceDatasourceTablePermissions = vi.mocked( listWorkspaceDatasourceTablePermissions ); const mockGetWorkspaceModelingGraph = vi.mocked(getWorkspaceModelingGraph); +const originalShowDetailsPanel = process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL; +const originalShowSchemaDeployPanels = + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS; describe("Modeling layout parity", () => { beforeEach(() => { vi.clearAllMocks(); fitViewMock.mockClear(); + process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL = "true"; + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = "true"; Object.defineProperty(window, "innerWidth", { writable: true, configurable: true, value: 1280 }); Element.prototype.scrollIntoView = vi.fn(); Object.defineProperty(window, "ResizeObserver", { @@ -230,6 +235,11 @@ describe("Modeling layout parity", () => { }); }); + afterEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL = originalShowDetailsPanel; + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = originalShowSchemaDeployPanels; + }); + it("renders flowchart-first workbench layout as left tree, center canvas, and right context", async () => { render(); diff --git a/apps/frontend/tests/unit/settings-modeling-schema-change.spec.tsx b/apps/frontend/tests/unit/settings-modeling-schema-change.spec.tsx index 278c8fb..c895767 100644 --- a/apps/frontend/tests/unit/settings-modeling-schema-change.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-schema-change.spec.tsx @@ -1,6 +1,6 @@ import { render, screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import ModelingWorkspacePage from "@/app/settings/modeling/page"; import { detectWorkspaceModelingSchemaChanges, @@ -36,9 +36,12 @@ const mockDetectWorkspaceModelingSchemaChanges = vi.mocked( const mockResolveWorkspaceModelingSchemaChange = vi.mocked( resolveWorkspaceModelingSchemaChange ); +const originalShowSchemaDeployPanels = + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS; describe("settings modeling schema change flow", () => { beforeEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = "true"; vi.clearAllMocks(); window.history.replaceState({}, "", "/modeling?workspaceId=ws-1&datasourceId=ds-1"); @@ -233,6 +236,10 @@ describe("settings modeling schema change flow", () => { }); }); + afterEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = originalShowSchemaDeployPanels; + }); + it("supports detect -> resolve -> re-detect and keeps residual count clear", async () => { const user = userEvent.setup(); render(); diff --git a/apps/frontend/tests/unit/settings-modeling-view-sync.spec.tsx b/apps/frontend/tests/unit/settings-modeling-view-sync.spec.tsx index e5e9061..2bf17d8 100644 --- a/apps/frontend/tests/unit/settings-modeling-view-sync.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-view-sync.spec.tsx @@ -1,6 +1,6 @@ import { render, screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import ModelingWorkspacePage from "@/app/settings/modeling/page"; import { SaveAsViewDialog } from "@/components/chat/save-as-view-dialog"; import { @@ -8,6 +8,7 @@ import { listWorkspaceDatasourceBindings, listWorkspaceDatasourceTablePermissions, listWorkspaces, + recommendModelingSetupRelationships, saveModelingViewFromRun } from "@/lib/admin-api-client"; @@ -27,7 +28,8 @@ vi.mock("@/lib/admin-api-client", async (importOriginal) => { listWorkspaces: vi.fn(), listWorkspaceDatasourceBindings: vi.fn(), listWorkspaceDatasourceTablePermissions: vi.fn(), - getWorkspaceModelingGraph: vi.fn() + getWorkspaceModelingGraph: vi.fn(), + recommendModelingSetupRelationships: vi.fn() }; }); @@ -38,6 +40,7 @@ const mockListWorkspaceDatasourceTablePermissions = vi.mocked( listWorkspaceDatasourceTablePermissions ); const mockGetWorkspaceModelingGraph = vi.mocked(getWorkspaceModelingGraph); +const mockRecommendModelingSetupRelationships = vi.mocked(recommendModelingSetupRelationships); describe("settings modeling view sync from chat", () => { beforeEach(() => { @@ -109,8 +112,11 @@ describe("settings modeling view sync from chat", () => { }); describe("settings modeling page view selection sync", () => { + const originalShowDetailsPanel = process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL; + beforeEach(() => { vi.clearAllMocks(); + process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL = "true"; window.sessionStorage.clear(); mockListWorkspaces.mockResolvedValue({ items: [{ id: "ws-sync", name: "Workspace Sync", isDefault: true }], @@ -134,6 +140,7 @@ describe("settings modeling page view selection sync", () => { tableNames: ["orders"], policyVersion: 4 }); + mockRecommendModelingSetupRelationships.mockResolvedValue([]); mockGetWorkspaceModelingGraph.mockResolvedValue({ workspaceId: "ws-sync", datasourceId: "ds-sync", @@ -171,6 +178,10 @@ describe("settings modeling page view selection sync", () => { }); }); + afterEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_DETAILS_PANEL = originalShowDetailsPanel; + }); + it("prefers query viewId and selects target view on page load", async () => { window.sessionStorage.setItem("text2sql.modeling.selectViewId", "view.old"); window.history.replaceState( diff --git a/apps/frontend/tests/unit/settings-modeling-workspace.spec.tsx b/apps/frontend/tests/unit/settings-modeling-workspace.spec.tsx index 940d289..0956635 100644 --- a/apps/frontend/tests/unit/settings-modeling-workspace.spec.tsx +++ b/apps/frontend/tests/unit/settings-modeling-workspace.spec.tsx @@ -1,6 +1,6 @@ import { render, screen, waitFor } from "@testing-library/react"; import userEvent from "@testing-library/user-event"; -import { beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import ModelingWorkspacePage from "@/app/settings/modeling/page"; import { ModelingCalculatedFieldEditor } from "@/components/settings/modeling/modeling-calculated-field-editor"; import { @@ -11,6 +11,7 @@ import { listWorkspaceDatasourceTablePermissions, listWorkspaces, precheckWorkspaceModelingDeploy, + recommendModelingSetupRelationships, upsertWorkspaceModelingGraph } from "@/lib/admin-api-client"; @@ -130,6 +131,7 @@ vi.mock("@/lib/admin-api-client", async (importOriginal) => { getWorkspaceModelingPreview: vi.fn(), getWorkspaceModelingGraph: vi.fn(), precheckWorkspaceModelingDeploy: vi.fn(), + recommendModelingSetupRelationships: vi.fn(), upsertWorkspaceModelingGraph: vi.fn() }; }); @@ -142,7 +144,10 @@ const mockListWorkspaceDatasourceTablePermissions = vi.mocked( const mockGetWorkspaceModelingPreview = vi.mocked(getWorkspaceModelingPreview); const mockGetWorkspaceModelingGraph = vi.mocked(getWorkspaceModelingGraph); const mockPrecheckWorkspaceModelingDeploy = vi.mocked(precheckWorkspaceModelingDeploy); +const mockRecommendModelingSetupRelationships = vi.mocked(recommendModelingSetupRelationships); const mockUpsertWorkspaceModelingGraph = vi.mocked(upsertWorkspaceModelingGraph); +const originalShowSchemaDeployPanels = + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS; async function waitForWorkspaceDatasourceReady(): Promise { await waitFor(() => { @@ -153,6 +158,7 @@ async function waitForWorkspaceDatasourceReady(): Promise { describe("ModelingWorkspacePage", () => { beforeEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = "true"; vi.clearAllMocks(); window.history.replaceState( {}, @@ -257,6 +263,7 @@ describe("ModelingWorkspacePage", () => { unresolvedSchemaChangeIds: [] } }); + mockRecommendModelingSetupRelationships.mockResolvedValue([]); mockUpsertWorkspaceModelingGraph.mockImplementation(async (workspaceId, datasourceId, input) => ({ workspaceId, @@ -278,6 +285,426 @@ describe("ModelingWorkspacePage", () => { })); }); + it("bootstraps recommendations once when draft has models but no relationships", async () => { + const user = userEvent.setup(); + mockRecommendModelingSetupRelationships.mockResolvedValueOnce([ + { + id: "rel-orders-customers", + name: "orders.customer_id -> customers.id", + confidence: 0.96, + reason: "foreign_key_constraint", + type: "many-to-one", + cardinality: "many-to-one", + left: { + dataset: "analytics", + table: "orders", + column: "customer_id" + }, + right: { + dataset: "analytics", + table: "customers", + column: "id" + } + } + ]); + + render(); + + await waitForWorkspaceDatasourceReady(); + await waitFor(() => { + expect(mockRecommendModelingSetupRelationships).toHaveBeenCalledWith("ws-2", "ds-2b", { + selectedTables: ["orders"] + }); + }); + await waitFor(() => { + expect(mockUpsertWorkspaceModelingGraph).toHaveBeenCalledWith( + "ws-2", + "ds-2b", + expect.objectContaining({ + relationships: [ + expect.objectContaining({ + id: "rel-orders-customers", + source: "fk", + bridge: expect.objectContaining({ + operator: "eq" + }) + }) + ] + }) + ); + }); + expect( + await screen.findByText("已自动补全 1 条关系（跳过 0 条）。") + ).toBeInTheDocument(); + + await user.click(screen.getByRole("button", { name: "trigger-node-action-add-calculated-field" })); + await waitFor(() => { + expect(mockRecommendModelingSetupRelationships).toHaveBeenCalledTimes(1); + }); + }); + + afterEach(() => { + process.env.NEXT_PUBLIC_MODELING_SHOW_SCHEMA_DEPLOY_PANELS = originalShowSchemaDeployPanels; + }); + + it("does not auto-bootstrap when draft already has relationships", async () => { + mockGetWorkspaceModelingGraph.mockResolvedValueOnce({ + workspaceId: "ws-2", + datasourceId: "ds-2b", + activeRevision: 1, + draft: { + policyVersion: 7, + revision: 2, + graphHash: "hash-r2-with-relationships", + updatedAt: "2026-04-23T00:00:00.000Z", + graphPayload: { + models: [ + { + id: "model.orders", + tableName: "orders", + modelName: "orders", + displayName: "Orders", + description: null, + columns: [] + }, + { + id: "model.customers", + tableName: "customers", + modelName: "customers", + displayName: "Customers", + description: null, + columns: [] + } + ], + relationships: [ + { + id: "rel-orders-customers", + source: "manual", + confidence: 0.9, + bridge: { + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" }, + operator: "eq", + confidence: 0.9 + } + } + ], + calculatedFields: [], + views: [], + schemaChanges: [] + } + } + }); + + render(); + + await waitForWorkspaceDatasourceReady(); + expect(await screen.findByText("orders.customer_id = customers.id")).toBeInTheDocument(); + expect(mockRecommendModelingSetupRelationships).not.toHaveBeenCalled(); + }); + + it("syncs database non-destructively and keeps manual reverse relationship unchanged", async () => { + const user = userEvent.setup(); + mockGetWorkspaceModelingGraph.mockResolvedValueOnce({ + workspaceId: "ws-2", + datasourceId: "ds-2b", + activeRevision: 1, + draft: { + policyVersion: 7, + revision: 2, + graphHash: "hash-r2-manual-reverse", + updatedAt: "2026-04-23T00:00:00.000Z", + graphPayload: { + models: [ + { + id: "model.orders", + tableName: "orders", + modelName: "orders", + displayName: "Orders", + description: null, + columns: [] + }, + { + id: "model.customers", + tableName: "customers", + modelName: "customers", + displayName: "Customers", + description: null, + columns: [] + }, + { + id: "model.regions", + tableName: "regions", + modelName: "regions", + displayName: "Regions", + description: null, + columns: [] + } + ], + relationships: [ + { + id: "rel-manual-customers-orders", + source: "manual", + confidence: 0.91, + bridge: { + left: { dataset: "analytics", table: "customers", column: "id" }, + right: { dataset: "analytics", table: "orders", column: "customer_id" }, + operator: "eq", + confidence: 0.91 + } + } + ], + calculatedFields: [], + views: [], + schemaChanges: [] + } + } + }); + mockRecommendModelingSetupRelationships.mockResolvedValueOnce([ + { + id: "rel-orders-customers-recommended", + name: "orders.customer_id -> customers.id", + confidence: 0.95, + reason: "foreign_key_constraint", + type: "many-to-one", + cardinality: "many-to-one", + left: { + dataset: "analytics", + table: "orders", + column: "customer_id" + }, + right: { + dataset: "analytics", + table: "customers", + column: "id" + } + }, + { + id: "rel-orders-regions", + name: "orders.region_id -> regions.id", + confidence: 0.84, + reason: "name_inference", + type: "many-to-one", + cardinality: "many-to-one", + left: { + dataset: "analytics", + table: "orders", + column: "region_id" + }, + right: { + dataset: "analytics", + table: "regions", + column: "id" + } + } + ]); + + render(); + + await waitForWorkspaceDatasourceReady(); + await user.click(screen.getByRole("button", { name: "同步数据库" })); + + await waitFor(() => { + expect(mockRecommendModelingSetupRelationships).toHaveBeenCalledWith("ws-2", "ds-2b", { + selectedTables: ["customers", "orders", "regions"] + }); + }); + await waitFor(() => { + expect(mockUpsertWorkspaceModelingGraph).toHaveBeenCalledWith( + "ws-2", + "ds-2b", + expect.objectContaining({ + relationships: [ + expect.objectContaining({ + id: "rel-manual-customers-orders", + source: "manual", + bridge: expect.objectContaining({ + left: expect.objectContaining({ + table: "customers", + column: "id" + }), + right: expect.objectContaining({ + table: "orders", + column: "customer_id" + }), + operator: "eq" + }) + }), + expect.objectContaining({ + id: "rel-orders-regions", + source: "inferred" + }) + ] + }) + ); + }); + expect(await screen.findByText("同步数据库完成：新增 1 条，跳过 1 条。")).toBeInTheDocument(); + expect(screen.getByText("customers.id = orders.customer_id")).toBeInTheDocument(); + expect(screen.getByText("orders.region_id = regions.id")).toBeInTheDocument(); + }); + + it("shows no-op feedback and skips save when sync finds no missing relationships", async () => { + const user = userEvent.setup(); + mockGetWorkspaceModelingGraph.mockResolvedValueOnce({ + workspaceId: "ws-2", + datasourceId: "ds-2b", + activeRevision: 1, + draft: { + policyVersion: 7, + revision: 2, + graphHash: "hash-r2-sync-noop", + updatedAt: "2026-04-23T00:00:00.000Z", + graphPayload: { + models: [ + { + id: "model.orders", + tableName: "orders", + modelName: "orders", + displayName: "Orders", + description: null, + columns: [] + }, + { + id: "model.customers", + tableName: "customers", + modelName: "customers", + displayName: "Customers", + description: null, + columns: [] + } + ], + relationships: [ + { + id: "rel-orders-customers", + source: "manual", + confidence: 0.9, + bridge: { + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" }, + operator: "eq", + confidence: 0.9 + } + } + ], + calculatedFields: [], + views: [], + schemaChanges: [] + } + } + }); + mockRecommendModelingSetupRelationships.mockResolvedValueOnce([ + { + id: "rel-orders-customers-recommended", + name: "orders.customer_id -> customers.id", + confidence: 0.95, + reason: "foreign_key_constraint", + type: "many-to-one", + cardinality: "many-to-one", + left: { + dataset: "analytics", + table: "orders", + column: "customer_id" + }, + right: { + dataset: "analytics", + table: "customers", + column: "id" + } + } + ]); + + render(); + + await waitForWorkspaceDatasourceReady(); + await user.click(screen.getByRole("button", { name: "同步数据库" })); + + expect( + await screen.findByText("同步数据库完成：无新增关系（跳过 1 条）。") + ).toBeInTheDocument(); + expect(mockUpsertWorkspaceModelingGraph).not.toHaveBeenCalled(); + }); + + it("keeps graph/pending state unchanged when sync save fails", async () => { + const user = userEvent.setup(); + mockGetWorkspaceModelingGraph.mockResolvedValueOnce({ + workspaceId: "ws-2", + datasourceId: "ds-2b", + activeRevision: 2, + draft: { + policyVersion: 7, + revision: 2, + graphHash: "hash-r2-sync-fail", + updatedAt: "2026-04-23T00:00:00.000Z", + graphPayload: { + models: [ + { + id: "model.orders", + tableName: "orders", + modelName: "orders", + displayName: "Orders", + description: null, + columns: [] + }, + { + id: "model.regions", + tableName: "regions", + modelName: "regions", + displayName: "Regions", + description: null, + columns: [] + } + ], + relationships: [ + { + id: "rel-orders-customers", + source: "manual", + confidence: 0.9, + bridge: { + left: { dataset: "analytics", table: "orders", column: "customer_id" }, + right: { dataset: "analytics", table: "customers", column: "id" }, + operator: "eq", + confidence: 0.9 + } + } + ], + calculatedFields: [], + views: [], + schemaChanges: [] + } + } + }); + mockRecommendModelingSetupRelationships.mockResolvedValueOnce([ + { + id: "rel-orders-regions", + name: "orders.region_id -> regions.id", + confidence: 0.84, + reason: "name_inference", + type: "many-to-one", + cardinality: "many-to-one", + left: { + dataset: "analytics", + table: "orders", + column: "region_id" + }, + right: { + dataset: "analytics", + table: "regions", + column: "id" + } + } + ]); + mockUpsertWorkspaceModelingGraph.mockRejectedValueOnce(new Error("sync save failed")); + + render(); + + await waitForWorkspaceDatasourceReady(); + await user.click(screen.getByRole("button", { name: "同步数据库" })); + + expect(await screen.findByText("sync save failed")).toBeInTheDocument(); + expect(screen.getByTestId("modeling-top-status-bar")).toHaveTextContent("Deploy State synced"); + expect(screen.queryByText("orders.region_id = regions.id")).not.toBeInTheDocument(); + }); + it("updates model metadata in workspace and saves via modeling graph upsert", async () => { const user = userEvent.setup(); render(); diff --git a/graphify-out/GRAPH_REPORT.md b/graphify-out/GRAPH_REPORT.md index f3c9874..f8ce71f 100644 --- a/graphify-out/GRAPH_REPORT.md +++ b/graphify-out/GRAPH_REPORT.md @@ -1,12 +1,12 @@ -# Graph Report - /Users/lienli/Documents/GitHub/text2sql (2026-04-24) +# Graph Report - /Users/lienli/Documents/GitHub/text2sql (2026-04-25) ## Corpus Check -- 664 files · ~960,874 words +- 673 files · ~1,006,016 words - Verdict: corpus is large enough that graph structure adds value. ## Summary -- 3252 nodes · 6127 edges · 473 communities detected -- Extraction: 78% EXTRACTED · 22% INFERRED · 0% AMBIGUOUS · INFERRED: 1327 edges (avg confidence: 0.8) +- 3421 nodes · 6510 edges · 479 communities detected +- Extraction: 79% EXTRACTED · 21% INFERRED · 0% AMBIGUOUS · INFERRED: 1377 edges (avg confidence: 0.8) - Token cost: 0 input · 0 output ## Community Hubs (Navigation) @@ -483,22 +483,28 @@ - [[_COMMUNITY_Community 470|Community 470]] - [[_COMMUNITY_Community 471|Community 471]] - [[_COMMUNITY_Community 472|Community 472]] +- [[_COMMUNITY_Community 473|Community 473]] +- [[_COMMUNITY_Community 474|Community 474]] +- [[_COMMUNITY_Community 475|Community 475]] +- [[_COMMUNITY_Community 476|Community 476]] +- [[_COMMUNITY_Community 477|Community 477]] +- [[_COMMUNITY_Community 478|Community 478]] ## God Nodes (most connected - your core abstractions) 1. `ok()` - 86 edges 2. `WorkspaceModelingService` - 77 edges -3. `GlossaryService` - 59 edges -4. `isRecord()` - 55 edges -5. `toAdminApiError()` - 51 edges -6. `WorkspaceDatasourcePolicyRepository` - 45 edges -7. `RagRetrievalService` - 43 edges +3. `RagRetrievalService` - 63 edges +4. `GlossaryService` - 59 edges +5. `isRecord()` - 55 edges +6. `toAdminApiError()` - 51 edges +7. `WorkspaceDatasourcePolicyRepository` - 45 edges 8. `ChatRepository` - 43 edges -9. `AppConfigService` - 38 edges -10. `SemanticSpineRepository` - 35 edges +9. `AppConfigService` - 41 edges +10. `DeliveryContractMapper` - 41 edges ## Surprising Connections (you probably didn't know these) -- `setError()` --calls--> `save()` [INFERRED] - /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/components/settings/workspace-management-panel.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/components/settings/modeling/modeling-relationship-editor.tsx +- `loadPreviewForDetails()` --calls--> `getWorkspaceModelingPreview()` [INFERRED] + /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/settings/modeling/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/lib/admin-api-client.ts - `load()` --calls--> `setError()` [INFERRED] /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/app/glossary/page.tsx → /Users/lienli/Documents/GitHub/text2sql/apps/frontend/src/components/settings/workspace-management-panel.tsx - `load()` --calls--> `listGlossaryTerms()` [INFERRED] @@ -512,11 +518,11 @@ ### Community 0 - "Community 0" Cohesion: 0.03 -Nodes (15): DataBootstrapService, PlatformDataBootstrapModule, ChatPolicyGuardService, ChatRepository, ChatRunPersistenceService, ChatService, EvalService, ExecuteMessageUsecase (+7 more) +Nodes (15): ChatController, ChatPolicyGuardService, ChatRepository, ChatRunPersistenceService, ChatService, EvalService, ExecuteMessageUsecase, ExecuteSqlNode (+7 more) ### Community 1 - "Community 1" -Cohesion: 0.03 -Nodes (131): addWorkspaceDatasourceBindings(), addWorkspaceMembers(), AdminApiError, commitModelingSetup(), composeApiUrl(), createGlossaryAnchor(), createGlossaryTerm(), createPromptTemplate() (+123 more) +Cohesion: 0.02 +Nodes (42): extractLatestUserText(), mapToThreadMessages(), BuildRagIndexJob, parseSseEvents(), buildContextEnvelopeFromDraft(), parseEntityMappings(), splitByDelimiters(), trimOrUndefined() (+34 more) ### Community 2 - "Community 2" Cohesion: 0.02 @@ -524,19 +530,19 @@ Nodes (39): AppConfigService, DatasourceRegistryService, countTracedRuns(), main ### Community 3 - "Community 3" Cohesion: 0.03 -Nodes (45): extractLatestUserText(), mapToThreadMessages(), ChatDeliveryEnrichmentService, parseSseEvents(), buildContextEnvelopeFromDraft(), parseEntityMappings(), splitByDelimiters(), trimOrUndefined() (+37 more) +Nodes (10): AuditLogRepository, DataBootstrapService, PlatformDataBootstrapModule, FakePgClient, DatasourceRepository, DatasourceService, DatasourceWorkflowService, PostgresExecutorService (+2 more) ### Community 4 - "Community 4" -Cohesion: 0.03 -Nodes (14): ok(), ChatController, DatasourceController, EvalController, GlossaryController, MemoryController, SettingsController, UserController (+6 more) +Cohesion: 0.04 +Nodes (10): ModelingSchemaChangeRepository, toScopeKey(), SaveViewFromRunUsecase, normalizeName(), SchemaChangeDetectorService, WorkspaceDatasourceController, normalizeName(), WorkspaceModelingSchemaChangeDetectorService (+2 more) ### Community 5 - "Community 5" -Cohesion: 0.04 -Nodes (9): ModelingSchemaChangeRepository, toScopeKey(), SaveViewFromRunUsecase, normalizeName(), SchemaChangeDetectorService, normalizeName(), WorkspaceModelingSchemaChangeDetectorService, normalizeTableName() (+1 more) +Cohesion: 0.05 +Nodes (104): addWorkspaceDatasourceBindings(), addWorkspaceMembers(), AdminApiError, commitModelingSetup(), composeApiUrl(), createGlossaryAnchor(), createGlossaryTerm(), createPromptTemplate() (+96 more) ### Community 6 - "Community 6" -Cohesion: 0.05 -Nodes (8): BuildRagIndexJob, asActor(), asActor(), GlossaryService, RagIndexBuilderService, RagIndexRepository, withActor(), asAdmin() +Cohesion: 0.04 +Nodes (11): ok(), DatasourceController, EvalController, GlossaryController, MemoryController, SettingsController, UserController, WorkspaceController (+3 more) ### Community 7 - "Community 7" Cohesion: 0.04 @@ -544,79 +550,79 @@ Nodes (9): LlmConfigRepository, toModelHealthStatus(), toProviderCode(), toProvi ### Community 8 - "Community 8" Cohesion: 0.06 -Nodes (19): handleKeyDown(), onSubmit(), submit(), toVariableRows(), validateEmail(), validateVariableKey(), validateVariableValue(), toPlatformUserStatus() (+11 more) +Nodes (6): asActor(), asActor(), GlossaryService, RagIndexRepository, withActor(), asAdmin() ### Community 9 - "Community 9" Cohesion: 0.04 -Nodes (45): BuildSemanticQueryNode, buildConversationKnowledgeAllowlistSet(), collectFiles(), compactLine(), createLineStarts(), extractImportReferences(), findRepoRoot(), indexToLineColumn() (+37 more) +Nodes (65): handleKeyDown(), isRecord(), nextFieldId(), normalizeFunctionGroupHints(), resolveCalculatedFieldSaveError(), save(), toForm(), upsertField() (+57 more) ### Community 10 - "Community 10" Cohesion: 0.04 -Nodes (7): ChatPostRunHooksService, MemoryPromotionPolicy, MemoryPromotionService, RagBudgetPolicy, RagReplayRepository, RagRerankService, RetrieveKnowledgeNode +Nodes (57): BuildSemanticQueryNode, buildConversationKnowledgeAllowlistSet(), collectFiles(), compactLine(), createLineStarts(), extractImportReferences(), findRepoRoot(), indexToLineColumn() (+49 more) ### Community 11 - "Community 11" -Cohesion: 0.06 -Nodes (58): createIdempotencyKey(), resolveDatasourceWorkflowApiError(), applySessionView(), dedupeSessions(), extractErrorCode(), init(), loadMessages(), mergeSessionMessages() (+50 more) +Cohesion: 0.04 +Nodes (9): ChatPostRunHooksService, HealthController, MysqlExecutorService, RagIngestionMetricsService, RagQualityController, RagQualityService, SemanticSpineShadowService, SqliteExecutorService (+1 more) ### Community 12 - "Community 12" Cohesion: 0.06 -Nodes (5): DatasourceRepository, DatasourceWorkflowService, ExecuteSqlNode, RelationshipDryRunService, WorkspaceDatasourceService +Nodes (58): createIdempotencyKey(), resolveDatasourceWorkflowApiError(), applySessionView(), dedupeSessions(), extractErrorCode(), init(), loadMessages(), mergeSessionMessages() (+50 more) ### Community 13 - "Community 13" -Cohesion: 0.08 -Nodes (7): normalizeValue(), WorkspaceAdminGuard, toMembershipKey(), toWorkspaceMemberRole(), toWorkspaceStatus(), WorkspaceRepository, WorkspaceService +Cohesion: 0.05 +Nodes (8): GraphAccelerationCircuitBreaker, GraphService, QueryExecutorRouterService, RelationshipPublishGateFacade, RowFilterRewriteService, SafetyCheckNode, SqlSafetyGuard, SqlTableAccessGuardService ### Community 14 - "Community 14" -Cohesion: 0.09 -Nodes (7): DatasourceAccessPolicyService, asNonEmptyString(), toBindingKey(), toRuleKey(), toTablePermissionKey(), toTablePermissionSetKey(), WorkspaceDatasourcePolicyRepository +Cohesion: 0.08 +Nodes (7): normalizeValue(), WorkspaceAdminGuard, toMembershipKey(), toWorkspaceMemberRole(), toWorkspaceStatus(), WorkspaceRepository, WorkspaceService ### Community 15 - "Community 15" -Cohesion: 0.07 -Nodes (8): LaneTimeoutError, RagRetrievalService, appendUnique(), fuseWithRrf(), laneOrder(), normalizeRankConstant(), stableSortLaneHits(), SkillRegistryService +Cohesion: 0.05 +Nodes (4): MemoryPromotionPolicy, MemoryPromotionService, RagReplayRepository, RagRerankService ### Community 16 - "Community 16" Cohesion: 0.06 -Nodes (6): FakePgClient, DatasourceService, MysqlExecutorService, PostgresExecutorService, SqliteExecutorService, SqliteQueryService +Nodes (14): appendPlanningWarnings(), buildExplicitPinningEvidence(), createLangGraphNodeHandlers(), createNodeRuntime(), extractQualifiedColumns(), getCallbacks(), normalizeIdentifier(), summarize() (+6 more) ### Community 17 - "Community 17" -Cohesion: 0.05 -Nodes (13): GraphBuilderService, createLangGraphNodeHandlers(), createNodeRuntime(), getCallbacks(), summarize(), truncate(), createLangGraphRuntime(), LangGraphRuntimeService (+5 more) +Cohesion: 0.06 +Nodes (8): getChunkProfileConfig(), RagBudgetPolicy, RagChunkingService, sha256(), RagDocumentFactory, sha256(), RagEventConsumerService, SemanticRegistryService ### Community 18 - "Community 18" -Cohesion: 0.06 -Nodes (7): getChunkProfileConfig(), RagChunkingService, sha256(), RagDocumentFactory, sha256(), RagEventConsumerService, SemanticRegistryService +Cohesion: 0.09 +Nodes (7): DatasourceAccessPolicyService, asNonEmptyString(), toBindingKey(), toRuleKey(), toTablePermissionKey(), toTablePermissionSetKey(), WorkspaceDatasourcePolicyRepository ### Community 19 - "Community 19" Cohesion: 0.06 -Nodes (5): HealthController, RagIngestionMetricsService, RagQualityController, RagQualityService, SemanticSpineShadowService +Nodes (5): GenerateSqlNode, RagCacheKeyFactory, SqlGenerationService, SqlOutputExtractor, SqlPromptBuilder ### Community 20 - "Community 20" -Cohesion: 0.1 -Nodes (6): ModelingGraphRepository, toScopeKey(), ModelingGraphValidator, nonEmpty(), normalizeId(), WorkspaceRelationshipService +Cohesion: 0.08 +Nodes (10): toPlatformUserStatus(), UserRepository, UserService, onConfirmBatchDelete(), onConfirmDeleteOne(), onConfirmResetPassword(), onSubmitEditor(), onToggleStatus() (+2 more) ### Community 21 - "Community 21" -Cohesion: 0.07 -Nodes (6): QueryExecutorRouterService, RelationshipPublishGateFacade, RowFilterRewriteService, SafetyCheckNode, SqlSafetyGuard, SqlTableAccessGuardService +Cohesion: 0.08 +Nodes (7): DeliveryContractMapper, createDeliverySandboxPolicy(), isSandboxFilesystemWriteAllowed(), isSandboxNetworkAllowed(), isSandboxProcessSpawnAllowed(), normalizeString(), SandboxRuntimeService ### Community 22 - "Community 22" -Cohesion: 0.11 -Nodes (36): ApiClientRequestError, composeApiUrl(), createDatasource(), createSession(), DatasourceApiError, deleteSession(), getMessages(), getRun() (+28 more) +Cohesion: 0.1 +Nodes (6): ModelingGraphRepository, toScopeKey(), ModelingGraphValidator, nonEmpty(), normalizeId(), WorkspaceRelationshipService ### Community 23 - "Community 23" Cohesion: 0.09 -Nodes (35): createRunId(), withActor(), formatGovernanceError(), patchModel(), readRunIdFromQuery(), readWorkspaceIdFromQuery(), refreshModels(), resolveWorkspaceId() (+27 more) +Nodes (19): BuildIntentPlanNode, buildRuleReasonCodes(), ClarificationFusionPolicy, mapRuleDecisionToEvidence(), unique(), ClarifyNode, buildDecision(), buildFallbackSlotFillingDecision() (+11 more) ### Community 24 - "Community 24" -Cohesion: 0.1 -Nodes (2): AuditLogRepository, RagAuditReplayService +Cohesion: 0.11 +Nodes (36): ApiClientRequestError, composeApiUrl(), createDatasource(), createSession(), DatasourceApiError, deleteSession(), getMessages(), getRun() (+28 more) ### Community 25 - "Community 25" -Cohesion: 0.13 -Nodes (1): SemanticSpineRepository +Cohesion: 0.09 +Nodes (35): createRunId(), withActor(), formatGovernanceError(), patchModel(), readRunIdFromQuery(), readWorkspaceIdFromQuery(), refreshModels(), resolveWorkspaceId() (+27 more) ### Community 26 - "Community 26" Cohesion: 0.11 -Nodes (5): GenerateSqlNode, RagCacheKeyFactory, SqlGenerationService, SqlOutputExtractor, SqlPromptBuilder +Nodes (5): GraphBuilderService, createInitialLangGraphState(), normalizeAccessContext(), normalizeTraceContext(), LangsmithTraceService ### Community 27 - "Community 27" Cohesion: 0.12 @@ -627,225 +633,225 @@ Cohesion: 0.12 Nodes (11): GeminiAdapter, resolveGeminiUrl(), isTimeoutAbortError(), LlmGatewayService, toErrorMessage(), LlmModelFactory, resolveProviderBaseUrl(), checkHealth() (+3 more) ### Community 29 - "Community 29" -Cohesion: 0.14 -Nodes (2): GraphAccelerationCircuitBreaker, GraphService - -### Community 30 - "Community 30" Cohesion: 0.25 Nodes (19): ensureRunLoaded(), isRecord(), mergeRunThinkingSteps(), normalizeArtifact(), normalizeDeliveryContract(), normalizeEvidence(), normalizeRunForVisibility(), normalizeSelectedContext() (+11 more) -### Community 31 - "Community 31" +### Community 30 - "Community 30" Cohesion: 0.13 Nodes (7): formatCell(), normalizeNullableText(), normalizeTableKey(), openEditDialog(), resolveModelRelationships(), resolveRelationshipCounterpartTable(), submitMetadataUpdate() +### Community 31 - "Community 31" +Cohesion: 0.25 +Nodes (1): ChatDeliveryEnrichmentService + ### Community 32 - "Community 32" -Cohesion: 0.26 -Nodes (10): createIdempotencyKey(), isConflictError(), isRecord(), normalizeTableNames(), readNumberCandidate(), resolveConflictPolicyVersion(), resolveConflictSummary(), setsEqual() (+2 more) +Cohesion: 0.24 +Nodes (1): RagAuditReplayService ### Community 33 - "Community 33" -Cohesion: 0.29 -Nodes (1): FileDatasourceExecutorService +Cohesion: 0.22 +Nodes (2): ClarificationSemanticEvaluatorService, SemanticEvaluatorTimeoutError ### Community 34 - "Community 34" -Cohesion: 0.31 -Nodes (12): buildCreateForm(), handleTableChange(), makeRelationshipId(), normalizeTableName(), openCreateDialog(), openEditDialog(), resolveColumnOptions(), resolveRelationshipType() (+4 more) +Cohesion: 0.26 +Nodes (10): createIdempotencyKey(), isConflictError(), isRecord(), normalizeTableNames(), readNumberCandidate(), resolveConflictPolicyVersion(), resolveConflictSummary(), setsEqual() (+2 more) ### Community 35 - "Community 35" +Cohesion: 0.29 +Nodes (1): FileDatasourceExecutorService + +### Community 36 - "Community 36" Cohesion: 0.28 Nodes (4): GraphAccelerationAdapter, GraphAccelerationError, GraphAccelerationTimeoutError, GraphAccelerationUnsupportedOperatorError -### Community 36 - "Community 36" +### Community 37 - "Community 37" Cohesion: 0.26 Nodes (1): SemanticSpineShadowGateService -### Community 37 - "Community 37" +### Community 38 - "Community 38" Cohesion: 0.19 Nodes (2): GateMetricsService, TraceService -### Community 38 - "Community 38" -Cohesion: 0.29 -Nodes (9): ClarifyNode, decideSlotFilling(), hasEntityMappings(), hasNonEmpty(), hasStringArray(), hasTimeRangeEnvelope(), resolveClarificationQuestion(), resolveReason() (+1 more) - ### Community 39 - "Community 39" +Cohesion: 0.36 +Nodes (1): SkillRegistryService + +### Community 40 - "Community 40" Cohesion: 0.25 Nodes (3): JsonFileSemanticSpineRepositoryDouble, SemanticSpineContractError, validateSemanticObject() -### Community 40 - "Community 40" +### Community 41 - "Community 41" Cohesion: 0.29 Nodes (2): BuildPhysicalPlanNode, PlannerCacheService -### Community 41 - "Community 41" +### Community 42 - "Community 42" Cohesion: 0.25 Nodes (0): -### Community 42 - "Community 42" -Cohesion: 0.46 -Nodes (7): isRecord(), nextFieldId(), normalizeFunctionGroupHints(), resolveCalculatedFieldSaveError(), save(), toForm(), upsertField() - ### Community 43 - "Community 43" -Cohesion: 0.29 -Nodes (3): InMemorySemanticSpineRepositoryDouble, SemanticSpineContractError, validateSemanticObject() +Cohesion: 0.25 +Nodes (0): ### Community 44 - "Community 44" Cohesion: 0.29 -Nodes (0): +Nodes (3): InMemorySemanticSpineRepositoryDouble, SemanticSpineContractError, validateSemanticObject() ### Community 45 - "Community 45" Cohesion: 0.29 Nodes (0): ### Community 46 - "Community 46" +Cohesion: 0.29 +Nodes (0): + +### Community 47 - "Community 47" Cohesion: 0.33 Nodes (2): providerStatusLabel(), providerStatusVariant() -### Community 47 - "Community 47" +### Community 48 - "Community 48" +Cohesion: 0.38 +Nodes (4): createModelingFlowFieldHandleId(), createModelingFlowRelationshipHandleId(), ModelingFlowNode(), normalizeHandleToken() + +### Community 49 - "Community 49" +Cohesion: 0.48 +Nodes (6): computeElkLayout(), ElkLayoutTimeoutError, sortEdges(), sortNodes(), toElapsedMs(), withTimeout() + +### Community 50 - "Community 50" Cohesion: 0.57 Nodes (2): IngestionSourceAdapter, uniqueNonEmpty() -### Community 48 - "Community 48" +### Community 51 - "Community 51" Cohesion: 0.29 Nodes (1): KnowledgeChatSupportFacade -### Community 49 - "Community 49" +### Community 52 - "Community 52" Cohesion: 0.33 Nodes (0): -### Community 50 - "Community 50" +### Community 53 - "Community 53" Cohesion: 0.33 Nodes (0): -### Community 51 - "Community 51" +### Community 54 - "Community 54" Cohesion: 0.33 Nodes (0): -### Community 52 - "Community 52" +### Community 55 - "Community 55" Cohesion: 0.33 Nodes (0): -### Community 53 - "Community 53" +### Community 56 - "Community 56" Cohesion: 0.4 Nodes (3): buildReadySnapshot(), buildSnakeCaseSnapshot(), SemanticSpineRepositoryStub -### Community 54 - "Community 54" +### Community 57 - "Community 57" Cohesion: 0.33 Nodes (1): PolicyEvaluatorService -### Community 55 - "Community 55" +### Community 58 - "Community 58" Cohesion: 0.33 Nodes (4): RelationshipBridgeDto, RelationshipBridgeEndpointDto, RelationshipGraphEdgeDto, ReplaceWorkspaceRelationshipGraphDto -### Community 56 - "Community 56" +### Community 59 - "Community 59" Cohesion: 0.4 Nodes (1): ResizeObserver -### Community 57 - "Community 57" +### Community 60 - "Community 60" Cohesion: 0.4 Nodes (0): -### Community 58 - "Community 58" +### Community 61 - "Community 61" Cohesion: 0.4 Nodes (0): -### Community 59 - "Community 59" +### Community 62 - "Community 62" Cohesion: 0.4 Nodes (0): -### Community 60 - "Community 60" +### Community 63 - "Community 63" Cohesion: 0.5 Nodes (2): CarouselNext(), useCarousel() -### Community 61 - "Community 61" +### Community 64 - "Community 64" Cohesion: 0.6 Nodes (4): nextEdgeId(), RelationshipEdgeEditorDialog(), toEdge(), toForm() -### Community 62 - "Community 62" -Cohesion: 0.4 -Nodes (0): - -### Community 63 - "Community 63" +### Community 65 - "Community 65" Cohesion: 0.4 Nodes (0): -### Community 64 - "Community 64" +### Community 66 - "Community 66" Cohesion: 0.4 Nodes (0): -### Community 65 - "Community 65" +### Community 67 - "Community 67" Cohesion: 0.5 Nodes (2): statusLabel(), statusVariant() -### Community 66 - "Community 66" +### Community 68 - "Community 68" Cohesion: 0.8 Nodes (4): normalizeConfidence(), normalizeEndpoint(), normalizeRelationshipGraphEdge(), normalizeRequired() -### Community 67 - "Community 67" +### Community 69 - "Community 69" Cohesion: 0.4 Nodes (4): UpsertDatasourceWorkflowDto, WorkflowDatasourcePayloadDto, WorkflowWorkspaceCreateDto, WorkflowWorkspacePayloadDto -### Community 68 - "Community 68" +### Community 70 - "Community 70" Cohesion: 0.5 Nodes (0): -### Community 69 - "Community 69" +### Community 71 - "Community 71" Cohesion: 0.5 Nodes (0): -### Community 70 - "Community 70" +### Community 72 - "Community 72" Cohesion: 0.5 Nodes (0): -### Community 71 - "Community 71" +### Community 73 - "Community 73" +Cohesion: 0.5 +Nodes (1): TimeoutEvaluator + +### Community 74 - "Community 74" Cohesion: 0.5 Nodes (1): FakePgClient -### Community 72 - "Community 72" +### Community 75 - "Community 75" Cohesion: 0.83 Nodes (3): buildModelingRepository(), buildModelingService(), buildRelationshipService() -### Community 73 - "Community 73" +### Community 76 - "Community 76" Cohesion: 0.5 Nodes (1): RagModule -### Community 74 - "Community 74" +### Community 77 - "Community 77" Cohesion: 0.5 Nodes (1): ListPromptTemplatesQueryDto -### Community 75 - "Community 75" +### Community 78 - "Community 78" Cohesion: 0.5 Nodes (1): UpsertModelingSetupDto -### Community 76 - "Community 76" +### Community 79 - "Community 79" Cohesion: 0.5 Nodes (1): CreateGlossaryTermDto -### Community 77 - "Community 77" +### Community 80 - "Community 80" Cohesion: 0.5 Nodes (1): UpdateGlossaryTermDto -### Community 78 - "Community 78" +### Community 81 - "Community 81" Cohesion: 0.5 Nodes (3): ContextEnvelopeDto, ContextEnvelopeEntityMappingDto, ContextEnvelopeTimeRangeDto -### Community 79 - "Community 79" +### Community 82 - "Community 82" Cohesion: 0.5 Nodes (0): -### Community 80 - "Community 80" +### Community 83 - "Community 83" Cohesion: 1.0 Nodes (2): createDatasourceAndOpenSetupWizard(), openCreateToStep2() -### Community 81 - "Community 81" -Cohesion: 0.67 -Nodes (0): - -### Community 82 - "Community 82" -Cohesion: 0.67 -Nodes (0): - -### Community 83 - "Community 83" -Cohesion: 0.67 -Nodes (0): - ### Community 84 - "Community 84" Cohesion: 0.67 Nodes (0): @@ -887,8 +893,8 @@ Cohesion: 0.67 Nodes (0): ### Community 94 - "Community 94" -Cohesion: 1.0 -Nodes (2): resolveBadgeLabel(), resolveBadgeVariant() +Cohesion: 0.67 +Nodes (0): ### Community 95 - "Community 95" Cohesion: 0.67 @@ -899,8 +905,8 @@ Cohesion: 0.67 Nodes (0): ### Community 97 - "Community 97" -Cohesion: 0.67 -Nodes (0): +Cohesion: 1.0 +Nodes (2): resolveBadgeLabel(), resolveBadgeVariant() ### Community 98 - "Community 98" Cohesion: 0.67 @@ -908,147 +914,147 @@ Nodes (0): ### Community 99 - "Community 99" Cohesion: 0.67 -Nodes (1): DomainError +Nodes (0): ### Community 100 - "Community 100" Cohesion: 0.67 -Nodes (1): OpenRouterAdapter +Nodes (0): ### Community 101 - "Community 101" Cohesion: 0.67 -Nodes (1): KimiAdapter +Nodes (0): ### Community 102 - "Community 102" Cohesion: 0.67 -Nodes (1): VolcengineAdapter +Nodes (0): ### Community 103 - "Community 103" Cohesion: 0.67 -Nodes (1): MinimaxAdapter +Nodes (1): DomainError ### Community 104 - "Community 104" Cohesion: 0.67 -Nodes (1): TencentHunyuanAdapter +Nodes (1): OpenRouterAdapter ### Community 105 - "Community 105" Cohesion: 0.67 -Nodes (1): SiliconflowAdapter +Nodes (1): KimiAdapter ### Community 106 - "Community 106" Cohesion: 0.67 -Nodes (1): DeepSeekAdapter +Nodes (1): VolcengineAdapter ### Community 107 - "Community 107" Cohesion: 0.67 -Nodes (1): OpenAiAdapter +Nodes (1): MinimaxAdapter ### Community 108 - "Community 108" Cohesion: 0.67 -Nodes (1): TongyiAdapter +Nodes (1): TencentHunyuanAdapter ### Community 109 - "Community 109" Cohesion: 0.67 -Nodes (1): MemoryModule +Nodes (1): SiliconflowAdapter ### Community 110 - "Community 110" Cohesion: 0.67 -Nodes (1): ChatModule +Nodes (1): DeepSeekAdapter ### Community 111 - "Community 111" Cohesion: 0.67 -Nodes (1): SendMessageDto +Nodes (1): OpenAiAdapter ### Community 112 - "Community 112" Cohesion: 0.67 -Nodes (1): ListSessionsDto +Nodes (1): TongyiAdapter ### Community 113 - "Community 113" Cohesion: 0.67 -Nodes (1): CreateSessionDto +Nodes (1): MemoryModule ### Community 114 - "Community 114" Cohesion: 0.67 -Nodes (1): RenameSessionDto +Nodes (1): ChatModule ### Community 115 - "Community 115" Cohesion: 0.67 -Nodes (1): AdminOnlyGuard +Nodes (1): SendMessageDto ### Community 116 - "Community 116" Cohesion: 0.67 -Nodes (1): PlatformChatRuntimeFacade +Nodes (1): ListSessionsDto ### Community 117 - "Community 117" Cohesion: 0.67 -Nodes (1): GlossaryModule +Nodes (1): CreateSessionDto ### Community 118 - "Community 118" Cohesion: 0.67 -Nodes (1): RelationshipImpactSimulatorService +Nodes (1): RenameSessionDto ### Community 119 - "Community 119" Cohesion: 0.67 -Nodes (1): SemanticRegistryModule +Nodes (1): AdminOnlyGuard ### Community 120 - "Community 120" Cohesion: 0.67 -Nodes (1): GovernanceChatAccessFacade +Nodes (1): PlatformChatRuntimeFacade ### Community 121 - "Community 121" Cohesion: 0.67 -Nodes (2): PreviewDatasourcePayloadDto, PreviewDatasourceTablesDto +Nodes (1): GlossaryModule ### Community 122 - "Community 122" Cohesion: 0.67 -Nodes (1): UpdatePromptTemplateDto +Nodes (1): RelationshipImpactSimulatorService ### Community 123 - "Community 123" Cohesion: 0.67 -Nodes (1): CreatePromptTemplateDto +Nodes (1): SemanticRegistryModule ### Community 124 - "Community 124" Cohesion: 0.67 -Nodes (1): UpsertModelingGraphDto +Nodes (1): GovernanceChatAccessFacade ### Community 125 - "Community 125" Cohesion: 0.67 -Nodes (1): ListWorkspaceDatasourceTablePermissionsDto +Nodes (2): PreviewDatasourcePayloadDto, PreviewDatasourceTablesDto ### Community 126 - "Community 126" Cohesion: 0.67 -Nodes (1): ResolveModelingSchemaChangeDto +Nodes (1): UpdatePromptTemplateDto ### Community 127 - "Community 127" Cohesion: 0.67 -Nodes (1): ReplaceWorkspaceDatasourceTablePermissionsDto +Nodes (1): CreatePromptTemplateDto ### Community 128 - "Community 128" Cohesion: 0.67 -Nodes (1): ListUsersDto +Nodes (1): UpsertModelingGraphDto ### Community 129 - "Community 129" Cohesion: 0.67 -Nodes (1): ListGlossaryTermsQueryDto +Nodes (1): ListWorkspaceDatasourceTablePermissionsDto ### Community 130 - "Community 130" Cohesion: 0.67 -Nodes (1): RollbackGlossaryAnchorDto +Nodes (1): ResolveModelingSchemaChangeDto ### Community 131 - "Community 131" Cohesion: 0.67 -Nodes (1): BuildIntentPlanNode +Nodes (1): ReplaceWorkspaceDatasourceTablePermissionsDto ### Community 132 - "Community 132" -Cohesion: 1.0 -Nodes (0): +Cohesion: 0.67 +Nodes (1): ListUsersDto ### Community 133 - "Community 133" -Cohesion: 1.0 -Nodes (0): +Cohesion: 0.67 +Nodes (1): ListGlossaryTermsQueryDto ### Community 134 - "Community 134" -Cohesion: 1.0 -Nodes (0): +Cohesion: 0.67 +Nodes (1): RollbackGlossaryAnchorDto ### Community 135 - "Community 135" Cohesion: 1.0 @@ -1072,7 +1078,7 @@ Nodes (0): ### Community 140 - "Community 140" Cohesion: 1.0 -Nodes (1): ELK +Nodes (0): ### Community 141 - "Community 141" Cohesion: 1.0 @@ -1084,7 +1090,7 @@ Nodes (0): ### Community 143 - "Community 143" Cohesion: 1.0 -Nodes (0): +Nodes (1): ELK ### Community 144 - "Community 144" Cohesion: 1.0 @@ -1360,7 +1366,7 @@ Nodes (0): ### Community 212 - "Community 212" Cohesion: 1.0 -Nodes (1): AppModule +Nodes (0): ### Community 213 - "Community 213" Cohesion: 1.0 @@ -1368,215 +1374,215 @@ Nodes (0): ### Community 214 - "Community 214" Cohesion: 1.0 -Nodes (1): LlmModule +Nodes (0): ### Community 215 - "Community 215" Cohesion: 1.0 -Nodes (1): ApplyMemoryFeedbackDto +Nodes (0): ### Community 216 - "Community 216" Cohesion: 1.0 -Nodes (1): AppConfigModule +Nodes (0): ### Community 217 - "Community 217" Cohesion: 1.0 -Nodes (1): PlatformModule +Nodes (1): AppModule ### Community 218 - "Community 218" Cohesion: 1.0 -Nodes (1): PlatformLlmModule +Nodes (0): ### Community 219 - "Community 219" Cohesion: 1.0 -Nodes (1): PlatformConfigModule +Nodes (1): LlmModule ### Community 220 - "Community 220" Cohesion: 1.0 -Nodes (1): PlatformObservabilityModule +Nodes (1): ApplyMemoryFeedbackDto ### Community 221 - "Community 221" Cohesion: 1.0 -Nodes (1): PlatformDataAccessModule +Nodes (1): AppConfigModule ### Community 222 - "Community 222" Cohesion: 1.0 -Nodes (1): PlatformDataQueryModule +Nodes (1): PlatformModule ### Community 223 - "Community 223" Cohesion: 1.0 -Nodes (1): PlatformDataModule +Nodes (1): PlatformLlmModule ### Community 224 - "Community 224" Cohesion: 1.0 -Nodes (1): PlatformDataPersistenceModule +Nodes (1): PlatformConfigModule ### Community 225 - "Community 225" Cohesion: 1.0 -Nodes (1): ObservabilityModule +Nodes (1): PlatformObservabilityModule ### Community 226 - "Community 226" Cohesion: 1.0 -Nodes (1): SystemModule +Nodes (1): PlatformDataAccessModule ### Community 227 - "Community 227" Cohesion: 1.0 -Nodes (1): KnowledgeModule +Nodes (1): PlatformDataQueryModule ### Community 228 - "Community 228" Cohesion: 1.0 -Nodes (1): SemanticSpineModule +Nodes (1): PlatformDataModule ### Community 229 - "Community 229" Cohesion: 1.0 -Nodes (1): SkillRegistryModule +Nodes (1): PlatformDataPersistenceModule ### Community 230 - "Community 230" Cohesion: 1.0 -Nodes (1): GovernanceModule +Nodes (1): ObservabilityModule ### Community 231 - "Community 231" Cohesion: 1.0 -Nodes (1): DatasourceModule +Nodes (1): SystemModule ### Community 232 - "Community 232" Cohesion: 1.0 -Nodes (1): CreateDatasourceDto +Nodes (1): KnowledgeModule ### Community 233 - "Community 233" Cohesion: 1.0 -Nodes (1): UploadFileDatasourceDto +Nodes (1): SemanticSpineModule ### Community 234 - "Community 234" Cohesion: 1.0 -Nodes (1): UpdateDatasourceDto +Nodes (1): SkillRegistryModule ### Community 235 - "Community 235" Cohesion: 1.0 -Nodes (1): SettingsModule +Nodes (1): GovernanceModule ### Community 236 - "Community 236" Cohesion: 1.0 -Nodes (1): BatchUpdateModelStatusDto +Nodes (1): DatasourceModule ### Community 237 - "Community 237" Cohesion: 1.0 -Nodes (1): UpdateModelStatusDto +Nodes (1): CreateDatasourceDto ### Community 238 - "Community 238" Cohesion: 1.0 -Nodes (1): RefreshProviderModelsDto +Nodes (1): UploadFileDatasourceDto ### Community 239 - "Community 239" Cohesion: 1.0 -Nodes (1): CreateProviderDto +Nodes (1): UpdateDatasourceDto ### Community 240 - "Community 240" Cohesion: 1.0 -Nodes (1): GovernanceAccessModule +Nodes (1): SettingsModule ### Community 241 - "Community 241" Cohesion: 1.0 -Nodes (1): WorkspaceModule +Nodes (1): BatchUpdateModelStatusDto ### Community 242 - "Community 242" Cohesion: 1.0 -Nodes (1): RemoveWorkspaceMembersDto +Nodes (1): UpdateModelStatusDto ### Community 243 - "Community 243" Cohesion: 1.0 -Nodes (1): RenameWorkspaceDto +Nodes (1): RefreshProviderModelsDto ### Community 244 - "Community 244" Cohesion: 1.0 -Nodes (1): GetModelingPreviewDto +Nodes (1): CreateProviderDto ### Community 245 - "Community 245" Cohesion: 1.0 -Nodes (1): UpdateWorkspaceMemberRoleDto +Nodes (1): GovernanceAccessModule ### Community 246 - "Community 246" Cohesion: 1.0 -Nodes (1): DetectModelingSchemaChangeDto +Nodes (1): WorkspaceModule ### Community 247 - "Community 247" Cohesion: 1.0 -Nodes (1): ListWorkspaceMembersDto +Nodes (1): RemoveWorkspaceMembersDto ### Community 248 - "Community 248" Cohesion: 1.0 -Nodes (1): DeployModelingGraphDto +Nodes (1): RenameWorkspaceDto ### Community 249 - "Community 249" Cohesion: 1.0 -Nodes (1): AddWorkspaceMemberDto +Nodes (1): GetModelingPreviewDto ### Community 250 - "Community 250" Cohesion: 1.0 -Nodes (1): CreateWorkspaceDto +Nodes (1): UpdateWorkspaceMemberRoleDto ### Community 251 - "Community 251" Cohesion: 1.0 -Nodes (1): UserModule +Nodes (1): DetectModelingSchemaChangeDto ### Community 252 - "Community 252" Cohesion: 1.0 -Nodes (1): BatchDeleteUsersDto +Nodes (1): ListWorkspaceMembersDto ### Community 253 - "Community 253" Cohesion: 1.0 -Nodes (1): UpdateUserStatusDto +Nodes (1): DeployModelingGraphDto ### Community 254 - "Community 254" Cohesion: 1.0 -Nodes (1): UpdateUserDto +Nodes (1): AddWorkspaceMemberDto ### Community 255 - "Community 255" Cohesion: 1.0 -Nodes (1): UserVariableDto +Nodes (1): CreateWorkspaceDto ### Community 256 - "Community 256" Cohesion: 1.0 -Nodes (1): CreateUserDto +Nodes (1): UserModule ### Community 257 - "Community 257" Cohesion: 1.0 -Nodes (1): EvalModule +Nodes (1): BatchDeleteUsersDto ### Community 258 - "Community 258" Cohesion: 1.0 -Nodes (1): RunEvaluationDto +Nodes (1): UpdateUserStatusDto ### Community 259 - "Community 259" Cohesion: 1.0 -Nodes (1): ConversationModule +Nodes (1): UpdateUserDto ### Community 260 - "Community 260" Cohesion: 1.0 -Nodes (1): SaveViewFromRunDto +Nodes (1): UserVariableDto ### Community 261 - "Community 261" Cohesion: 1.0 -Nodes (1): AgentModule +Nodes (1): CreateUserDto ### Community 262 - "Community 262" Cohesion: 1.0 -Nodes (0): +Nodes (1): EvalModule ### Community 263 - "Community 263" Cohesion: 1.0 -Nodes (0): +Nodes (1): RunEvaluationDto ### Community 264 - "Community 264" Cohesion: 1.0 -Nodes (0): +Nodes (1): ConversationModule ### Community 265 - "Community 265" Cohesion: 1.0 -Nodes (0): +Nodes (1): SaveViewFromRunDto ### Community 266 - "Community 266" Cohesion: 1.0 -Nodes (0): +Nodes (1): AgentModule ### Community 267 - "Community 267" Cohesion: 1.0 @@ -2402,20 +2408,44 @@ Nodes (0): Cohesion: 1.0 Nodes (0): +### Community 473 - "Community 473" +Cohesion: 1.0 +Nodes (0): + +### Community 474 - "Community 474" +Cohesion: 1.0 +Nodes (0): + +### Community 475 - "Community 475" +Cohesion: 1.0 +Nodes (0): + +### Community 476 - "Community 476" +Cohesion: 1.0 +Nodes (0): + +### Community 477 - "Community 477" +Cohesion: 1.0 +Nodes (0): + +### Community 478 - "Community 478" +Cohesion: 1.0 +Nodes (0): + ## Knowledge Gaps - **80 isolated node(s):** `ELK`, `ElkLayoutTimeoutError`, `AppModule`, `LlmModule`, `ApplyMemoryFeedbackDto` (+75 more) These have ≤1 connection - possible missing edges or undocumented components. -- **Thin community `Community 132`** (2 nodes): `pickSelectOption()`, `settings-modeling-relationship-editor.spec.tsx` +- **Thin community `Community 135`** (2 nodes): `pickSelectOption()`, `settings-modeling-relationship-editor.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 133`** (2 nodes): `openCreateToStep2()`, `data-sources-page.spec.tsx` +- **Thin community `Community 136`** (2 nodes): `openCreateToStep2()`, `data-sources-page.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 134`** (2 nodes): `SectionHarness()`, `rag-delivery-section.spec.tsx` +- **Thin community `Community 137`** (2 nodes): `SectionHarness()`, `rag-delivery-section.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 135`** (2 nodes): `waitForWorkspaceDatasourceReady()`, `settings-modeling-workspace.spec.tsx` +- **Thin community `Community 138`** (2 nodes): `waitForWorkspaceDatasourceReady()`, `settings-modeling-workspace.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 136`** (2 nodes): `createDelivery()`, `rag-delivery-panel.spec.tsx` +- **Thin community `Community 139`** (2 nodes): `createDelivery()`, `rag-delivery-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 137`** (2 nodes): `[ +- **Thin community `Community 140`** (2 nodes): `[ { id: "model.orders", tableName: "orders", @@ -2426,686 +2456,690 @@ Nodes (0): } ]()`, `settings-modeling-details-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 138`** (2 nodes): `onUnhandledRejection()`, `chat-panel.spec.tsx` +- **Thin community `Community 141`** (2 nodes): `onUnhandledRejection()`, `chat-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 139`** (2 nodes): `createSettingsView()`, `settings-mobile-smoke.spec.tsx` +- **Thin community `Community 142`** (2 nodes): `createSettingsView()`, `settings-mobile-smoke.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 140`** (2 nodes): `ELK`, `elkjs.d.ts` +- **Thin community `Community 143`** (2 nodes): `ELK`, `elkjs.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 141`** (2 nodes): `RootLayout()`, `layout.tsx` +- **Thin community `Community 144`** (2 nodes): `RootLayout()`, `layout.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 142`** (2 nodes): `ChatPage()`, `page.tsx` +- **Thin community `Community 145`** (2 nodes): `ChatPage()`, `page.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 143`** (2 nodes): `AppShell()`, `app-shell.tsx` +- **Thin community `Community 146`** (2 nodes): `AppShell()`, `app-shell.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 144`** (2 nodes): `AspectRatio()`, `aspect-ratio.tsx` +- **Thin community `Community 147`** (2 nodes): `AspectRatio()`, `aspect-ratio.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 145`** (2 nodes): `StateBlock()`, `state-block.tsx` +- **Thin community `Community 148`** (2 nodes): `StateBlock()`, `state-block.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 146`** (2 nodes): `DirectionProvider()`, `direction.tsx` +- **Thin community `Community 149`** (2 nodes): `DirectionProvider()`, `direction.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 147`** (2 nodes): `cn()`, `tabs.tsx` +- **Thin community `Community 150`** (2 nodes): `cn()`, `tabs.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 148`** (2 nodes): `ButtonGroup()`, `button-group.tsx` +- **Thin community `Community 151`** (2 nodes): `ButtonGroup()`, `button-group.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 149`** (2 nodes): `cn()`, `card.tsx` +- **Thin community `Community 152`** (2 nodes): `cn()`, `card.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 150`** (2 nodes): `InputGroup()`, `input-group.tsx` +- **Thin community `Community 153`** (2 nodes): `InputGroup()`, `input-group.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 151`** (2 nodes): `cn()`, `input-otp.tsx` +- **Thin community `Community 154`** (2 nodes): `cn()`, `input-otp.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 152`** (2 nodes): `HoverCard()`, `hover-card.tsx` +- **Thin community `Community 155`** (2 nodes): `HoverCard()`, `hover-card.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 153`** (2 nodes): `cn()`, `field.tsx` +- **Thin community `Community 156`** (2 nodes): `cn()`, `field.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 154`** (2 nodes): `Label()`, `label.tsx` +- **Thin community `Community 157`** (2 nodes): `Label()`, `label.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 155`** (2 nodes): `cn()`, `empty.tsx` +- **Thin community `Community 158`** (2 nodes): `cn()`, `empty.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 156`** (2 nodes): `TooltipContent()`, `tooltip.tsx` +- **Thin community `Community 159`** (2 nodes): `TooltipContent()`, `tooltip.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 157`** (2 nodes): `cn()`, `alert.tsx` +- **Thin community `Community 160`** (2 nodes): `cn()`, `alert.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 158`** (2 nodes): `Switch()`, `switch.tsx` +- **Thin community `Community 161`** (2 nodes): `Switch()`, `switch.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 159`** (2 nodes): `cn()`, `command.tsx` +- **Thin community `Community 162`** (2 nodes): `cn()`, `command.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 160`** (2 nodes): `ToggleGroup()`, `toggle-group.tsx` +- **Thin community `Community 163`** (2 nodes): `ToggleGroup()`, `toggle-group.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 161`** (2 nodes): `Avatar()`, `avatar.tsx` +- **Thin community `Community 164`** (2 nodes): `Avatar()`, `avatar.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 162`** (2 nodes): `cn()`, `kbd.tsx` +- **Thin community `Community 165`** (2 nodes): `cn()`, `kbd.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 163`** (2 nodes): `SectionHeader()`, `section-header.tsx` +- **Thin community `Community 166`** (2 nodes): `SectionHeader()`, `section-header.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 164`** (2 nodes): `Badge()`, `badge.tsx` +- **Thin community `Community 167`** (2 nodes): `Badge()`, `badge.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 165`** (2 nodes): `cn()`, `table.tsx` +- **Thin community `Community 168`** (2 nodes): `cn()`, `table.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 166`** (2 nodes): `Separator()`, `separator.tsx` +- **Thin community `Community 169`** (2 nodes): `Separator()`, `separator.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 167`** (2 nodes): `Checkbox()`, `checkbox.tsx` +- **Thin community `Community 170`** (2 nodes): `Checkbox()`, `checkbox.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 168`** (2 nodes): `Spinner()`, `spinner.tsx` +- **Thin community `Community 171`** (2 nodes): `Spinner()`, `spinner.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 169`** (2 nodes): `Collapsible()`, `collapsible.tsx` +- **Thin community `Community 172`** (2 nodes): `Collapsible()`, `collapsible.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 170`** (2 nodes): `cn()`, `textarea.tsx` +- **Thin community `Community 173`** (2 nodes): `cn()`, `textarea.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 171`** (2 nodes): `Input()`, `input.tsx` +- **Thin community `Community 174`** (2 nodes): `Input()`, `input.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 172`** (2 nodes): `Skeleton()`, `skeleton.tsx` +- **Thin community `Community 175`** (2 nodes): `Skeleton()`, `skeleton.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 173`** (2 nodes): `workspace-editor-dialog.tsx`, `WorkspaceEditorDialog()` +- **Thin community `Community 176`** (2 nodes): `workspace-editor-dialog.tsx`, `WorkspaceEditorDialog()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 174`** (2 nodes): `handleKeyDown()`, `modeling-context-drawer.tsx` +- **Thin community `Community 177`** (2 nodes): `handleKeyDown()`, `modeling-context-drawer.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 175`** (2 nodes): `resolveBlockingReasonGuidance()`, `modeling-deploy-panel.tsx` +- **Thin community `Community 178`** (2 nodes): `resolveBlockingReasonGuidance()`, `modeling-deploy-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 176`** (2 nodes): `TextPart()`, `assistant-message.tsx` +- **Thin community `Community 179`** (2 nodes): `TextPart()`, `assistant-message.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 177`** (2 nodes): `resolveMessageRunId()`, `assistant-thread.tsx` +- **Thin community `Community 180`** (2 nodes): `resolveMessageRunId()`, `assistant-thread.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 178`** (2 nodes): `ModelSelector()`, `model-selector.tsx` +- **Thin community `Community 181`** (2 nodes): `ModelSelector()`, `model-selector.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 179`** (2 nodes): `SessionActions()`, `session-actions.tsx` +- **Thin community `Community 182`** (2 nodes): `SessionActions()`, `session-actions.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 180`** (2 nodes): `SqlDebugDetails()`, `sql-debug-details.tsx` +- **Thin community `Community 183`** (2 nodes): `SqlDebugDetails()`, `sql-debug-details.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 181`** (2 nodes): `SqlRunSummary()`, `sql-run-summary.tsx` +- **Thin community `Community 184`** (2 nodes): `SqlRunSummary()`, `sql-run-summary.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 182`** (2 nodes): `isNumericValue()`, `sql-result-table.tsx` +- **Thin community `Community 185`** (2 nodes): `isNumericValue()`, `sql-result-table.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 183`** (2 nodes): `statusVariant()`, `sql-execution-timeline.tsx` +- **Thin community `Community 186`** (2 nodes): `statusVariant()`, `sql-execution-timeline.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 184`** (2 nodes): `useIsMobile()`, `use-mobile.ts` +- **Thin community `Community 187`** (2 nodes): `useIsMobile()`, `use-mobile.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 185`** (2 nodes): `utils.ts`, `cn()` +- **Thin community `Community 188`** (2 nodes): `utils.ts`, `cn()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 186`** (2 nodes): `createService()`, `sql-generation.service.spec.ts` +- **Thin community `Community 189`** (2 nodes): `createService()`, `sql-generation.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 187`** (2 nodes): `createService()`, `semantic-registry.service.spec.ts` +- **Thin community `Community 190`** (2 nodes): `createService()`, `semantic-registry.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 188`** (2 nodes): `createBaseRun()`, `delivery-contract.mapper.spec.ts` +- **Thin community `Community 191`** (2 nodes): `createBaseRun()`, `delivery-contract.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 189`** (2 nodes): `createRequest()`, `request-actor.middleware.spec.ts` +- **Thin community `Community 192`** (2 nodes): `createRequest()`, `request-actor.middleware.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 190`** (2 nodes): `workspace-admin.guard.spec.ts`, `createContext()` +- **Thin community `Community 193`** (2 nodes): `workspace-admin.guard.spec.ts`, `createContext()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 191`** (2 nodes): `reset()`, `langsmith-config.spec.ts` +- **Thin community `Community 194`** (2 nodes): `reset()`, `langsmith-config.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 192`** (2 nodes): `createContext()`, `admin-only.guard.spec.ts` +- **Thin community `Community 195`** (2 nodes): `createNode()`, `build-semantic-query.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 193`** (2 nodes): `resetAppEnv()`, `config.module.spec.ts` +- **Thin community `Community 196`** (2 nodes): `createContext()`, `admin-only.guard.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 194`** (2 nodes): `writeRepoFile()`, `capability-boundary-check.spec.ts` +- **Thin community `Community 197`** (2 nodes): `resetAppEnv()`, `config.module.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 195`** (2 nodes): `buildLongSchemaText()`, `rag-chunking.service.spec.ts` +- **Thin community `Community 198`** (2 nodes): `writeRepoFile()`, `capability-boundary-check.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 196`** (2 nodes): `datasource()`, `query-executor-router.spec.ts` +- **Thin community `Community 199`** (2 nodes): `createPolicy()`, `clarification-fusion.policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 197`** (2 nodes): `createBaseRun()`, `sandbox-failover.spec.ts` +- **Thin community `Community 200`** (2 nodes): `buildLongSchemaText()`, `rag-chunking.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 198`** (2 nodes): `buildUsecase()`, `save-view-from-run.spec.ts` +- **Thin community `Community 201`** (2 nodes): `datasource()`, `query-executor-router.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 199`** (2 nodes): `workspace-relationship-api.spec.ts`, `buildService()` +- **Thin community `Community 202`** (2 nodes): `createNode()`, `build-intent-plan.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 200`** (2 nodes): `buildArtifact()`, `sandbox-isolation.spec.ts` +- **Thin community `Community 203`** (2 nodes): `createBaseRun()`, `sandbox-failover.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 201`** (2 nodes): `createRun()`, `rag-audit-replay.spec.ts` +- **Thin community `Community 204`** (2 nodes): `buildUsecase()`, `save-view-from-run.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 202`** (2 nodes): `datasource()`, `query-executor-router-table-permissions.spec.ts` +- **Thin community `Community 205`** (2 nodes): `workspace-relationship-api.spec.ts`, `buildService()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 203`** (2 nodes): `sleep()`, `rag-datasource-gate-metrics.spec.ts` +- **Thin community `Community 206`** (2 nodes): `buildArtifact()`, `sandbox-isolation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 204`** (2 nodes): `buildSnapshot()`, `semantic-spine-compiler.spec.ts` +- **Thin community `Community 207`** (2 nodes): `createRun()`, `rag-audit-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 205`** (2 nodes): `workspace-modeling-setup.spec.ts`, `buildService()` +- **Thin community `Community 208`** (2 nodes): `datasource()`, `query-executor-router-table-permissions.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 206`** (2 nodes): `createRun()`, `memory-promotion.spec.ts` +- **Thin community `Community 209`** (2 nodes): `sleep()`, `rag-datasource-gate-metrics.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 207`** (2 nodes): `buildSamples()`, `glossary-selected-context-gate.spec.ts` +- **Thin community `Community 210`** (2 nodes): `buildSnapshot()`, `semantic-spine-compiler.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 208`** (2 nodes): `createSeededSqliteFixture()`, `sqlite-fixture.ts` +- **Thin community `Community 211`** (2 nodes): `workspace-modeling-setup.spec.ts`, `buildService()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 209`** (2 nodes): `readErrorMessage()`, `user-workspace-authz.spec.ts` +- **Thin community `Community 212`** (2 nodes): `createRun()`, `memory-promotion.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 210`** (2 nodes): `readErrorMessage()`, `rule-group-authz.spec.ts` +- **Thin community `Community 213`** (2 nodes): `buildSamples()`, `glossary-selected-context-gate.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 211`** (2 nodes): `workspace-datasource-authz.spec.ts`, `readErrorMessage()` +- **Thin community `Community 214`** (2 nodes): `readErrorMessage()`, `user-workspace-authz.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 212`** (2 nodes): `AppModule`, `app.module.ts` +- **Thin community `Community 215`** (2 nodes): `readErrorMessage()`, `rule-group-authz.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 213`** (2 nodes): `requestIdMiddleware()`, `request-id.middleware.ts` +- **Thin community `Community 216`** (2 nodes): `workspace-datasource-authz.spec.ts`, `readErrorMessage()` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 214`** (2 nodes): `LlmModule`, `llm.module.ts` +- **Thin community `Community 217`** (2 nodes): `AppModule`, `app.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 215`** (2 nodes): `ApplyMemoryFeedbackDto`, `apply-memory-feedback.dto.ts` +- **Thin community `Community 218`** (2 nodes): `requestIdMiddleware()`, `request-id.middleware.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 216`** (2 nodes): `AppConfigModule`, `config.module.ts` +- **Thin community `Community 219`** (2 nodes): `LlmModule`, `llm.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 217`** (2 nodes): `PlatformModule`, `platform.module.ts` +- **Thin community `Community 220`** (2 nodes): `ApplyMemoryFeedbackDto`, `apply-memory-feedback.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 218`** (2 nodes): `PlatformLlmModule`, `llm.module.ts` +- **Thin community `Community 221`** (2 nodes): `AppConfigModule`, `config.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 219`** (2 nodes): `PlatformConfigModule`, `config.module.ts` +- **Thin community `Community 222`** (2 nodes): `PlatformModule`, `platform.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 220`** (2 nodes): `PlatformObservabilityModule`, `observability.module.ts` +- **Thin community `Community 223`** (2 nodes): `PlatformLlmModule`, `llm.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 221`** (2 nodes): `PlatformDataAccessModule`, `access.module.ts` +- **Thin community `Community 224`** (2 nodes): `PlatformConfigModule`, `config.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 222`** (2 nodes): `PlatformDataQueryModule`, `query.module.ts` +- **Thin community `Community 225`** (2 nodes): `PlatformObservabilityModule`, `observability.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 223`** (2 nodes): `PlatformDataModule`, `data.module.ts` +- **Thin community `Community 226`** (2 nodes): `PlatformDataAccessModule`, `access.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 224`** (2 nodes): `PlatformDataPersistenceModule`, `persistence.module.ts` +- **Thin community `Community 227`** (2 nodes): `PlatformDataQueryModule`, `query.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 225`** (2 nodes): `ObservabilityModule`, `observability.module.ts` +- **Thin community `Community 228`** (2 nodes): `PlatformDataModule`, `data.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 226`** (2 nodes): `SystemModule`, `system.module.ts` +- **Thin community `Community 229`** (2 nodes): `PlatformDataPersistenceModule`, `persistence.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 227`** (2 nodes): `KnowledgeModule`, `knowledge.module.ts` +- **Thin community `Community 230`** (2 nodes): `ObservabilityModule`, `observability.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 228`** (2 nodes): `SemanticSpineModule`, `semantic-spine.module.ts` +- **Thin community `Community 231`** (2 nodes): `SystemModule`, `system.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 229`** (2 nodes): `SkillRegistryModule`, `skill-registry.module.ts` +- **Thin community `Community 232`** (2 nodes): `KnowledgeModule`, `knowledge.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 230`** (2 nodes): `GovernanceModule`, `governance.module.ts` +- **Thin community `Community 233`** (2 nodes): `SemanticSpineModule`, `semantic-spine.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 231`** (2 nodes): `DatasourceModule`, `datasource.module.ts` +- **Thin community `Community 234`** (2 nodes): `SkillRegistryModule`, `skill-registry.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 232`** (2 nodes): `CreateDatasourceDto`, `create-datasource.dto.ts` +- **Thin community `Community 235`** (2 nodes): `GovernanceModule`, `governance.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 233`** (2 nodes): `UploadFileDatasourceDto`, `upload-file-datasource.dto.ts` +- **Thin community `Community 236`** (2 nodes): `DatasourceModule`, `datasource.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 234`** (2 nodes): `UpdateDatasourceDto`, `update-datasource.dto.ts` +- **Thin community `Community 237`** (2 nodes): `CreateDatasourceDto`, `create-datasource.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 235`** (2 nodes): `SettingsModule`, `settings.module.ts` +- **Thin community `Community 238`** (2 nodes): `UploadFileDatasourceDto`, `upload-file-datasource.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 236`** (2 nodes): `BatchUpdateModelStatusDto`, `batch-update-model-status.dto.ts` +- **Thin community `Community 239`** (2 nodes): `UpdateDatasourceDto`, `update-datasource.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 237`** (2 nodes): `UpdateModelStatusDto`, `update-model-status.dto.ts` +- **Thin community `Community 240`** (2 nodes): `SettingsModule`, `settings.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 238`** (2 nodes): `RefreshProviderModelsDto`, `refresh-provider-models.dto.ts` +- **Thin community `Community 241`** (2 nodes): `BatchUpdateModelStatusDto`, `batch-update-model-status.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 239`** (2 nodes): `CreateProviderDto`, `create-provider.dto.ts` +- **Thin community `Community 242`** (2 nodes): `UpdateModelStatusDto`, `update-model-status.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 240`** (2 nodes): `GovernanceAccessModule`, `access.module.ts` +- **Thin community `Community 243`** (2 nodes): `RefreshProviderModelsDto`, `refresh-provider-models.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 241`** (2 nodes): `workspace.module.ts`, `WorkspaceModule` +- **Thin community `Community 244`** (2 nodes): `CreateProviderDto`, `create-provider.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 242`** (2 nodes): `RemoveWorkspaceMembersDto`, `remove-workspace-members.dto.ts` +- **Thin community `Community 245`** (2 nodes): `GovernanceAccessModule`, `access.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 243`** (2 nodes): `RenameWorkspaceDto`, `rename-workspace.dto.ts` +- **Thin community `Community 246`** (2 nodes): `workspace.module.ts`, `WorkspaceModule` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 244`** (2 nodes): `GetModelingPreviewDto`, `get-modeling-preview.dto.ts` +- **Thin community `Community 247`** (2 nodes): `RemoveWorkspaceMembersDto`, `remove-workspace-members.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 245`** (2 nodes): `UpdateWorkspaceMemberRoleDto`, `update-workspace-member-role.dto.ts` +- **Thin community `Community 248`** (2 nodes): `RenameWorkspaceDto`, `rename-workspace.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 246`** (2 nodes): `DetectModelingSchemaChangeDto`, `detect-modeling-schema-change.dto.ts` +- **Thin community `Community 249`** (2 nodes): `GetModelingPreviewDto`, `get-modeling-preview.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 247`** (2 nodes): `ListWorkspaceMembersDto`, `list-workspace-members.dto.ts` +- **Thin community `Community 250`** (2 nodes): `UpdateWorkspaceMemberRoleDto`, `update-workspace-member-role.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 248`** (2 nodes): `DeployModelingGraphDto`, `deploy-modeling-graph.dto.ts` +- **Thin community `Community 251`** (2 nodes): `DetectModelingSchemaChangeDto`, `detect-modeling-schema-change.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 249`** (2 nodes): `AddWorkspaceMemberDto`, `add-workspace-member.dto.ts` +- **Thin community `Community 252`** (2 nodes): `ListWorkspaceMembersDto`, `list-workspace-members.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 250`** (2 nodes): `CreateWorkspaceDto`, `create-workspace.dto.ts` +- **Thin community `Community 253`** (2 nodes): `DeployModelingGraphDto`, `deploy-modeling-graph.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 251`** (2 nodes): `UserModule`, `user.module.ts` +- **Thin community `Community 254`** (2 nodes): `AddWorkspaceMemberDto`, `add-workspace-member.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 252`** (2 nodes): `BatchDeleteUsersDto`, `batch-delete-users.dto.ts` +- **Thin community `Community 255`** (2 nodes): `CreateWorkspaceDto`, `create-workspace.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 253`** (2 nodes): `UpdateUserStatusDto`, `update-user-status.dto.ts` +- **Thin community `Community 256`** (2 nodes): `UserModule`, `user.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 254`** (2 nodes): `UpdateUserDto`, `update-user.dto.ts` +- **Thin community `Community 257`** (2 nodes): `BatchDeleteUsersDto`, `batch-delete-users.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 255`** (2 nodes): `UserVariableDto`, `user-variable.dto.ts` +- **Thin community `Community 258`** (2 nodes): `UpdateUserStatusDto`, `update-user-status.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 256`** (2 nodes): `CreateUserDto`, `create-user.dto.ts` +- **Thin community `Community 259`** (2 nodes): `UpdateUserDto`, `update-user.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 257`** (2 nodes): `EvalModule`, `eval.module.ts` +- **Thin community `Community 260`** (2 nodes): `UserVariableDto`, `user-variable.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 258`** (2 nodes): `RunEvaluationDto`, `run-evaluation.dto.ts` +- **Thin community `Community 261`** (2 nodes): `CreateUserDto`, `create-user.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 259`** (2 nodes): `ConversationModule`, `conversation.module.ts` +- **Thin community `Community 262`** (2 nodes): `EvalModule`, `eval.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 260`** (2 nodes): `SaveViewFromRunDto`, `save-view-from-run.dto.ts` +- **Thin community `Community 263`** (2 nodes): `RunEvaluationDto`, `run-evaluation.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 261`** (2 nodes): `AgentModule`, `agent.module.ts` +- **Thin community `Community 264`** (2 nodes): `ConversationModule`, `conversation.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 262`** (1 nodes): `semantic-spine.ts` +- **Thin community `Community 265`** (2 nodes): `SaveViewFromRunDto`, `save-view-from-run.dto.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 263`** (1 nodes): `modeling.ts` +- **Thin community `Community 266`** (2 nodes): `AgentModule`, `agent.module.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 264`** (1 nodes): `api.d.ts` +- **Thin community `Community 267`** (1 nodes): `semantic-spine.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 265`** (1 nodes): `api.ts` +- **Thin community `Community 268`** (1 nodes): `modeling.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 266`** (1 nodes): `index.ts` +- **Thin community `Community 269`** (1 nodes): `api.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 267`** (1 nodes): `index.d.ts` +- **Thin community `Community 270`** (1 nodes): `api.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 268`** (1 nodes): `api.contract.spec.ts` +- **Thin community `Community 271`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 269`** (1 nodes): `next-env.d.ts` +- **Thin community `Community 272`** (1 nodes): `index.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 270`** (1 nodes): `vitest.config.ts` +- **Thin community `Community 273`** (1 nodes): `api.contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 271`** (1 nodes): `assistant-thinking-panel.spec.tsx` +- **Thin community `Community 274`** (1 nodes): `next-env.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 272`** (1 nodes): `session-sidebar.spec.tsx` +- **Thin community `Community 275`** (1 nodes): `vitest.config.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 273`** (1 nodes): `run-visibility-mapper.spec.ts` +- **Thin community `Community 276`** (1 nodes): `assistant-thinking-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 274`** (1 nodes): `chat-rag-sync-stream-consistency.spec.tsx` +- **Thin community `Community 277`** (1 nodes): `session-sidebar.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 275`** (1 nodes): `settings-modeling-page-context.spec.tsx` +- **Thin community `Community 278`** (1 nodes): `run-visibility-mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 276`** (1 nodes): `settings-relationship-publish-gate.spec.tsx` +- **Thin community `Community 279`** (1 nodes): `chat-rag-sync-stream-consistency.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 277`** (1 nodes): `settings-users-management.spec.tsx` +- **Thin community `Community 280`** (1 nodes): `settings-modeling-page-context.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 278`** (1 nodes): `settings-modeling-model-drawer.spec.tsx` +- **Thin community `Community 281`** (1 nodes): `settings-relationship-publish-gate.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 279`** (1 nodes): `admin-api-client-modeling-contract.spec.ts` +- **Thin community `Community 282`** (1 nodes): `settings-users-management.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 280`** (1 nodes): `settings-modeling-deploy-panel.spec.tsx` +- **Thin community `Community 283`** (1 nodes): `settings-modeling-model-drawer.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 281`** (1 nodes): `settings-modeling-schema-change-panel.spec.tsx` +- **Thin community `Community 284`** (1 nodes): `admin-api-client-modeling-contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 282`** (1 nodes): `settings-modeling-sidebar-tree.spec.tsx` +- **Thin community `Community 285`** (1 nodes): `settings-modeling-deploy-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 283`** (1 nodes): `settings-modeling-deploy.spec.tsx` +- **Thin community `Community 286`** (1 nodes): `settings-modeling-schema-change-panel.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 284`** (1 nodes): `settings-workspace-management.spec.tsx` +- **Thin community `Community 287`** (1 nodes): `settings-modeling-sidebar-tree.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 285`** (1 nodes): `chat-context-envelope-input.spec.tsx` +- **Thin community `Community 288`** (1 nodes): `settings-modeling-deploy.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 286`** (1 nodes): `chat-sql-preview.integration.spec.tsx` +- **Thin community `Community 289`** (1 nodes): `settings-workspace-management.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 287`** (1 nodes): `settings-workspace-datasource-table-permissions.spec.tsx` +- **Thin community `Community 290`** (1 nodes): `chat-context-envelope-input.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 288`** (1 nodes): `settings-modeling-view-sync.spec.tsx` +- **Thin community `Community 291`** (1 nodes): `chat-sql-preview.integration.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 289`** (1 nodes): `settings-retired-governance-management.spec.tsx` +- **Thin community `Community 292`** (1 nodes): `settings-workspace-datasource-table-permissions.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 290`** (1 nodes): `sql-preview.spec.tsx` +- **Thin community `Community 293`** (1 nodes): `settings-modeling-view-sync.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 291`** (1 nodes): `settings-relationship-modeling-canvas.spec.tsx` +- **Thin community `Community 294`** (1 nodes): `settings-retired-governance-management.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 292`** (1 nodes): `settings-modeling-schema-change.spec.tsx` +- **Thin community `Community 295`** (1 nodes): `sql-preview.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 293`** (1 nodes): `app-shell.spec.tsx` +- **Thin community `Community 296`** (1 nodes): `settings-relationship-modeling-canvas.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 294`** (1 nodes): `settings-workspace-table-permission-entry.spec.tsx` +- **Thin community `Community 297`** (1 nodes): `settings-modeling-schema-change.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 295`** (1 nodes): `settings-modeling-calculated-field-editor.spec.tsx` +- **Thin community `Community 298`** (1 nodes): `app-shell.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 296`** (1 nodes): `settings-modeling-complete-parity.spec.tsx` +- **Thin community `Community 299`** (1 nodes): `settings-workspace-table-permission-entry.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 297`** (1 nodes): `chat-save-as-view.spec.tsx` +- **Thin community `Community 300`** (1 nodes): `settings-modeling-calculated-field-editor.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 298`** (1 nodes): `rag-foundation-status-card.spec.tsx` +- **Thin community `Community 301`** (1 nodes): `settings-modeling-complete-parity.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 299`** (1 nodes): `settings-modeling-metadata-editor.spec.tsx` +- **Thin community `Community 302`** (1 nodes): `chat-save-as-view.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 300`** (1 nodes): `settings-modeling-flow-node-edge.spec.tsx` +- **Thin community `Community 303`** (1 nodes): `rag-foundation-status-card.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 301`** (1 nodes): `chat-demo-flow.spec.tsx` +- **Thin community `Community 304`** (1 nodes): `settings-modeling-metadata-editor.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 302`** (1 nodes): `chat-mobile-smoke.spec.tsx` +- **Thin community `Community 305`** (1 nodes): `settings-modeling-flow-node-edge.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 303`** (1 nodes): `platform-pages-smoke.spec.tsx` +- **Thin community `Community 306`** (1 nodes): `chat-demo-flow.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 304`** (1 nodes): `page.tsx` +- **Thin community `Community 307`** (1 nodes): `chat-mobile-smoke.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 305`** (1 nodes): `sql-preview.tsx` +- **Thin community `Community 308`** (1 nodes): `platform-pages-smoke.spec.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 306`** (1 nodes): `steps.tsx` +- **Thin community `Community 309`** (1 nodes): `page.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 307`** (1 nodes): `slider.tsx` +- **Thin community `Community 310`** (1 nodes): `sql-preview.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 308`** (1 nodes): `progress.tsx` +- **Thin community `Community 311`** (1 nodes): `steps.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 309`** (1 nodes): `sonner.tsx` +- **Thin community `Community 312`** (1 nodes): `slider.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 310`** (1 nodes): `button.tsx` +- **Thin community `Community 313`** (1 nodes): `progress.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 311`** (1 nodes): `toggle.tsx` +- **Thin community `Community 314`** (1 nodes): `sonner.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 312`** (1 nodes): `workspace-datasource-table-permissions-dialog.tsx` +- **Thin community `Community 315`** (1 nodes): `button.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 313`** (1 nodes): `relationship-publish-panel.tsx` +- **Thin community `Community 316`** (1 nodes): `toggle.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 314`** (1 nodes): `modeling-flow-toolbar.tsx` +- **Thin community `Community 317`** (1 nodes): `workspace-datasource-table-permissions-dialog.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 315`** (1 nodes): `modeling-schema-change-panel.tsx` +- **Thin community `Community 318`** (1 nodes): `relationship-publish-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 316`** (1 nodes): `elk-layout.types.ts` +- **Thin community `Community 319`** (1 nodes): `modeling-flow-toolbar.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 317`** (1 nodes): `sql-inline-panel.tsx` +- **Thin community `Community 320`** (1 nodes): `modeling-schema-change-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 318`** (1 nodes): `message-list.tsx` +- **Thin community `Community 321`** (1 nodes): `elk-layout.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 319`** (1 nodes): `assistant-composer.tsx` +- **Thin community `Community 322`** (1 nodes): `sql-inline-panel.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 320`** (1 nodes): `workspace-selector-inline.tsx` +- **Thin community `Community 323`** (1 nodes): `message-list.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 321`** (1 nodes): `jest.config.ts` +- **Thin community `Community 324`** (1 nodes): `assistant-composer.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 322`** (1 nodes): `jest.setup.ts` +- **Thin community `Community 325`** (1 nodes): `workspace-selector-inline.tsx` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 323`** (1 nodes): `sandbox-policy.spec.ts` +- **Thin community `Community 326`** (1 nodes): `jest.config.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 324`** (1 nodes): `sql-table-access-guard.spec.ts` +- **Thin community `Community 327`** (1 nodes): `jest.setup.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 325`** (1 nodes): `redis-buffer.service.spec.ts` +- **Thin community `Community 328`** (1 nodes): `sandbox-policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 326`** (1 nodes): `llm-gateway.service.spec.ts` +- **Thin community `Community 329`** (1 nodes): `sql-table-access-guard.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 327`** (1 nodes): `chat.service.spec.ts` +- **Thin community `Community 330`** (1 nodes): `redis-buffer.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 328`** (1 nodes): `openai-compatible.client.spec.ts` +- **Thin community `Community 331`** (1 nodes): `llm-gateway.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 329`** (1 nodes): `langgraph-runtime.spec.ts` +- **Thin community `Community 332`** (1 nodes): `chat.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 330`** (1 nodes): `sql-prompt.builder.spec.ts` +- **Thin community `Community 333`** (1 nodes): `openai-compatible.client.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 331`** (1 nodes): `tool-execution-guard.spec.ts` +- **Thin community `Community 334`** (1 nodes): `slot-filling-context.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 332`** (1 nodes): `clarify.node.spec.ts` +- **Thin community `Community 335`** (1 nodes): `sql-prompt.builder.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 333`** (1 nodes): `planner-version-lock.service.spec.ts` +- **Thin community `Community 336`** (1 nodes): `tool-execution-guard.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 334`** (1 nodes): `planner-cache.service.spec.ts` +- **Thin community `Community 337`** (1 nodes): `clarify.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 335`** (1 nodes): `tool-events.mapper.spec.ts` +- **Thin community `Community 338`** (1 nodes): `planner-version-lock.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 336`** (1 nodes): `tool-registry.service.spec.ts` +- **Thin community `Community 339`** (1 nodes): `planner-cache.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 337`** (1 nodes): `semantic-spine-context-pack.mapper.spec.ts` +- **Thin community `Community 340`** (1 nodes): `tool-events.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 338`** (1 nodes): `llm-model-factory.spec.ts` +- **Thin community `Community 341`** (1 nodes): `tool-registry.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 339`** (1 nodes): `rag-event-consumer.spec.ts` +- **Thin community `Community 342`** (1 nodes): `semantic-spine-context-pack.mapper.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 340`** (1 nodes): `bootstrap.spec.ts` +- **Thin community `Community 343`** (1 nodes): `llm-model-factory.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 341`** (1 nodes): `skill-registry.service.spec.ts` +- **Thin community `Community 344`** (1 nodes): `rag-event-consumer.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 342`** (1 nodes): `rrf-fusion.spec.ts` +- **Thin community `Community 345`** (1 nodes): `bootstrap.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 343`** (1 nodes): `sql-output-extractor.spec.ts` +- **Thin community `Community 346`** (1 nodes): `skill-registry.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 344`** (1 nodes): `safety-check.node.spec.ts` +- **Thin community `Community 347`** (1 nodes): `rrf-fusion.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 345`** (1 nodes): `knowledge-facade-contract.spec.ts` +- **Thin community `Community 348`** (1 nodes): `sql-output-extractor.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 346`** (1 nodes): `chat-langsmith-trace.spec.ts` +- **Thin community `Community 349`** (1 nodes): `safety-check.node.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 347`** (1 nodes): `query-executors.spec.ts` +- **Thin community `Community 350`** (1 nodes): `knowledge-facade-contract.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 348`** (1 nodes): `audit-log.repository.spec.ts` +- **Thin community `Community 351`** (1 nodes): `chat-langsmith-trace.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 349`** (1 nodes): `rag-foundation-replay.spec.ts` +- **Thin community `Community 352`** (1 nodes): `query-executors.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 350`** (1 nodes): `rag-retrieval.service.spec.ts` +- **Thin community `Community 353`** (1 nodes): `audit-log.repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 351`** (1 nodes): `user-workspace-repository.spec.ts` +- **Thin community `Community 354`** (1 nodes): `rag-foundation-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 352`** (1 nodes): `rag-foundation-observability.spec.ts` +- **Thin community `Community 355`** (1 nodes): `rag-retrieval.service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 353`** (1 nodes): `rag-incremental-refresh.spec.ts` +- **Thin community `Community 356`** (1 nodes): `user-workspace-repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 354`** (1 nodes): `graph-acceleration-fallback.spec.ts` +- **Thin community `Community 357`** (1 nodes): `rag-foundation-observability.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 355`** (1 nodes): `agent-rag-main-flow.spec.ts` +- **Thin community `Community 358`** (1 nodes): `rag-incremental-refresh.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 356`** (1 nodes): `data-flow.spec.ts` +- **Thin community `Community 359`** (1 nodes): `graph-acceleration-fallback.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 357`** (1 nodes): `rule-group-schema.spec.ts` +- **Thin community `Community 360`** (1 nodes): `agent-rag-main-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 358`** (1 nodes): `workspace-modeling-deploy.spec.ts` +- **Thin community `Community 361`** (1 nodes): `data-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 359`** (1 nodes): `session-repository.spec.ts` +- **Thin community `Community 362`** (1 nodes): `rule-group-schema.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 360`** (1 nodes): `trace-lineage.spec.ts` +- **Thin community `Community 363`** (1 nodes): `workspace-modeling-deploy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 361`** (1 nodes): `rag-rerank.integration.spec.ts` +- **Thin community `Community 364`** (1 nodes): `session-repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 362`** (1 nodes): `relationship-impact-simulation.spec.ts` +- **Thin community `Community 365`** (1 nodes): `trace-lineage.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 363`** (1 nodes): `rag-multi-datasource-orchestrator.spec.ts` +- **Thin community `Community 366`** (1 nodes): `rag-rerank.integration.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 364`** (1 nodes): `rag-run-replay.spec.ts` +- **Thin community `Community 367`** (1 nodes): `relationship-impact-simulation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 365`** (1 nodes): `workspace-modeling-schema-change.spec.ts` +- **Thin community `Community 368`** (1 nodes): `rag-multi-datasource-orchestrator.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 366`** (1 nodes): `skill-registry-rag.spec.ts` +- **Thin community `Community 369`** (1 nodes): `rag-run-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 367`** (1 nodes): `semantic-registry.spec.ts` +- **Thin community `Community 370`** (1 nodes): `workspace-modeling-schema-change.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 368`** (1 nodes): `chat-run-template-trace.spec.ts` +- **Thin community `Community 371`** (1 nodes): `skill-registry-rag.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 369`** (1 nodes): `workspace-datasource-table-permissions-schema.spec.ts` +- **Thin community `Community 372`** (1 nodes): `semantic-registry.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 370`** (1 nodes): `policy-evaluator.spec.ts` +- **Thin community `Community 373`** (1 nodes): `chat-run-template-trace.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 371`** (1 nodes): `rag-cache-version-invalidation.spec.ts` +- **Thin community `Community 374`** (1 nodes): `workspace-datasource-table-permissions-schema.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 372`** (1 nodes): `rag-index-builder.spec.ts` +- **Thin community `Community 375`** (1 nodes): `policy-evaluator.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 373`** (1 nodes): `prisma-graph-baseline.spec.ts` +- **Thin community `Community 376`** (1 nodes): `rag-cache-version-invalidation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 374`** (1 nodes): `agent-rag-degrade-flow.spec.ts` +- **Thin community `Community 377`** (1 nodes): `rag-index-builder.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 375`** (1 nodes): `langsmith-coverage.spec.ts` +- **Thin community `Community 378`** (1 nodes): `prisma-graph-baseline.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 376`** (1 nodes): `session-management.spec.ts` +- **Thin community `Community 379`** (1 nodes): `agent-rag-degrade-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 377`** (1 nodes): `planner-cache-replay.spec.ts` +- **Thin community `Community 380`** (1 nodes): `langsmith-coverage.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 378`** (1 nodes): `planner-version-lock.spec.ts` +- **Thin community `Community 381`** (1 nodes): `session-management.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 379`** (1 nodes): `agent-context-pack.spec.ts` +- **Thin community `Community 382`** (1 nodes): `planner-cache-replay.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 380`** (1 nodes): `graph-acceleration-circuit-breaker.spec.ts` +- **Thin community `Community 383`** (1 nodes): `planner-version-lock.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 381`** (1 nodes): `agent-sql-prompt-template-runtime.spec.ts` +- **Thin community `Community 384`** (1 nodes): `agent-context-pack.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 382`** (1 nodes): `agent-main-flow.spec.ts` +- **Thin community `Community 385`** (1 nodes): `graph-acceleration-circuit-breaker.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 383`** (1 nodes): `datasource-repository.spec.ts` +- **Thin community `Community 386`** (1 nodes): `agent-sql-prompt-template-runtime.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 384`** (1 nodes): `rag-performance-budget.spec.ts` +- **Thin community `Community 387`** (1 nodes): `agent-main-flow.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 385`** (1 nodes): `agent-relationship-correction-loop.spec.ts` +- **Thin community `Community 388`** (1 nodes): `datasource-repository.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 386`** (1 nodes): `chat-context-envelope-accuracy.spec.ts` +- **Thin community `Community 389`** (1 nodes): `rag-performance-budget.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 387`** (1 nodes): `rag-index-activation.spec.ts` +- **Thin community `Community 390`** (1 nodes): `agent-relationship-correction-loop.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 388`** (1 nodes): `datasource-access-policy.spec.ts` +- **Thin community `Community 391`** (1 nodes): `chat-context-envelope-accuracy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 389`** (1 nodes): `eval-langsmith-trace.spec.ts` +- **Thin community `Community 392`** (1 nodes): `rag-index-activation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 390`** (1 nodes): `rag-quality.spec.ts` +- **Thin community `Community 393`** (1 nodes): `datasource-access-policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 391`** (1 nodes): `eval-report.spec.ts` +- **Thin community `Community 394`** (1 nodes): `eval-langsmith-trace.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 392`** (1 nodes): `semantic-spine-shadow-gate.spec.ts` +- **Thin community `Community 395`** (1 nodes): `rag-quality.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 393`** (1 nodes): `chat-persistence-retry.spec.ts` +- **Thin community `Community 396`** (1 nodes): `eval-report.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 394`** (1 nodes): `r3-gate-rehearsal.spec.ts` +- **Thin community `Community 397`** (1 nodes): `semantic-spine-shadow-gate.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 395`** (1 nodes): `r3-semantic-registry-migration.spec.ts` +- **Thin community `Community 398`** (1 nodes): `chat-persistence-retry.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 396`** (1 nodes): `workspace-service.spec.ts` +- **Thin community `Community 399`** (1 nodes): `r3-gate-rehearsal.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 397`** (1 nodes): `rag-document-factory.spec.ts` +- **Thin community `Community 400`** (1 nodes): `r3-semantic-registry-migration.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 398`** (1 nodes): `r6-gate-readiness.spec.ts` +- **Thin community `Community 401`** (1 nodes): `workspace-service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 399`** (1 nodes): `user-service.spec.ts` +- **Thin community `Community 402`** (1 nodes): `rag-document-factory.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 400`** (1 nodes): `provider-fallback.spec.ts` +- **Thin community `Community 403`** (1 nodes): `r6-gate-readiness.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 401`** (1 nodes): `workspace-api.spec.ts` +- **Thin community `Community 404`** (1 nodes): `user-service.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 402`** (1 nodes): `r6-release-rollback-rehearsal.e2e-spec.ts` +- **Thin community `Community 405`** (1 nodes): `provider-fallback.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 403`** (1 nodes): `datasource-visibility-api.spec.ts` +- **Thin community `Community 406`** (1 nodes): `workspace-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 404`** (1 nodes): `workspace-datasource-api.spec.ts` +- **Thin community `Community 407`** (1 nodes): `r6-release-rollback-rehearsal.e2e-spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 405`** (1 nodes): `graph-acceleration-chaos.e2e-spec.ts` +- **Thin community `Community 408`** (1 nodes): `datasource-visibility-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 406`** (1 nodes): `datasource-workflow-api.spec.ts` +- **Thin community `Community 409`** (1 nodes): `workspace-datasource-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 407`** (1 nodes): `rag-multi-datasource-isolation.spec.ts` +- **Thin community `Community 410`** (1 nodes): `graph-acceleration-chaos.e2e-spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 408`** (1 nodes): `settings-api.spec.ts` +- **Thin community `Community 411`** (1 nodes): `clarification-hybrid-balance.acceptance.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 409`** (1 nodes): `workspace-datasource-audit.spec.ts` +- **Thin community `Community 412`** (1 nodes): `datasource-workflow-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 410`** (1 nodes): `chat-api.spec.ts` +- **Thin community `Community 413`** (1 nodes): `rag-multi-datasource-isolation.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 411`** (1 nodes): `rule-group-api.spec.ts` +- **Thin community `Community 414`** (1 nodes): `settings-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 412`** (1 nodes): `chat-table-permissions-policy.spec.ts` +- **Thin community `Community 415`** (1 nodes): `workspace-datasource-audit.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 413`** (1 nodes): `rag-r6-mixed-load.spec.ts` +- **Thin community `Community 416`** (1 nodes): `chat-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 414`** (1 nodes): `rag-cache-budget-load.spec.ts` +- **Thin community `Community 417`** (1 nodes): `rule-group-api.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 415`** (1 nodes): `browser.ts` +- **Thin community `Community 418`** (1 nodes): `chat-table-permissions-policy.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 416`** (1 nodes): `client.ts` +- **Thin community `Community 419`** (1 nodes): `rag-r6-mixed-load.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 417`** (1 nodes): `models.ts` +- **Thin community `Community 420`** (1 nodes): `rag-cache-budget-load.spec.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 418`** (1 nodes): `commonInputTypes.ts` +- **Thin community `Community 421`** (1 nodes): `browser.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 419`** (1 nodes): `enums.ts` +- **Thin community `Community 422`** (1 nodes): `client.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 420`** (1 nodes): `prismaNamespace.ts` +- **Thin community `Community 423`** (1 nodes): `models.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 421`** (1 nodes): `prismaNamespaceBrowser.ts` +- **Thin community `Community 424`** (1 nodes): `commonInputTypes.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 422`** (1 nodes): `WorkspaceDatasourceTablePermission.ts` +- **Thin community `Community 425`** (1 nodes): `enums.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 423`** (1 nodes): `WorkspaceDatasourceTablePermissionSet.ts` +- **Thin community `Community 426`** (1 nodes): `prismaNamespace.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 424`** (1 nodes): `ModelCatalog.ts` +- **Thin community `Community 427`** (1 nodes): `prismaNamespaceBrowser.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 425`** (1 nodes): `RagDocument.ts` +- **Thin community `Community 428`** (1 nodes): `WorkspaceDatasourceTablePermission.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 426`** (1 nodes): `WorkspaceDatasourceBinding.ts` +- **Thin community `Community 429`** (1 nodes): `WorkspaceDatasourceTablePermissionSet.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 427`** (1 nodes): `AgentAuditLog.ts` +- **Thin community `Community 430`** (1 nodes): `ModelCatalog.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 428`** (1 nodes): `SqlRun.ts` +- **Thin community `Community 431`** (1 nodes): `RagDocument.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 429`** (1 nodes): `Workspace.ts` +- **Thin community `Community 432`** (1 nodes): `WorkspaceDatasourceBinding.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 430`** (1 nodes): `WorkspaceMember.ts` +- **Thin community `Community 433`** (1 nodes): `AgentAuditLog.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 431`** (1 nodes): `RagChunkIndexEntry.ts` +- **Thin community `Community 434`** (1 nodes): `SqlRun.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 432`** (1 nodes): `SemanticEdge.ts` +- **Thin community `Community 435`** (1 nodes): `Workspace.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 433`** (1 nodes): `GraphSnapshot.ts` +- **Thin community `Community 436`** (1 nodes): `WorkspaceMember.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 434`** (1 nodes): `RagIndexVersion.ts` +- **Thin community `Community 437`** (1 nodes): `RagChunkIndexEntry.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 435`** (1 nodes): `GlossaryTerm.ts` +- **Thin community `Community 438`** (1 nodes): `SemanticEdge.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 436`** (1 nodes): `SemanticMemory.ts` +- **Thin community `Community 439`** (1 nodes): `GraphSnapshot.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 437`** (1 nodes): `Datasource.ts` +- **Thin community `Community 440`** (1 nodes): `RagIndexVersion.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 438`** (1 nodes): `ProviderConfig.ts` +- **Thin community `Community 441`** (1 nodes): `GlossaryTerm.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 439`** (1 nodes): `WorkspaceModelingGraphRevision.ts` +- **Thin community `Community 442`** (1 nodes): `SemanticMemory.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 440`** (1 nodes): `SemanticSpineSnapshot.ts` +- **Thin community `Community 443`** (1 nodes): `Datasource.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 441`** (1 nodes): `Message.ts` +- **Thin community `Community 444`** (1 nodes): `ProviderConfig.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 442`** (1 nodes): `Session.ts` +- **Thin community `Community 445`** (1 nodes): `WorkspaceModelingGraphRevision.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 443`** (1 nodes): `RagChunk.ts` +- **Thin community `Community 446`** (1 nodes): `SemanticSpineSnapshot.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 444`** (1 nodes): `SemanticRegistryTerm.ts` +- **Thin community `Community 447`** (1 nodes): `Message.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 445`** (1 nodes): `GlossaryAnchor.ts` +- **Thin community `Community 448`** (1 nodes): `Session.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 446`** (1 nodes): `PlatformUser.ts` +- **Thin community `Community 449`** (1 nodes): `RagChunk.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 447`** (1 nodes): `SemanticRegistryVersion.ts` +- **Thin community `Community 450`** (1 nodes): `SemanticRegistryTerm.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 448`** (1 nodes): `RagRunReplay.ts` +- **Thin community `Community 451`** (1 nodes): `GlossaryAnchor.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 449`** (1 nodes): `EvaluationReport.ts` +- **Thin community `Community 452`** (1 nodes): `PlatformUser.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 450`** (1 nodes): `PromptTemplate.ts` +- **Thin community `Community 453`** (1 nodes): `SemanticRegistryVersion.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 451`** (1 nodes): `express.d.ts` +- **Thin community `Community 454`** (1 nodes): `RagRunReplay.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 452`** (1 nodes): `llm-gateway.interface.ts` +- **Thin community `Community 455`** (1 nodes): `EvaluationReport.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 453`** (1 nodes): `provider-capabilities.ts` +- **Thin community `Community 456`** (1 nodes): `PromptTemplate.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 454`** (1 nodes): `provider-adapter.interface.ts` +- **Thin community `Community 457`** (1 nodes): `express.d.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 455`** (1 nodes): `index.ts` +- **Thin community `Community 458`** (1 nodes): `llm-gateway.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 456`** (1 nodes): `index.ts` +- **Thin community `Community 459`** (1 nodes): `provider-capabilities.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 457`** (1 nodes): `modeling-graph.types.ts` +- **Thin community `Community 460`** (1 nodes): `provider-adapter.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 458`** (1 nodes): `index.ts` +- **Thin community `Community 461`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 459`** (1 nodes): `langsmith.types.ts` +- **Thin community `Community 462`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 460`** (1 nodes): `rag-retrieval.types.ts` +- **Thin community `Community 463`** (1 nodes): `modeling-graph.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 461`** (1 nodes): `knowledge-rag.contract.ts` +- **Thin community `Community 464`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 462`** (1 nodes): `knowledge-memory.contract.ts` +- **Thin community `Community 465`** (1 nodes): `langsmith.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 463`** (1 nodes): `knowledge-facade.contract.ts` +- **Thin community `Community 466`** (1 nodes): `rag-retrieval.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 464`** (1 nodes): `knowledge-semantic-registry.contract.ts` +- **Thin community `Community 467`** (1 nodes): `knowledge-rag.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 465`** (1 nodes): `knowledge-glossary.contract.ts` +- **Thin community `Community 468`** (1 nodes): `knowledge-memory.contract.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 466`** (1 nodes): `rag-retrieval.types.ts` +- **Thin community `Community 469`** (1 nodes): `knowledge-facade.contract.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 470`** (1 nodes): `knowledge-semantic-registry.contract.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 471`** (1 nodes): `knowledge-glossary.contract.ts` + Too small to be a meaningful cluster - may be noise or needs more connections extracted. +- **Thin community `Community 472`** (1 nodes): `rag-retrieval.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 467`** (1 nodes): `modeling-graph.validator.ts` +- **Thin community `Community 473`** (1 nodes): `modeling-graph.validator.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 468`** (1 nodes): `modeling-graph.types.ts` +- **Thin community `Community 474`** (1 nodes): `modeling-graph.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 469`** (1 nodes): `semantic-spine.types.ts` +- **Thin community `Community 475`** (1 nodes): `semantic-spine.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 470`** (1 nodes): `index.ts` +- **Thin community `Community 476`** (1 nodes): `index.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 471`** (1 nodes): `query-executor.interface.ts` +- **Thin community `Community 477`** (1 nodes): `query-executor.interface.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. -- **Thin community `Community 472`** (1 nodes): `agent.types.ts` +- **Thin community `Community 478`** (1 nodes): `agent.types.ts` Too small to be a meaningful cluster - may be noise or needs more connections extracted. ## Suggested Questions _Questions this graph is uniquely positioned to answer:_ -- **Why does `ok()` connect `Community 4` to `Community 0`, `Community 19`, `Community 5`?** - _High betweenness centrality (0.042) - this node is a cross-community bridge._ -- **Why does `setError()` connect `Community 1` to `Community 34`, `Community 8`, `Community 42`, `Community 23`, `Community 27`?** - _High betweenness centrality (0.022) - this node is a cross-community bridge._ -- **Why does `WorkspaceModelingService` connect `Community 5` to `Community 4`?** - _High betweenness centrality (0.016) - this node is a cross-community bridge._ +- **Why does `ok()` connect `Community 6` to `Community 0`, `Community 11`, `Community 4`?** + _High betweenness centrality (0.027) - this node is a cross-community bridge._ +- **Why does `DatasourceService` connect `Community 3` to `Community 0`, `Community 18`?** + _High betweenness centrality (0.020) - this node is a cross-community bridge._ - **Are the 85 inferred relationships involving `ok()` (e.g. with `.applyFeedback()` and `.createSession()`) actually correct?** _`ok()` has 85 INFERRED edges - model-reasoned connections that need verification._ - **What connects `ELK`, `ElkLayoutTimeoutError`, `AppModule` to the rest of the system?** @@ -3113,4 +3147,6 @@ _Questions this graph is uniquely positioned to answer:_ - **Should `Community 0` be split into smaller, more focused modules?** _Cohesion score 0.03 - nodes in this community are weakly interconnected._ - **Should `Community 1` be split into smaller, more focused modules?** - _Cohesion score 0.03 - nodes in this community are weakly interconnected._ \ No newline at end of file + _Cohesion score 0.02 - nodes in this community are weakly interconnected._ +- **Should `Community 2` be split into smaller, more focused modules?** + _Cohesion score 0.02 - nodes in this community are weakly interconnected._ \ No newline at end of file diff --git a/graphify-out/graph.html b/graphify-out/graph.html index 567494c..e1b8914 100644 --- a/graphify-out/graph.html +++ b/graphify-out/graph.html @@ -50,12 +50,12 @@

Node Info

Communities

-
3252 nodes · 6127 edges · 473 communities
+
3421 nodes · 6510 edges · 479 communities