diff --git a/.env.example b/.env.example index 15493c6d..67e08912 100644 --- a/.env.example +++ b/.env.example @@ -20,6 +20,11 @@ # Generic LLM API key (passed to litellm) # VEKTRA_LLM_API_KEY= +# Custom API base URL for OpenAI-compatible providers (vLLM, LM Studio, etc.) +# When set, litellm routes requests to this URL instead of the provider default. +# Example: VEKTRA_LLM_API_BASE=http://localhost:8000/v1 +# VEKTRA_LLM_API_BASE= + # Or set provider-specific keys directly: # OPENAI_API_KEY=sk-... # ANTHROPIC_API_KEY=sk-ant-... @@ -53,7 +58,7 @@ # Embedding provider and model. # VEKTRA_EMBEDDING_PROVIDER=sentence-transformers -# VEKTRA_EMBEDDING_MODEL=all-MiniLM-L6-v2 +# VEKTRA_EMBEDDING_MODEL=paraphrase-multilingual-MiniLM-L12-v2 # -------------------------------------------------------------------------- # Vector store @@ -62,26 +67,43 @@ # VEKTRA_VECTOR_STORE_PROVIDER=pgvector # VEKTRA_ACTIVE_INDEX_VERSION=1 +# Qdrant settings (only used when VEKTRA_VECTOR_STORE_PROVIDER=qdrant) +# VEKTRA_QDRANT_URL=http://localhost:6333 +# VEKTRA_QDRANT_API_KEY= +# VEKTRA_QDRANT_COLLECTION=vektra + # -------------------------------------------------------------------------- # Query pipeline # -------------------------------------------------------------------------- -# VEKTRA_QUERY_PIPELINE=simple +# VEKTRA_QUERY_PIPELINE=advanced # VEKTRA_MIN_RELEVANCE_SCORE=0.3 # VEKTRA_CHUNK_DEDUP_ENABLED=true -# VEKTRA_RESPONSE_TOKEN_RESERVE=1024 +# VEKTRA_RESPONSE_TOKEN_RESERVE=2048 # VEKTRA_CONTEXT_CHUNK_RATIO=0.6 # VEKTRA_PROMPT_TEMPLATES_DIR= +# Query rewriting (AdvancedQueryPipeline) +# VEKTRA_QUERY_REWRITE_ENABLED=true +# VEKTRA_QUERY_REWRITE_MODEL= + +# Reranking +# VEKTRA_RERANK_ENABLED=true +# VEKTRA_RERANK_PROVIDER=flashrank +# VEKTRA_RERANK_MODEL= +# VEKTRA_RERANK_TOP_K=5 + # -------------------------------------------------------------------------- # Ingestion # -------------------------------------------------------------------------- # VEKTRA_CHUNKING_STRATEGY=fixed -# VEKTRA_CHUNK_SIZE=1000 -# VEKTRA_CHUNK_OVERLAP=200 +# VEKTRA_CHUNK_SIZE=500 +# VEKTRA_CHUNK_OVERLAP=100 # VEKTRA_MAX_FILE_SIZE_MB=50 # VEKTRA_DOCUMENT_EXTRACTOR=pdfplumber +# VEKTRA_TABLE_SPLIT=false +# VEKTRA_PARENT_CHILD_LEVELS=0 # -------------------------------------------------------------------------- # LLM fallback @@ -89,7 +111,7 @@ # Fallback model when primary times out. # VEKTRA_LLM_FALLBACK_MODEL= -# VEKTRA_LLM_FALLBACK_TIMEOUT_MS=30000 +# VEKTRA_LLM_FALLBACK_TIMEOUT_MS=60000 # VEKTRA_LLM_CONTEXT_ONLY_ENABLED=true # -------------------------------------------------------------------------- @@ -99,6 +121,11 @@ # VEKTRA_PORT=8000 # VEKTRA_STARTUP_LLM_CHECK=true +# Optional service ports (host-side). Override to avoid port conflicts. +# OLLAMA_PORT=11434 +# QDRANT_PORT=6333 +# TEI_PORT=8080 + # -------------------------------------------------------------------------- # Observability # -------------------------------------------------------------------------- @@ -108,13 +135,35 @@ # VEKTRA_EVAL_MODE=false # -------------------------------------------------------------------------- -# Phase 2 (not active in Phase 1) +# Security # -------------------------------------------------------------------------- # VEKTRA_SAFEGUARD_MODE=passthrough # VEKTRA_MULTI_TENANT=false # VEKTRA_CONVERSATION_KEY= +# VEKTRA_PII_CHUNK_THRESHOLD=3 # VEKTRA_RETENTION_DAYS= # VEKTRA_ANALYTICS_RETENTION_DAYS= # VEKTRA_SPARSE_EMBEDDING_PROVIDER= # VEKTRA_SPARSE_EMBEDDING_MODEL= + +# -------------------------------------------------------------------------- +# Webhooks +# -------------------------------------------------------------------------- + +# Webhook endpoint for event delivery. Disabled when unset. +# VEKTRA_WEBHOOK_URL= +# VEKTRA_WEBHOOK_SECRET= +# VEKTRA_WEBHOOK_TIMEOUT=5.0 + +# -------------------------------------------------------------------------- +# Learn (e-learning vertical) +# -------------------------------------------------------------------------- + +# JWT signing secret for dashboard tokens. Required when vektra-learn is active. +# VEKTRA_LEARN_JWT_SECRET= + +# Require Vektra enrollment record for learn queries. +# Set to false when an external LMS (e.g. Moodle) manages enrollment. +# When false, namespace is derived from JWT (namespace claim or course_id fallback). +# VEKTRA_LEARN_REQUIRE_ENROLLMENT=true diff --git a/.gitignore b/.gitignore index aa952f14..6fdd6674 100644 --- a/.gitignore +++ b/.gitignore @@ -24,6 +24,7 @@ logs/ .env.*.local .claude/settings.local.json CLAUDE.local.md +docker-compose.ports.yml # Build outputs dist/ diff --git a/.s2s/BACKLOG.md b/.s2s/BACKLOG.md index 5410475d..e38b0ab2 100644 --- a/.s2s/BACKLOG.md +++ b/.s2s/BACKLOG.md @@ -1,6 +1,6 @@ # Vektra Backlog -**Updated**: 2026-03-14 +**Updated**: 2026-02-28 **Format**: Single markdown file for tracking work items --- @@ -54,16 +54,21 @@ --- -### DEBT-003: ~~`post_retrieval` safeguard trust boundary not called~~ +### DEBT-003: `post_retrieval` safeguard trust boundary not called -**Status**: completed | **Priority**: low | **Created**: 2026-02-19 | **Completed**: 2026-03-01 -**Resolved in**: Phase 2 Wave 2 (core-pipeline-v2) — post_retrieval called in both execute() and _stream(), PresidioPIISafeguard implements chunk filtering +**Status**: planned | **Priority**: low | **Created**: 2026-02-19 +**Blocked by**: Phase 2 (PassthroughSafeguard covers Phase 1) +**PR #2 review**: Confirmed as deferred. Adding the boundary requires calling `post_retrieval` in both `execute()` and `_stream()` after retrieval filter, plus implementing chunk filtering via `SafeguardResult.filtered_ids`. Acceptable for Phase 1 with PassthroughSafeguard. Comment 2833984647. + +**Context**: ARCH-049 defines 3 SafeguardHook trust boundary points: `pre_query` (called in `api.py`), `post_retrieval` (not called anywhere), `pre_response` (called in `pipeline.execute()` and `pipeline._stream()`). The middle boundary - triggered after chunks are retrieved and before the prompt is built - is entirely absent. This means chunk-level PII filtering or namespace isolation checks are not enforced. + +**Traceability**: ARCH-049 (safeguard content modification), REQ-044, vektra_shared/protocols.py SafeguardHook **Acceptance Criteria**: -- [x] `pipeline.execute()` calls `safeguard.post_retrieval(chunk_ids, sg_ctx)` after retrieval filter, before build_prompt -- [x] `pipeline._stream()` calls the same boundary -- [x] SafeguardHook Protocol documents the expected signature for `post_retrieval` -- [x] PassthroughSafeguard implements `post_retrieval` as a no-op +- [ ] `pipeline.execute()` calls `safeguard.post_retrieval(chunk_ids, sg_ctx)` after retrieval filter, before build_prompt +- [ ] `pipeline._stream()` calls the same boundary +- [ ] SafeguardHook Protocol documents the expected signature for `post_retrieval` +- [ ] PassthroughSafeguard implements `post_retrieval` as a no-op --- @@ -132,17 +137,22 @@ --- -### DEBT-008: ~~LRU cache stores plaintext API keys in memory~~ +### DEBT-008: LRU cache stores plaintext API keys in memory + +**Status**: planned | **Priority**: low | **Created**: 2026-02-28 +**Blocked by**: Phase 2 +**Origin**: PR #2 review, CodeRabbit comment 2867565397 -**Status**: completed | **Priority**: low | **Created**: 2026-02-28 | **Completed**: 2026-03-01 -**Resolved in**: Phase 2 Wave 1 (admin-enforcement) — replaced lru_cache with cachetools.TTLCache (300s TTL, maxsize 512). PR #38 further improved: only cache True (successful) verifications to prevent cache poisoning. +**Context**: `verify_key()` in `vektra_admin/keys.py` uses `functools.lru_cache(maxsize=512)` keyed by `(key_hash, plaintext)`. The plaintext API key remains in the Python heap for the entire process lifetime (or until LRU eviction). `functools.lru_cache` is size-bounded only, not TTL-bounded. While the plaintext is already in memory during each request (Authorization header), the cache extends exposure from request-scoped to process-scoped. Replace with `cachetools.TTLCache` (e.g. TTL=300s, maxsize=512) to limit temporal exposure. + +**Traceability**: REQ-023, ARCH-023, PR #2 comment 2867565397 **Acceptance Criteria**: -- [x] Replace `functools.lru_cache` with `cachetools.TTLCache` in `_cached_verify` -- [x] TTL configured via constant (default 300s) -- [x] Cache key uses `(key_hash, plaintext)` as before -- [x] Unit test verifies cache expiration after TTL -- [x] `cachetools` added to vektra-admin dependencies +- [ ] Replace `functools.lru_cache` with `cachetools.TTLCache` in `_cached_verify` +- [ ] TTL configured via constant (default 300s) +- [ ] Cache key uses `(key_hash, plaintext)` as before (or hash-based fingerprint) +- [ ] Unit test verifies cache expiration after TTL +- [ ] `cachetools` added to vektra-admin dependencies --- @@ -452,95 +462,528 @@ Plan generation follows a three-phase approach (lesson learned from Phase 1): --- -### DEBT-009: Scope sys.modules mock in test_qdrant_provider.py +### FEAT-005: LLM fallback for no-context queries (greetings, courtesies, off-topic) -**Status**: planned | **Priority**: low | **Created**: 2026-03-14 -**Origin**: PR #38 review, CodeRabbit Major (test_qdrant_provider.py:25-32) +**Status**: draft | **Priority**: medium | **Created**: 2026-03-17 +**Origin**: Moodle integration testing — greetings like "ciao", "buongiorno" trigger `no_relevant_context` and produce empty/unhelpful responses -**Context**: `test_qdrant_provider.py` injects a mock `qdrant_client` module into `sys.modules` at module level. This mutation persists for the entire pytest session. If another test file imports the real `qdrant_client`, it will get the mock instead, causing silent failures. Today no other test does this, but adding one would trigger the bug without any obvious cause. Fix: use `monkeypatch.setitem(sys.modules, ...)` in a fixture. +**Context**: Both `SimpleQueryPipeline` and `AdvancedQueryPipeline` short-circuit when no chunks pass the relevance threshold (`min_relevance_score`): they return `answer: null` + `no_relevant_context: true` without ever calling the LLM. This is correct for retrieval quality (ARCH-056, REQ-066) — the system should not hallucinate answers from non-relevant chunks. -**Traceability**: vektra-index/tests/test_qdrant_provider.py +However, for the learn chatbot widget (and any conversational interface), this creates a poor UX for: +- **Greetings and courtesies**: "ciao", "buongiorno", "come stai?" — the assistant should acknowledge and redirect to course materials +- **Meta questions**: "chi sei?", "cosa puoi fare?" — the assistant should explain its role +- **Off-topic but harmless**: "che ore sono?" — the assistant should politely decline -**Acceptance Criteria**: -- [ ] sys.modules patching scoped to test via monkeypatch or patch.dict -- [ ] Tests still pass with the scoped mock +The existing **query rewriting** (ADR-0023, `AdvancedQueryPipeline`) only activates when conversation history exists and resolves pronoun references — it does not help with greetings. + +The existing **SafeguardHook** (`pre_query`, `post_retrieval`, `pre_response`) could catch prompt injection and abusive content at the `pre_query` stage, but with `VEKTRA_SAFEGUARD_MODE=passthrough` they are no-ops. Even with safeguards active, they filter/block — they don't generate friendly responses. + +**Proposed approach**: When `no_relevant_context` is detected, instead of short-circuiting, invoke the LLM with a modified system prompt that instructs it to respond to greetings, explain its role, and redirect to course-related questions — without fabricating information from missing context. This keeps the retrieval quality gate intact while allowing the LLM to handle conversational basics. + +**Alternatives considered**: +- **Intent classification pre-retrieval**: separate LLM call to classify intent before retrieval. More precise but adds latency and cost for every query. +- **Client-side pattern matching**: widget detects greetings and responds locally. Fragile, language-dependent, doesn't help other clients. + +**Traceability**: ARCH-056, REQ-066, ADR-0021, ADR-0025 + +**Acceptance Criteria** (tentative): +- [ ] Greetings/courtesies receive a friendly response acknowledging the user and explaining the assistant's role +- [ ] Off-topic queries receive a polite redirect to course-related questions +- [ ] The LLM is NOT given fabricated context — it knows no relevant chunks were found +- [ ] Retrieval quality gate unchanged — `no_relevant_context` flag still set in QueryTrace +- [ ] Safeguard hooks still apply (pre_query can block before LLM call) +- [ ] Works with both SimpleQueryPipeline and AdvancedQueryPipeline +- [ ] System prompt for no-context fallback is configurable via Jinja2 template --- -### DEBT-010: Harden reindex.sh error handling +### FEAT-007: Markdown rendering in widget chat messages -**Status**: planned | **Priority**: low | **Created**: 2026-03-14 -**Origin**: PR #38 review, CodeRabbit Major (reindex.sh:66-85, 93-97) +**Status**: draft | **Priority**: medium | **Created**: 2026-03-20 +**Origin**: Moodle integration testing (2026-03-20) -**Context**: `scripts/reindex.sh` has two robustness gaps: (1) The trigger request uses `curl -f` which collapses 4xx/5xx into a generic error, masking validation failures. (2) The polling loop retries on all failures including permanent errors (401, 404), and masks malformed JSON with `?` fallbacks. The reindex API is a skeleton today (tracks progress without executing re-embedding), but these gaps will matter when the API becomes functional. +**Context**: The learn chatbot widget (`vektra-chat.js`) renders all messages as plain text via `textContent`. LLM responses typically contain Markdown formatting (bold, italic, lists, code blocks, headings) which is displayed as raw syntax. This makes responses harder to read, especially for structured answers with bullet points or code examples. -**Traceability**: scripts/reindex.sh, ARCH-045 +The widget is deliberately vanilla JS with zero dependencies (ADR-0025). Adding Markdown rendering requires either a lightweight library (e.g., `marked`, ~7KB minified) or a minimal custom parser for the most common patterns. -**Acceptance Criteria**: -- [ ] Trigger: use `curl -sS` (not `-f`), check HTTP status explicitly, fail loud on 4xx -- [ ] Polling: retry only on transient errors (5xx), fail immediately on 4xx -- [ ] JSON parsing: validate required fields (job_id, status), fail loud on missing +**Scope**: only assistant messages need rendering. User messages stay as plain text. Sources section is already structured HTML. + +**Security**: rendered HTML must be sanitized to prevent XSS. The LLM output is not user-controlled but defense in depth applies. Use a sanitizer or restrict to a safe subset of HTML tags. + +**Traceability**: ADR-0025, ARCH-063 + +**Acceptance Criteria** (tentative): +- [ ] Assistant messages render bold, italic, lists (ordered/unordered), code inline/blocks, and headings +- [ ] User messages remain plain text +- [ ] Streaming tokens render progressively (Markdown applied incrementally or on completion) +- [ ] Output is sanitized against XSS +- [ ] Widget bundle size increase is documented and reasonable (<10KB) --- -### DEBT-011: Route vektra-index REST API through ProviderRegistry +### FEAT-008: Per-namespace prompt template customization -**Status**: planned | **Priority**: medium | **Created**: 2026-03-14 -**Origin**: PR #38 review, CodeRabbit Major (index/api.py:147-167, 244-259) +**Status**: draft | **Priority**: medium | **Created**: 2026-03-20 +**Origin**: Moodle integration testing - generic system prompt not suitable for diverse course contexts -**Context**: The REST routes in `vektra-index/src/vektra_index/api.py` instantiate `PgvectorProvider` directly instead of resolving via ProviderRegistry. When `VEKTRA_VECTOR_STORE_PROVIDER=qdrant`, the query pipeline (vektra-core) correctly uses Qdrant via Registry, but the index REST endpoints (`/search`, `/stats`, `/documents`, `/health`) still hit pgvector. This causes a data/behavior mismatch if an operator calls these endpoints directly. Also, the sparse embedding lookup uses `app.state.sparse_embedding_provider` instead of the Registry. +**Context**: The current prompt template system (ARCH-054, ADR-0020) supports only global customization via `VEKTRA_PROMPT_TEMPLATES_DIR`. All namespaces share the same system.j2, context.j2, and conversation.j2. This is limiting for the e-learning vertical where each course (namespace) may have different needs: -**Traceability**: ARCH-039 (ProviderRegistry), vektra-index/src/vektra_index/api.py +- A professor wants a specific greeting or tone ("you are the teaching assistant for Advanced Calculus, taught by Prof. Rossi") +- A course requires answers in a specific language regardless of the student's UI language +- Some courses want the assistant to refuse certain question types (e.g., "do not solve exercises directly, guide the student step by step") +- A course may need domain-specific instructions ("when discussing legal cases, always cite the article number") +- Info about the course, the professor, office hours, exam dates, etc. -**Acceptance Criteria**: -- [ ] Index API routes resolve vector store provider via ProviderRegistry -- [ ] Sparse embedding resolved via `registry.get("sparse_embedding", "default")` -- [ ] `/search`, `/stats`, `/health` reflect the configured provider (not always pgvector) -- [ ] Tests cover both pgvector and qdrant provider resolution paths +**Proposed approach**: Two complementary sources of template variables, plus per-namespace template override. + +### Template variable sources + +**A. Dynamic metadata via JWT claims (preferred for LMS integrations)**: +The upstream system (Moodle, or any LMS/application) passes metadata in the token generation request. Vektra includes them as JWT claims. At query time, the pipeline extracts the claims and injects them as Jinja2 variables. This requires no storage in Vektra - data comes from the source system at each page load and is always up to date. + +Flow: `LMS page load -> read course/instructor info -> POST /learn/tokens { ..., metadata: { course_name, instructor, ... } } -> JWT claims -> query -> pipeline extracts claims -> template variables` + +This is LMS-agnostic: any system that calls the token endpoint can pass arbitrary key-value metadata. Moodle, Canvas, custom apps - all use the same mechanism. + +**B. Static metadata in Vektra database (fallback for non-LMS use cases)**: +For namespaces not backed by an LMS (e.g., standalone knowledge bases, internal tools), metadata is stored in the namespace entity (ARCH-047) and managed via admin API. The pipeline reads namespace metadata at query time. + +**C. Merge strategy**: dynamic JWT claims take precedence over static DB metadata. Both are merged and passed to the Jinja2 context. A template can use variables from either source transparently. + +### Per-namespace template override + +1. **Resolution order**: namespace-specific template > global override (`VEKTRA_PROMPT_TEMPLATES_DIR`) > built-in default. If a namespace defines only `system.j2`, the global `context.j2` and `conversation.j2` still apply. +2. **Storage**: namespace templates stored in the database (via admin API) or as files in a convention-based directory structure (`{PROMPT_TEMPLATES_DIR}/{namespace}/system.j2`). +3. **Management**: API endpoints for CRUD on namespace prompt templates (admin scope). In the learn vertical, the Moodle plugin or admin UI could expose this to course coordinators. + +### Example + +A Moodle plugin sends metadata at token generation: +```json +{ "student_id": "jdoe", "course_id": "calc-201", "metadata": { + "course_name": "Advanced Calculus", + "instructor": "Prof. Rossi", + "custom_instructions": "Guide students step by step, do not solve exercises directly." +}} +``` + +The namespace `calc-201` has a custom `system.j2`: +```jinja2 +You are the teaching assistant for {{ course_name }}, taught by {{ instructor }}. +{{ custom_instructions }} +Answer based on the provided context. Do not invent information. +``` + +If no custom template exists, the global system.j2 still has access to the same variables (they just won't be referenced unless the template uses them). + +**Not in scope**: per-student templates (per-namespace only). Runtime template editing by students (admin/instructor only). + +**Traceability**: ARCH-054, ADR-0020, ARCH-047 (namespace as first-class entity) + +**Acceptance Criteria** (tentative): +- [ ] Token generation endpoint accepts optional `metadata` dict (arbitrary key-value pairs) +- [ ] Metadata included as JWT claims, extracted at query time +- [ ] Namespace can store static metadata in database (admin API) +- [ ] JWT claims override DB metadata on key collision +- [ ] All metadata available as Jinja2 template variables +- [ ] Namespace can define custom system.j2 that overrides the global one +- [ ] Missing namespace templates fall back to global, then built-in +- [ ] API endpoints for managing namespace prompt templates (admin scope) +- [ ] Existing global `VEKTRA_PROMPT_TEMPLATES_DIR` continues to work unchanged --- -### DEBT-012: Add filter key validation at SearchRequest API layer +### FEAT-006: Widget error feedback when Vektra API is unreachable -**Status**: planned | **Priority**: low | **Created**: 2026-03-14 -**Origin**: PR #38 review, CodeRabbit Minor (pgvector.py:321-338) +**Status**: draft | **Priority**: medium | **Created**: 2026-03-20 +**Origin**: Moodle integration testing on remote machine (2026-03-20) -**Context**: `SearchRequest.filters` accepts arbitrary `dict[str, Any]` from client requests and passes keys directly to JSONB path construction in `_apply_filters`. SQLAlchemy's JSONB operator prevents SQL injection, but validating keys (e.g., alphanumeric + underscore pattern) at the API layer provides defense-in-depth and clearer error messages for malformed requests. +**Context**: When the chatbot widget JS (`vektra-chat.js`) cannot reach the Vektra API (missing SSH tunnel, CORS misconfiguration, Vektra container down), the floating chat button silently fails to appear. No error is shown to the user or the admin. The Moodle block still displays "AI Assistant is active" because the server-side token generation succeeded (PHP runs inside Docker, reaches Vektra on the internal network), but the browser-side widget cannot load or connect. -**Traceability**: ARCH-044, vektra-index/src/vektra_index/providers/pgvector.py +This makes troubleshooting difficult: the admin sees "active" but students see nothing. The root cause (network/CORS/port) is invisible without opening browser dev tools. -**Acceptance Criteria**: -- [ ] Filter keys validated against allowlist pattern (e.g., `^[A-Za-z0-9_]+$`) -- [ ] Invalid keys rejected with 400 error and clear message -- [ ] Validation applied at API/parsing layer, not inside provider +**Proposed approach**: The widget JS should detect load/connection failures and surface them: +- If the script loads but cannot reach the API (fetch error, CORS block): show a subtle error state in the chat button or a dismissible banner +- If the script itself fails to load (network error): the Moodle plugin could add a `