Skip to content

chore: sync main into develop#45

Merged
fvadicamo merged 27 commits into
developfrom
chore/sync-main-into-develop
Mar 22, 2026
Merged

chore: sync main into develop#45
fvadicamo merged 27 commits into
developfrom
chore/sync-main-into-develop

Conversation

@fvadicamo

@fvadicamo fvadicamo commented Mar 21, 2026

Copy link
Copy Markdown
Contributor

What

Back-merge main into develop to resolve diverged history.

Why

PR #38 (release/v0.2.0) was merged into main but not back-merged into develop. This causes merge conflicts on PR #44 (develop -> main for v0.3.0 release).

All conflicts resolved keeping develop versions (v0.3.0 supersedes v0.2.0).

Summary by CodeRabbit

  • New Features

    • Added support for ingesting Markdown (.md) files.
  • Bug Fixes

    • Prevented LLM processing when post-retrieval safeguards remove all context.
    • Better handling for duplicate namespace creation in the admin UI.
    • Improved timeout and race-condition handling in vector storage operations.
    • API key verification failures are no longer cached.
  • Improvements

    • Request-body namespace now takes precedence for write operations.
    • Admin logout now requires authentication; conversation-store errors no longer break pipelines.
    • Updated a few top-of-file help/documentation labels.

francescoscalzo and others added 21 commits March 14, 2026 18:43
- CONTEXT.md: component table, cross-cutting concerns, protocol interfaces
  updated to reflect Phase 2 as implemented (not planned)
- requirements.md: scope section with Phase 2 deliverables, 7 EX-* exclusions
  resolved, 3 OQ-* resolved, version bumped to 1.6.0
- architecture.md: component table expanded with vektra-analytics and
  vektra-learn, protocol implementations updated (Qdrant, Unstructured,
  DualStrategyChunking, AdvancedQueryPipeline, FastEmbedBM25, LogEventEmitter),
  glossary and traceability corrected, version bumped to 2.0
- docker-compose.yml: profile comments updated (removed "Phase 2" labels)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- All 8 pyproject.toml: 0.2.0-dev -> 0.2.0
- CHANGELOG.md: add [0.2.0] section with Phase 2 features and fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
For POST/PUT/PATCH, read namespace from JSON body first. Query param is
only used as fallback when body has no namespace field. Prevents a
crafted ?namespace= from overriding the payload in multi-tenant mode.

PR #38 review comment 2935607772 (CodeRabbit Critical).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use .columns(score=Float).c.score.label() instead of
.columns(score=Float).label() which labels the TextAsFrom object
instead of the score column, breaking sparse search at runtime.

PR #38 review comment 2935607773 (CodeRabbit Critical).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On TimeoutError, Qdrant may have already persisted the points
server-side. Running compensating delete would cause data loss.
Only run cleanup on non-timeout errors where Qdrant explicitly
rejected the batch. Deterministic point IDs (uuid5) make re-ingest
idempotent.

PR #38 review comment 2935607776 (CodeRabbit Critical).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New migration 0006: ALTER CHECK constraint on source_documents to
  include 'pipeline_failure' (fixes modified-migration issue from 0001)
- vektra-app/__init__.py: 0.2.0-dev -> 0.2.0

PR #38 review comments (Gemini Medium, CodeRabbit Major).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wrap create_collection in try/except and recheck existence on failure.
Prevents startup crash when concurrent replicas both detect missing
collection simultaneously.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wrap add_turn() in try/except so DB or encryption failures do not
turn a successful query into a 500. Applies to both execute() and
_stream() paths in AdvancedQueryPipeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Do not cache False results in TTLCache to prevent cache poisoning
from a flood of invalid tokens evicting valid entries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Raise ValueError for unknown safeguard modes and let ImportError
propagate for presidio, instead of silently falling back to
passthrough. Catches misconfiguration at startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Check isinstance(data, dict) before calling .get() to prevent
AttributeError when request body is a JSON array or scalar.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When post_retrieval filters all chunks, set no_relevant_context=True
so the pipeline skips LLM instead of sending an empty-context prompt.
Applies to both Simple and Advanced pipelines.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t action

- Add _require_admin_ui dependency to logout for audit completeness
- Catch IntegrityError on namespace create, return flash error
- Skip helper segments (create/revoke) in _derive_action to avoid
  "create_create" audit entries
- Use strftime instead of isoformat in audit pagination to avoid
  unencoded +00:00 in URL

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 2 ships a Markdown extractor but batch-ingest.sh was not
updated to scan for .md files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- vektra-analytics/__init__.py: 0.2.0-dev -> 0.2.0
- vektra-shared/__init__.py: 0.2.0-dev -> 0.2.0
- Makefile: reindex target description reflects skeleton status

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
qdrant-client wraps timeouts as ResponseHandlingException (HTTP)
or AioRpcError/DEADLINE_EXCEEDED (gRPC), not Python TimeoutError.
Add _is_timeout() helper that checks exception type by name to
avoid hard imports of optional qdrant-client internals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use strftime %f to preserve sub-second precision in cursor_ts,
preventing row skipping when multiple entries share the same second.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New items from CodeRabbit review:
- DEBT-009: scope sys.modules mock in qdrant tests
- DEBT-010: harden reindex.sh error handling
- DEBT-011: route index API through ProviderRegistry (medium priority)
- DEBT-012: filter key validation at API layer
- DEBT-013: POST logout with CSRF token

Marked completed (Phase 2):
- DEBT-003: post_retrieval safeguard trust boundary
- DEBT-008: TTLCache replaces lru_cache + only-True caching

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Phase 2 advanced RAG: AdvancedQueryPipeline with hybrid search, query rewriting, reranking
- E-learning vertical: vektra-learn backend + chatbot widget, vektra-analytics with QueryTrace
- Admin dashboard: HTMX + Jinja2 UI with namespace/key/audit management
- Infrastructure: Qdrant provider, UnstructuredExtractor with OCR, DualStrategyChunking, Presidio PII safeguard
- 15 review fixes: fail-closed safeguards, TTL cache poisoning, RLS body-first, qdrant timeout detection, sparse SQL fix

Reviews: 47/47 addressed (CodeRabbit 47, Gemini 0 new)
Tests: all passing (17/17 CI checks green)
Refs: Phase 2 plans 1-11, DEBT-009 to DEBT-013 tracked
Main had commits from release/v0.2.0 (PR #38) that were not
back-merged into develop. All conflicts resolved keeping develop
versions (v0.3.0 > v0.2.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@fvadicamo fvadicamo added this to the v0.3.0 milestone Mar 21, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation component:core vektra-core component component:index vektra-index component component:admin vektra-admin component infra Infrastructure, Docker, deployment labels Mar 21, 2026
@coderabbitai

coderabbitai Bot commented Mar 21, 2026

Copy link
Copy Markdown

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (1)
  • .s2s/architecture.md is excluded by !.s2s/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 15045048-3c0a-4833-87ab-04a3d2c8b412

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds Markdown ingestion support, tightens API-key caching to verified-only, reorders namespace precedence for write requests, refines action derivation tail handling, adds post-safeguard emptiness checks and resilient conversation-store writes, and hardens Qdrant collection creation and timeout handling; plus small doc/help text updates.

Changes

Cohort / File(s) Summary
Docs & build
Makefile, docker-compose.yml, vektra-core/src/vektra_core/safeguards/__init__.py
Updated help/comments and docstrings (reindex help, compose profile descriptions, safeguard module text). No runtime logic changes.
Ingest / Batch scripts
scripts/ingest.sh, scripts/batch-ingest.sh
Added Markdown (.md) support: content-type mapping and find discovery updated; help text adjusted accordingly.
Admin auth & request routing
vektra-admin/src/vektra_admin/keys.py, vektra-admin/src/vektra_admin/middleware.py, vektra-admin/src/vektra_admin/rls.py, vektra-admin/src/vektra_admin/ui.py
verify_key() caches only truthy results; _derive_action() recognizes helper tail verbs (`create
Tests updated
vektra-admin/tests/test_rls.py, vektra-admin/tests/test_ttlcache.py
Tests adjusted to reflect body-first namespace precedence for writes and to assert failed key verifications are not cached.
Pipeline behavior
vektra-core/src/vektra_core/advanced_pipeline.py, vektra-core/src/vektra_core/pipeline.py
After post_retrieval_safeguard, set no_relevant_context when filtered becomes empty; conversation-store add_turn wrapped in try/except and warnings to avoid propagation on persistence failures.
Index providers
vektra-index/src/vektra_index/providers/pgvector.py, vektra-index/src/vektra_index/providers/qdrant.py
pgvector: adjust SQLAlchemy score expression reference. qdrant: add _is_timeout() classifier; make ensure_collection() resilient to concurrent-creation races; treat timeout-like store errors differently (log and re-raise without compensating delete) versus other errors (attempt cleanup then re-raise).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~28 minutes

Possibly related PRs

Poem

🐰
I hopped through docs and scripts today,
Found markdown crumbs along the way,
I cached the truths, let failures flee,
Checked tails and namespaces carefully,
And nudged Qdrant to retry with glee. 🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the primary purpose of this PR: a back-merge of main into develop to resolve diverged history after PR #38 was merged to main but not back-merged into develop.
Docstring Coverage ✅ Passed Docstring coverage is 89.29% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/sync-main-into-develop

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can generate a title for your PR based on the changes with custom instructions.

Set the reviews.auto_title_instructions setting to generate a title for your PR based on the changes in the PR with custom instructions.

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request synchronizes the main branch into the develop branch, incorporating Phase 2 features and resolving merge conflicts. The primary goal is to align the develop branch with the latest changes in main, ensuring the develop branch is up-to-date with the latest features and fixes.

Highlights

  • Phase 2 Implementation: This PR merges the main branch into develop, resolving diverged history and integrating Phase 2 features into the develop branch.
  • Dependency Updates: The PR includes updates to dependencies, configurations, and documentation to reflect the completion of Phase 2 features.
  • Conflict Resolution: Merge conflicts were resolved, prioritizing the develop branch versions to ensure v0.3.0 supersedes v0.2.0.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request back-merges changes from main into develop, primarily updating documentation and code to reflect the completion of "Phase 2" features. The changes include numerous documentation updates across CONTEXT.md, architecture.md, and requirements.md, along with several code improvements for robustness and security. Key code changes include making conversation history saves best-effort, improving timeout and race condition handling in the Qdrant provider, adding authentication to the logout endpoint for better auditing, and fixing a potential parameter pollution vulnerability in namespace resolution. The test suite has been updated to reflect these changes. I have one suggestion regarding the chronological ordering of version history in the architecture.md document for improved clarity.

Comment thread .s2s/architecture.md
@fvadicamo fvadicamo self-assigned this Mar 21, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
vektra-core/src/vektra_core/pipeline.py (1)

678-694: ⚠️ Potential issue | 🟡 Minor

Streaming path missing parallel safeguard recheck.

The execute() method now rechecks whether filtered became empty after post_retrieval_safeguard (lines 377-379). However, the _stream() method lacks this recheck. If the safeguard allows the request but filters all chunks via filtered_ids, the streaming path will proceed to build_prompt and call the LLM with no context chunks.

Consider adding the same recheck after line 678:

🛠️ Proposed fix
             steps.append(
                 StepTrace(
                     name="post_retrieval_safeguard",
                     duration_ms=_elapsed_ms(t0),
                     metadata={"skipped": True, "error": str(exc)},
                 )
             )

+        # Recheck: post_retrieval safeguard may have filtered all chunks
+        if not no_relevant_context and not filtered:
+            no_relevant_context = True
+            yield QueryChunk(type="sources", data=[])
+            trace = QueryTrace(
+                response_id=response_id,
+                steps=steps,
+                total_duration_ms=_elapsed_ms(t_total),
+                chunks_retrieved=[],
+                llm_model=self._llm_config.provider,
+                prompt_version=self._renderer.prompt_version,
+                created_at=datetime.now(UTC),
+            )
+            yield QueryChunk(type="trace", data=_trace_to_dict(trace))
+            yield QueryChunk(type="done", data="")
+            return
+
         # Safeguard blocked all results → early return
         if safeguard_blocked:
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-core/src/vektra_core/pipeline.py` around lines 678 - 694, The
streaming path in _stream() misses the post_retrieval_safeguard recheck done in
execute(), so if post_retrieval_safeguard filtered out all chunks (via filtered
or filtered_ids) the method will continue to build_prompt and call the LLM with
no context; update _stream() to re-evaluate whether filtered (or filtered_ids)
is empty immediately after post_retrieval_safeguard returns and, if empty, yield
the same three early-return QueryChunk sequence (sources empty, trace, done)
constructed using response_id, steps, _elapsed_ms(t_total),
llm_model/self._llm_config.provider,
prompt_version/self._renderer.prompt_version and created_at=datetime.now(UTC) to
mirror execute()’s behavior.
🧹 Nitpick comments (6)
vektra-admin/src/vektra_admin/middleware.py (1)

113-118: Consider hoisting _helper_segments to module level.

The helper-tail detection logic is correct and will prevent actions like "create_create". However, _helper_segments is defined inside the function and recreated on every call, whereas _METHOD_VERBS is a module-level constant. For consistency and marginal efficiency, consider defining it at module scope.

♻️ Proposed refactor
 _METHOD_VERBS = {
     "GET": "read",
     "POST": "create",
     "PUT": "update",
     "PATCH": "update",
     "DELETE": "delete",
 }
+
+_HELPER_TAILS = frozenset({"create", "delete", "revoke", "update"})


 def _derive_action(method: str, path: str) -> str:
     """Derive a human-readable action from HTTP method and path.

     Examples: "create_ingest", "read_health", "delete_api-keys".
     Skips helper tails like /create, /revoke to avoid "create_create".
     """
     verb = _METHOD_VERBS.get(method.upper(), method.lower())
     # Use the last meaningful path segment (strip /api/v1/ prefix and IDs)
     segments = [s for s in path.strip("/").split("/") if s and not _is_uuid_like(s)]
-    _helper_segments = {"create", "delete", "revoke", "update"}
-    if len(segments) >= 2 and segments[-1] in _helper_segments:
+    if len(segments) >= 2 and segments[-1] in _HELPER_TAILS:
         resource = segments[-2]
     else:
         resource = segments[-1] if segments else "unknown"
     return f"{verb}_{resource}"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-admin/src/vektra_admin/middleware.py` around lines 113 - 118, Hoist
the _helper_segments set out of the function to module scope (alongside
_METHOD_VERBS) to avoid recreating it on every call; update the function that
computes the resource/verb pair (the block using _helper_segments, segments,
resource, and returning f"{verb}_{resource}") to reference the module-level
_helper_segments instead of defining it locally so behavior stays the same but
with consistent placement and marginal performance improvement.
vektra-admin/tests/test_rls.py (1)

66-74: Test name implies body comparison but no body is sent.

The test name test_get_uses_query_param_over_body suggests it verifies query param takes precedence over body for GET requests, but no body is actually provided in the request. To fully validate the "over body" claim, send both a body with a namespace and a query param.

🧪 Suggested test improvement
     `@pytest.mark.asyncio`
     async def test_get_uses_query_param_over_body(self):
         """For GET requests, query param is used (body not read)."""
+        import json
+
+        body = json.dumps({"namespace": "body-ns"}).encode()
         request = _make_request(
             method="GET",
             query_params={"namespace": "query-ns"},
+            body=body,
         )
         result = await _resolve_namespace(request)
         assert result == "query-ns"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-admin/tests/test_rls.py` around lines 66 - 74, The test named
test_get_uses_query_param_over_body claims to prefer query params over body but
doesn't include a body; update the test to include a request body containing a
conflicting "namespace" value while keeping
query_params={"namespace":"query-ns"} so the test asserts
_resolve_namespace(request) returns "query-ns". Locate the test function and the
helper _make_request (and behavior in _resolve_namespace) and add a body payload
(e.g., JSON or form data with namespace set to a different value) to exercise
the "over body" precedence.
vektra-index/src/vektra_index/providers/qdrant.py (1)

118-139: Race condition handling looks correct, minor logging gap.

The logic correctly handles the race where another replica creates the collection between the existence check and the create call. The re-raise on line 137 ensures real errors aren't masked.

However, when the race condition is detected and suppressed (lines 134-138), there's no log indicating this occurred. Adding a debug/info log would improve observability for troubleshooting startup issues.

📝 Optional: Add logging when race is detected
         except Exception:
             # Race: another replica may have created it between our check and create.
             collections = await self._client.get_collections()
             if self._collection_name not in {c.name for c in collections.collections}:
                 raise
+            logger.debug(
+                "Qdrant collection %s already exists (race resolved)",
+                self._collection_name,
+            )
             return
         logger.info("Created Qdrant collection: %s", self._collection_name)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-index/src/vektra_index/providers/qdrant.py` around lines 118 - 139,
When the race condition is detected in the except block after calling
self._client.create_collection (i.e., after verifying the collection exists via
self._client.get_collections and suppressing the exception for
self._collection_name), add a log line (e.g., using logger.debug or logger.info)
indicating that collection creation was skipped because another replica created
it; update the except block around create_collection/get_collections to emit
"Skipped creation of Qdrant collection %s: already exists (race detected)" or
similar, referencing self._collection_name so startup behavior is observable
without changing the existing re-raise logic.
vektra-admin/src/vektra_admin/ui.py (1)

442-457: Narrow IntegrityError handling to duplicate-key cases

Line 444 currently treats any IntegrityError as “already exists.” That can mask unrelated integrity failures and return an incorrect user message.

Suggested refactor
-    try:
-        await session.commit()
-    except IntegrityError:
-        await session.rollback()
-        namespaces = await _load_namespaces(session)
-        return templates.TemplateResponse(
-            request=request,
-            name="partials/namespaces_table.html",
-            context={
-                "namespaces": namespaces,
-                "flash": {
-                    "type": "error",
-                    "message": f"Namespace '{name}' already exists.",
-                },
-            },
-        )
+    try:
+        await session.commit()
+    except IntegrityError as exc:
+        await session.rollback()
+        sqlstate = getattr(getattr(exc, "orig", None), "sqlstate", None) or getattr(
+            getattr(exc, "orig", None), "pgcode", None
+        )
+        if sqlstate != "23505":  # unique_violation (PostgreSQL)
+            raise
+        namespaces = await _load_namespaces(session)
+        return templates.TemplateResponse(
+            request=request,
+            name="partials/namespaces_table.html",
+            context={
+                "namespaces": namespaces,
+                "flash": {
+                    "type": "error",
+                    "message": f"Namespace '{name}' already exists.",
+                },
+            },
+        )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-admin/src/vektra_admin/ui.py` around lines 442 - 457, The except block
currently treats any IntegrityError as a duplicate-name case; change it to
"except IntegrityError as e:" then inspect the underlying DB error (e.orig) to
detect a duplicate-key/unique-violation (e.g. for Postgres check e.orig.pgcode
== "23505", for SQLite/MySQL check error message contains "UNIQUE" or "unique
constraint" / "UNIQUE constraint failed"); only when that check confirms a
duplicate violation perform await session.rollback(), call
_load_namespaces(session) and return the templates.TemplateResponse with the
"Namespace '{name}' already exists." flash, otherwise re-raise the
IntegrityError so other integrity problems are not misreported.
vektra-core/src/vektra_core/advanced_pipeline.py (2)

527-535: Reasonable resilience trade-off, but silent failures have side effects.

The best-effort pattern prevents a DB failure from turning a successful query into a 500 error—sensible for UX. However, per vektra-core/src/vektra_core/conversation.py:183-244, a failed add_turn silently skips:

  • Incrementing turn_count
  • Updating updated_at timestamp
  • Pruning old turns

Subsequent get_history calls will return incomplete context (missing the failed turn). The warning log provides visibility for operators, but consider:

  1. Adding a response-level flag indicating conversation persistence failed, so clients can retry or warn users.
  2. Implementing a retry mechanism for transient DB failures.

These are optional enhancements given the clear "best-effort" design intent.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-core/src/vektra_core/advanced_pipeline.py` around lines 527 - 535, The
current best-effort save in advanced_pipeline.py silently drops conversation
persistence when _conversation_store.add_turn fails, causing missed
turn_count/updated_at updates and pruning; update the handler in the code block
that calls self._conversation_store.add_turn to (1) attach a response-level flag
(e.g., conversation_persist_failed=True) to the query/response object returned
by process so clients can surface/retry, and (2) implement a simple retry for
transient errors (e.g., 2 retries with backoff) around
_conversation_store.add_turn before falling back to logging the warning;
reference the call site to _conversation_store.add_turn and the response object
produced by advanced_pipeline so the flag is added in the same flow.

693-700: Inconsistent condition with execute() for conversation saving.

In execute() (line 529), the conversation turn is saved regardless of whether answer is None. In _stream() (line 694), the condition is if query.conversation_id is not None and full_answer:, which skips saving when full_answer is empty/None.

This creates a behavioral difference:

  • execute(): Saves turn even when answer is None (e.g., safeguard blocked)
  • _stream(): Only saves when there's a valid answer

If intentional (streaming shouldn't persist incomplete interactions), this is fine. Otherwise, consider aligning the behavior.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@vektra-core/src/vektra_core/advanced_pipeline.py` around lines 693 - 700, The
save condition in _stream() is stricter than execute(): change the if in
_stream() from "if query.conversation_id is not None and full_answer:" to match
execute() and only check "if query.conversation_id is not None" so that
_conversation_store.add_turn(query.conversation_id, query.question, full_answer)
is called even when full_answer is None; keep the existing try/except and
logging around _conversation_store.add_turn to preserve error handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scripts/batch-ingest.sh`:
- Line 64: The ingest.sh content-type mapping doesn't include Markdown, so .md
files discovered by batch-ingest.sh will be sent as application/octet-stream and
rejected; update the case statement in scripts/ingest.sh (the content-type
mapping block around the function/logic that maps file extensions to MIME types)
to add handlers for the .md extension (e.g., map ".md" to "text/markdown" and/or
"text/x-markdown"), ensuring both lowercase and uppercase extensions are covered
or normalized before the case. Keep the existing mappings for .pdf, .docx, .pptx
intact and add the .md entry in the same case pattern so Markdown files are sent
with the correct content-type.

In `@vektra-admin/src/vektra_admin/templates/partials/audit_rows.html`:
- Line 18: The pagination cursor is being serialized with
entries[-1].created_at.strftime(...) which drops the timezone and leads to a
naive datetime when parsed by datetime.fromisoformat() in ui.py, causing
comparisons with the timezone-aware AuditLogOrm.created_at
(TIMESTAMP(timezone=True)) to fail; update the template button URL to call
entries[-1].created_at.isoformat() so the offset is preserved, and ensure ui.py
continues using datetime.fromisoformat() to parse the cursor_ts into an aware
datetime for comparisons with AuditLogOrm.created_at.

In `@vektra-index/src/vektra_index/providers/qdrant.py`:
- Around line 54-56: The gRPC deadline check in qdrant.py is incorrect and never
matches; update the comparison inside the block that checks exc_type ==
"AioRpcError" so it compares the actual status name (e.g., use str(code) ==
"DEADLINE_EXCEEDED" or code.name == "DEADLINE_EXCEEDED") instead of
"StatusCode.DEADLINE_EXCEEDED"; locate the exc_type == "AioRpcError" branch and
change the assignment/condition around code = getattr(exc, "code", lambda:
None)() to compare the status name accordingly so deadline-exceeded errors are
detected properly.

---

Outside diff comments:
In `@vektra-core/src/vektra_core/pipeline.py`:
- Around line 678-694: The streaming path in _stream() misses the
post_retrieval_safeguard recheck done in execute(), so if
post_retrieval_safeguard filtered out all chunks (via filtered or filtered_ids)
the method will continue to build_prompt and call the LLM with no context;
update _stream() to re-evaluate whether filtered (or filtered_ids) is empty
immediately after post_retrieval_safeguard returns and, if empty, yield the same
three early-return QueryChunk sequence (sources empty, trace, done) constructed
using response_id, steps, _elapsed_ms(t_total),
llm_model/self._llm_config.provider,
prompt_version/self._renderer.prompt_version and created_at=datetime.now(UTC) to
mirror execute()’s behavior.

---

Nitpick comments:
In `@vektra-admin/src/vektra_admin/middleware.py`:
- Around line 113-118: Hoist the _helper_segments set out of the function to
module scope (alongside _METHOD_VERBS) to avoid recreating it on every call;
update the function that computes the resource/verb pair (the block using
_helper_segments, segments, resource, and returning f"{verb}_{resource}") to
reference the module-level _helper_segments instead of defining it locally so
behavior stays the same but with consistent placement and marginal performance
improvement.

In `@vektra-admin/src/vektra_admin/ui.py`:
- Around line 442-457: The except block currently treats any IntegrityError as a
duplicate-name case; change it to "except IntegrityError as e:" then inspect the
underlying DB error (e.orig) to detect a duplicate-key/unique-violation (e.g.
for Postgres check e.orig.pgcode == "23505", for SQLite/MySQL check error
message contains "UNIQUE" or "unique constraint" / "UNIQUE constraint failed");
only when that check confirms a duplicate violation perform await
session.rollback(), call _load_namespaces(session) and return the
templates.TemplateResponse with the "Namespace '{name}' already exists." flash,
otherwise re-raise the IntegrityError so other integrity problems are not
misreported.

In `@vektra-admin/tests/test_rls.py`:
- Around line 66-74: The test named test_get_uses_query_param_over_body claims
to prefer query params over body but doesn't include a body; update the test to
include a request body containing a conflicting "namespace" value while keeping
query_params={"namespace":"query-ns"} so the test asserts
_resolve_namespace(request) returns "query-ns". Locate the test function and the
helper _make_request (and behavior in _resolve_namespace) and add a body payload
(e.g., JSON or form data with namespace set to a different value) to exercise
the "over body" precedence.

In `@vektra-core/src/vektra_core/advanced_pipeline.py`:
- Around line 527-535: The current best-effort save in advanced_pipeline.py
silently drops conversation persistence when _conversation_store.add_turn fails,
causing missed turn_count/updated_at updates and pruning; update the handler in
the code block that calls self._conversation_store.add_turn to (1) attach a
response-level flag (e.g., conversation_persist_failed=True) to the
query/response object returned by process so clients can surface/retry, and (2)
implement a simple retry for transient errors (e.g., 2 retries with backoff)
around _conversation_store.add_turn before falling back to logging the warning;
reference the call site to _conversation_store.add_turn and the response object
produced by advanced_pipeline so the flag is added in the same flow.
- Around line 693-700: The save condition in _stream() is stricter than
execute(): change the if in _stream() from "if query.conversation_id is not None
and full_answer:" to match execute() and only check "if query.conversation_id is
not None" so that _conversation_store.add_turn(query.conversation_id,
query.question, full_answer) is called even when full_answer is None; keep the
existing try/except and logging around _conversation_store.add_turn to preserve
error handling.

In `@vektra-index/src/vektra_index/providers/qdrant.py`:
- Around line 118-139: When the race condition is detected in the except block
after calling self._client.create_collection (i.e., after verifying the
collection exists via self._client.get_collections and suppressing the exception
for self._collection_name), add a log line (e.g., using logger.debug or
logger.info) indicating that collection creation was skipped because another
replica created it; update the except block around
create_collection/get_collections to emit "Skipped creation of Qdrant collection
%s: already exists (race detected)" or similar, referencing
self._collection_name so startup behavior is observable without changing the
existing re-raise logic.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: df5906b9-2343-417f-82c7-334cc7bb7819

📥 Commits

Reviewing files that changed from the base of the PR and between b8fe0b0 and fa03257.

⛔ Files ignored due to path filters (4)
  • .s2s/CONTEXT.md is excluded by !.s2s/**
  • .s2s/architecture.md is excluded by !.s2s/**
  • .s2s/requirements.md is excluded by !.s2s/**
  • migrations/versions/0006_add_pipeline_failure_reason.py is excluded by !**/migrations/**
📒 Files selected for processing (16)
  • Makefile
  • docker-compose.yml
  • scripts/batch-ingest.sh
  • vektra-admin/src/vektra_admin/keys.py
  • vektra-admin/src/vektra_admin/middleware.py
  • vektra-admin/src/vektra_admin/rls.py
  • vektra-admin/src/vektra_admin/templates/partials/audit_rows.html
  • vektra-admin/src/vektra_admin/ui.py
  • vektra-admin/tests/test_rls.py
  • vektra-admin/tests/test_ttlcache.py
  • vektra-core/src/vektra_core/advanced_pipeline.py
  • vektra-core/src/vektra_core/pipeline.py
  • vektra-core/src/vektra_core/safeguards/__init__.py
  • vektra-core/tests/test_safeguards.py
  • vektra-index/src/vektra_index/providers/pgvector.py
  • vektra-index/src/vektra_index/providers/qdrant.py

Comment thread scripts/batch-ingest.sh
Comment thread vektra-admin/src/vektra_admin/templates/partials/audit_rows.html Outdated
Comment thread vektra-index/src/vektra_index/providers/qdrant.py Outdated
fvadicamo and others added 4 commits March 21, 2026 18:26
The merge of main into develop auto-resolved safeguards/__init__.py
incorrectly, taking main's raise-ValueError approach over develop's
graceful-fallback approach (try/except with passthrough fallback).

Restores the develop behavior: unknown modes log a warning and fall
back to passthrough, presidio import failures fall back gracefully.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- scripts/ingest.sh: add .md content-type mapping (text/markdown)
- audit_rows.html: use isoformat() to preserve timezone in cursor
- qdrant.py: fix gRPC deadline check with "in" instead of exact match

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mirror the execute() recheck in _stream(): if post_retrieval
safeguard filtered all chunks, set no_relevant_context and return
early instead of calling LLM with empty context.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- middleware.py: hoist _helper_segments to module-level _HELPER_TAILS
- test_rls.py: add conflicting body to GET query param precedence test
- qdrant.py: add debug log on race condition resolution
- ui.py: narrow IntegrityError to unique violation (pgcode 23505)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@fvadicamo

Copy link
Copy Markdown
Contributor Author

Review comments addressed in 5298a03..ceb8957:

Fixed:

  • batch-ingest.sh/ingest.sh .md content-type (CRITICAL): 5298a03
  • audit_rows.html timezone in cursor (CRITICAL): 5298a03
  • qdrant.py gRPC deadline check (CRITICAL): 5298a03
  • pipeline.py streaming post_retrieval recheck (MINOR): 29f830e
  • middleware.py _HELPER_TAILS at module scope (NITPICK): ceb8957
  • test_rls.py add body to GET precedence test (NITPICK): ceb8957
  • qdrant.py race condition debug log (NITPICK): ceb8957
  • ui.py narrow IntegrityError to pgcode 23505 (NITPICK): ceb8957

Won't fix:

  • advanced_pipeline.py conversation retry/flag: over-engineering for a sync PR. The best-effort pattern is intentional and sufficient.
  • advanced_pipeline.py stream vs execute save condition: intentional. Streaming should not persist empty turns from incomplete interactions.

Won't fix:

  • architecture.md version ordering: generated by S2S tooling, not manually edited.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@vektra-admin/src/vektra_admin/middleware.py`:
- Around line 115-119: The audit action currently collapses helper endpoints
because when segments[-1] is in _HELPER_TAILS you drop that tail and only use
segments[-2] as the resource; change the return to preserve the helper tail by
including segments[-1] in the action string (referencing the same variables and
symbols: _HELPER_TAILS, segments, resource and the return statement that
currently does return f"{verb}_{resource}"); for example, when the last segment
is a helper tail return an action like f"{verb}_{segments[-1]}_{resource}" (or
otherwise append the tail to the verb) so distinct helper operations
(create/revoke) produce distinct audit actions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 1ee78011-334b-4c29-8bb7-028d04e4407d

📥 Commits

Reviewing files that changed from the base of the PR and between 9b503ac and ceb8957.

📒 Files selected for processing (6)
  • scripts/ingest.sh
  • vektra-admin/src/vektra_admin/middleware.py
  • vektra-admin/src/vektra_admin/ui.py
  • vektra-admin/tests/test_rls.py
  • vektra-core/src/vektra_core/pipeline.py
  • vektra-index/src/vektra_index/providers/qdrant.py
✅ Files skipped from review due to trivial changes (1)
  • scripts/ingest.sh
🚧 Files skipped from review as they are similar to previous changes (3)
  • vektra-core/src/vektra_core/pipeline.py
  • vektra-admin/tests/test_rls.py
  • vektra-index/src/vektra_index/providers/qdrant.py

Comment thread vektra-admin/src/vektra_admin/middleware.py
fvadicamo and others added 2 commits March 22, 2026 10:33
POST /api-keys/revoke now produces "revoke_api-keys" instead of
"create_api-keys". The path tail (/create, /revoke, /delete, /update)
overrides the HTTP method verb when present.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Version 1.10 (2026-03-01, OQ-018/OQ-019 resolution) was listed after
Version 2.0 (2026-03-14) due to merge order. Moved to correct position.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@fvadicamo fvadicamo merged commit d0a7c86 into develop Mar 22, 2026
19 checks passed
@fvadicamo fvadicamo deleted the chore/sync-main-into-develop branch March 22, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:admin vektra-admin component component:core vektra-core component component:index vektra-index component documentation Improvements or additions to documentation infra Infrastructure, Docker, deployment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants