feat: optional multi-query support (parallel multiquery) via Helm + langgraph subgraph

[robodev-r2d2]

Summary

Add an optional multi-query capability that lets the service accept and execute multiple queries in parallel, retrieving and composing results using a composite retriever implemented in a langgraph subgraph. The feature must be toggleable and configurable via the Helm chart (default: disabled), align with the repo’s dependency-injection pattern, and allow administrators to specify how many queries may be served concurrently.

Motivation

• Improve throughput and user experience for workflows that benefit from multiple simultaneous query formulations (e.g., multiple paraphrases, different retrieval heuristics, or multi-prompt strategies).
• Offload per-query orchestration into an explicit langgraph subgraph to keep application code simple and to leverage graph-based composition and observability.
• Provide operator control via Helm so multi-query can be enabled only where cluster resources and DB load can support it.

Goals / Requirements

• Optional feature: multi-query must be disabled by default and enabled via a Helm values flag.
• Langgraph subgraph: place the multiquery orchestration (fan-out, per-query retrieval, merging) inside a dedicated langgraph subgraph (e.g., "multiquery" or "composite-retriever") rather than scattering logic in the API service.
• Configurable number of queries: allow operators to decide the maximum number of queries to run per request (e.g., 1..N).
• Parallel processing: queries are executed in parallel (async / concurrent pool) in the subgraph and API-level endpoints should remain responsive with per-request timeouts.
• Composite retriever: within the subgraph provide a composite retriever node that merges results from each sub-retriever, handles deduplication and ranking, and returns a single candidate set to the downstream pipeline.
• DI compliance: register retriever, multi-query service, and configuration via the project’s DI container (existing admin dependency container referenced in the repo), ensuring unit-testable abstractions and easy overrides.
• Backwards compatible API: existing single-query behavior must be unchanged when the feature is disabled. If the API expands (e.g., accept multiple queries in one call), document the new payload and provide fallbacks.

Design (high level)

1. Langgraph subgraph (recommended name: "multiquery-subgraph")

   • Input port: list/array of queries + per-query parameters (optional).
   • Fan-out node: instantiates N query nodes (N = configured number, bounded per-request).
   • Retriever node(s): each query node invokes the configured retriever (vector DB calls) — can reuse existing retriever implementations via DI.
   • Merge node: composite retriever merges results, deduplicates, and sorts by relevance/scoring; optionally re-rank with LLM.
   • Output: unified candidate list + metadata (per-query stats, latencies).

2. API & Request model

   • New endpoint or optional field on existing query endpoint:
     • Option A: existing endpoint accepts "queries": string[] (when missing, fallback to single-query behavior).
     • Option B: new endpoint POST /v1/query/multi with payload { queries: string[], options?: {...} }.
   • Validate queries count against configured maxQueries.
   • When multiquery disabled in Helm, API rejects array payloads (or internally runs only the first query).

3. Helm / configuration (values.yaml additions)

   • Add a top-level block, e.g.:

     multiQuery:
       enabled: false
       maxQueries: 3            # maximum queries allowed per request
       perRequestTimeoutSec: 10 # timeout applied to the multiquery orchestration
       parallelism: 3           # how many queries to run in parallel (<= maxQueries)
       langgraph:
         subgraphName: "multiquery-subgraph"
         image: ""             # optional override for a dedicated subgraph image
       retriever:
         topK_per_query: 10
     

   • Ensure values are documented in chart README and defaults keep the feature off.

4. DI / implementation strategy

   • Define interfaces (abstractions) for:
     • IMultiQueryService: orchestrates queries and invokes the langgraph subgraph.
     • ICompositeRetriever: merging/dedup behavior.
   • Update existing dependency container (admin dependency container referenced in repository PRs/issues) to optionally register MultiQueryService and CompositeRetriever when multiQuery.enabled is true.
   • Keep concrete implementations behind interfaces to respect existing DI patterns and allow for test doubles in unit tests.

5. Concurrency & safety concerns

   • Use async/await + asyncio.gather or a bounded semaphore to limit per-request concurrency.
   • Use per-query timeouts and global orchestration timeout to avoid resource exhaustion.
   • Rate-limit/queue if necessary at ingress or via Helm-configured concurrency throttles.
   • Ensure vector DB (e.g., Qdrant) client concurrency is safe — reuse pooled connections rather than creating many clients.

Implementation steps (suggested)

1. Add Helm chart values (values.yaml + README) with defaults to disable multi-query.
2. Create the langgraph subgraph definition (subgraph files, nodes) for multiquery orchestration and composite retriever logic.
3. Add server-side wiring:
   • New service implementation that calls the langgraph subgraph or, if lightweight, orchestrates call to retrievers and uses CompositeRetriever implementation.
   • Register the service and retriever in the project DI container only when enabled.
4. Add API changes:
   • Accept list of queries and validate count.
   • Preserve existing single-query semantics if "queries" not provided or feature disabled.
5. Implement unit tests:
   • MultiQueryService unit tests with mocks for retriever and langgraph client.
   • Composite retriever merging/dedup tests.
6. Add integration tests:
   • Local Docker Compose or k3d+tilt test that runs langgraph subgraph, vector DB, and API to validate end-to-end flow.
7. Document the feature in README and Helm chart values document.

Acceptance criteria

• Helm values contain a clear toggle and configuration for multi-query (enabled: false by default).
• Langgraph subgraph implementing parallel query orchestration and composite retriever exists in repo (or clearly documented if external).
• API accepts multiple queries when feature is enabled (or provides new endpoint), with validation and a configured maximum.
• The multi-query behavior processes queries in parallel and returns a single merged ranked list.
• Implementation registers services via the repository’s DI container and keeps retriever abstractions testable.
• Unit and integration tests exist demonstrating basic parallel multi-query execution and merging.
• Documentation and example values.yaml are included.

Edge cases and notes

• Decide whether each "query" should use identical retriever config or allow per-query retriever options.
• Consider dedupe logic for identical/doc-overlap results and how to surface per-query provenance in the response (optional).
• Add observability: per-query latencies, counts, and errors.

Additional notes for implementation

• For the rephrasing step, leverage the existing rephrasing chain with its prompt template. For each incoming request, use both the original question and its rephrased form(s) as queries to the multi-query system. This ensures the retrieval pipeline benefits from rephrasings while also including the original phrasing.

Suggested labels

• enhancement
• LlmOnIt

[robodev-r2d2]

@codex according to the .github/codex-instructions.md, please implement and provide a PR for this issue.

[chatgpt-codex-connector]

Summary

• Introduced configurable multi-query retrieval with dedicated settings, a langgraph subgraph, orchestration service, and dependency-injection wiring into the chat graph to run and merge concurrent queries. libs/rag-core-api/src/rag_core_api/impl/settings/multi_query_settings.pyL1-L50 libs/rag-core-api/src/rag_core_api/impl/graph/multiquery_subgraph.pyL1-L133 libs/rag-core-api/src/rag_core_api/multi_query/service.pyL1-L137 libs/rag-core-api/src/rag_core_api/dependency_container.pyL35-L255 libs/rag-core-api/src/rag_core_api/impl/graph/chat_graph.pyL18-L309
• Extended chat requests and the OpenAPI schema to accept optional query arrays while maintaining the existing single-query behaviour. libs/rag-core-api/src/rag_core_api/models/chat_request.pyL32-L97 libs/rag-core-api/openapi.yamlL174-L192
• Added Helm values, configmaps, deployment wiring, and documentation so operators can toggle and configure the multi-query feature at deployment time. infrastructure/rag/values.yamlL220-L231 infrastructure/rag/templates/_backend_helpers.tplL40-L110 infrastructure/rag/templates/backend/configmap.yamlL1-L163 infrastructure/rag/templates/backend/deployment.yamlL80-L149 infrastructure/README.mdL300-L362
• Updated the composite retriever to support cross-query merging and introduced unit tests covering both the orchestrator and deduplication logic. libs/rag-core-api/src/rag_core_api/impl/retriever/composite_retriever.pyL18-L130 libs/rag-core-api/src/rag_core_api/retriever/composite_retriever_interface.pyL1-L22 libs/rag-core-api/tests/multi_query_service_test.pyL1-L106 libs/rag-core-api/tests/composite_retriever_test.pyL1-L41

Testing

• ⚠️ pytest libs/rag-core-api/tests (fails: missing langchain_core, langchain_community, fastapi, pytest_asyncio dependencies in the current environment)

View task →