feat: add nimble_web_search data source

[wildcard]

Summary

• Adds sources/nimble_web_search, a NAT data source that wraps langchain-nimble's NimbleSearchRetriever, mirroring the existing exa_web_search and tavily_web_search packages (typed config, stub-on-missing-key, retries, content truncation, XML-tagged output).
• Exposes lite / fast / deep search_depth, typed as a Literal so invalid values fail at config-parse time. lite is the default — metadata-only, token-cheap, works on any account. fast is enterprise-tier and surfaces a clear 403 entitlement message on non-enterprise keys.
• Adds a typed focus mode (default general, validated Literal), country / locale regional controls, and an optional max_content_length per-result cap.
• Wires the plugin into the workspace, deploy/Dockerfile, and scripts/setup.sh so it installs in dev, Docker, and container builds. The Docker layer also installs Nimble's runtime deps (langchain-nimble, nimble-python, lockfile-pinned) so _type: nimble_web_search resolves in built images, where the --no-dev sync would otherwise omit them.
• Documentation across the config reference, extending guides, installation, quick-start, deployment (docker-build, docker-compose, kubernetes), FAQ, and troubleshooting (with 401 and 403-enterprise rows).

Motivation

AI-Q ships Tavily- and Exa-backed web search today. Nimble provides web search and content extraction for AI agents; this adds it as a first-class alternative with the same ergonomics and config surface — handy for users who already have a Nimble subscription, prefer its regional coverage, or want to test across multiple search backends. The default provider is unchanged (Tavily stays the documented default).

It wraps the official langchain-nimble package (maintained by Nimble) rather than calling the HTTP API directly, so retry, auth, and response normalization come from the upstream integration — the same rationale as the Exa source (#181).

Configuration

functions:
  web_search_tool:
    _type: nimble_web_search
    max_results: 5
    search_depth: lite      # lite (default) | fast (enterprise) | deep
    focus: general          # general (default) | news | location | shopping | geo | social
    country: US
    locale: en

NIMBLE_API_KEY=...   # or set api_key: in the YAML

How it works

A real lite query with NIMBLE_API_KEY set, trimmed:

<Document href="https://docs.nvidia.com/aiq-blueprint/1.2.1/index.html">
<title>
NVIDIA AI-Q Blueprint
</title>
AI-Q combines intelligent query routing, multi-agent research pipelines, and
pluggable knowledge retrieval to deliver comprehensive, citation-backed answers.
</Document>

---

<Document href="https://build.nvidia.com/nvidia/aiq">
<title>
NVIDIA AI-Q Blueprint for intelligent agents
</title>
The NVIDIA AI-Q Blueprint enables developers to build fully customizable AI
agents that they own, inspect and control. Built on LangChain…
</Document>

Each result renders as an XML <Document> block — the same shape the Tavily and Exa sources produce — so existing AI-Q agents consume it with no changes. To try it, point any existing web-search config at _type: nimble_web_search and run nat run (swap advanced_search: true → search_depth: deep).

How this was tested

• uv run pytest sources/nimble_web_search — 32 passed, credential-free (the SDK is mocked; no live network in CI).
• uv run pytest sources/exa_web_search sources/nimble_web_search — 46 passed, confirming the new package co-runs cleanly with a sibling source. The test module has a unique name and no tests/__init__.py, so there's no pytest collection collision when sources are collected together.
• ruff check and ruff format --check — clean (whole repo). uv lock --check — no drift.
• Repo pre-commit hooks pass on the changed files: detect-secrets, markdown-link-check (all README/docs links resolve), end-of-file-fixer, trailing-whitespace, check-added-large-files, and uv-lock — matching the AIQ CI lint job.
• nat info components --types function lists nimble_web_search (1.0.0) next to exa_web_search and tavily_web_search, so _type: nimble_web_search resolves in a workflow.
• Container runtime: langchain-nimble==3.0.0 + nimble-python==0.18.0 install and import cleanly in a fresh environment the same way deploy/Dockerfile installs them, so _type: nimble_web_search resolves in built images — not only in editable dev installs.
• Live smoke with a real NIMBLE_API_KEY across lite and deep, plus the non-enterprise fast path (returns the friendly 403 entitlement message). Output is redacted; no key is logged by construction.

Coverage: config defaults / all fields / invalid-enum rejection (incl. focus) / out-of-range numeric fields rejected / focus defaults to general and reaches the SDK / non-default focus passthrough / include_answer absent from config and kwargs / FunctionBaseConfig inheritance, the missing-key stub + warn-once, key-from-config env hydration, result rendering + description fallback, markup escaping of untrusted fields, search_depth and country/locale passthrough, query and content truncation (incl. small-limit hard-cut), empty-result handling, retry-then-succeed, non-transient (401/403) errors short-circuiting without retry, final-retry failure, and the 401 / 403 branches.

How this was reviewed

• Diffed against the merged Exa source to keep structure, retry loop, truncation, and output format at parity; the deviations below are deliberate.
• Confirmed credential-free CI behavior and co-run safety with a sibling source.
• Scanned the package for secrets and for hardcoded search-endpoint names — none.

Deviations from the Exa source (all deliberate)

1. search_depth (3-value enum), a typed focus mode (default general), plus country / locale, mirroring langchain-nimble's surface, where Exa exposes search_type / full_text / highlights. focus is a workflow-config setting, not an agent parameter, so general research queries cannot drift to news.
2. Falls back to the result's description when page_content is empty — Nimble's lite mode returns metadata only.
3. A 403 branch that turns Nimble's enterprise-tier gating into a clear, actionable message. Exa has no tier gating, so no equivalent.
4. include_answer (answer generation) is intentionally not exposed in this initial integration. It can be added in a follow-up.
5. Untrusted result fields (url, title, body) are HTML-escaped before rendering into the <Document> markup, so a result can't break the block or inject into downstream parsers.
6. Numeric config fields are bounded: max_results 1-100 (matching langchain-nimble's own ge=1, le=100), max_retries ge=1, max_content_length ge=1 (use None to disable truncation). Invalid values fail at config-parse time, and content truncation hard-cuts safely for very small limits.

Known limitations

• max_results is a soft cap — Nimble may return up to N+2 documents for N. The provider returns them all; downstream consumers can slice.
• lite mode returns empty page_content; the provider renders the description (~150 chars, organic-result quality).
• The non-enterprise fast path is characterized via its 403 message; the enterprise fast behavior itself isn't exercised here.

Scope

In: the nimble_web_search provider, config/docs/deploy wiring, 32 unit tests, README, troubleshooting rows.

Not in (easy follow-ups): Nimble Extract / Map / Crawl / Agents; include_answer; framework integrations beyond AI-Q's data-source mechanism; any change to the default provider.

Security

• No secrets committed — deploy/.env.example carries a commented NIMBLE_API_KEY= placeholder only.
• Key read from env or a SecretStr config field; never logged.
• Unit tests need no credentials; the live smoke uses an inline env var and redacted output.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a0088290a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[greptile-apps]

Greptile Summary

This PR adds nimble_web_search as a first-class NAT data source, mirroring the existing exa_web_search and tavily_web_search packages with typed config, a stub on missing key, retry/backoff, content truncation, and HTML-escaped XML output. It wires the plugin into the workspace, Dockerfile, and setup.sh, and ships 32 credential-free unit tests alongside documentation updates across config reference, installation, quick-start, and troubleshooting pages.

• sources/nimble_web_search/src/register.py: typed Pydantic config (search_depth, focus, country, locale, max_content_length), NimbleSearchRetriever wrapping with non-transient short-circuit (401/403/ValueError return immediately, transient errors use exponential backoff), and HTML-escaped <Document> rendering with description fallback for lite-mode empty page_content.
• Dockerfile / pyproject.toml / uv.lock: langchain-nimble==3.0.0 and nimble-python==0.18.0 pinned in the Docker layer so the plugin resolves in --no-dev container builds; workspace membership added for dev installs.
• Documentation: config reference, installation, quick-start, troubleshooting, and the package README all updated to include Nimble as a peer of Tavily and Exa.

Confidence Score: 5/5

Safe to merge — the new plugin follows the established pattern, previous review feedback has been addressed, and the test suite is thorough.

The implementation is structurally sound: typed config with validated enums, non-transient error short-circuit (previously addressed), small-limit truncation guard (previously addressed), HTML-escaped output, and 32 credential-free tests. The only findings are a stale test count in the README verification checklist and an unreachable safety-net return at the end of the retry loop — neither affects runtime behavior.

No files require special attention. sources/nimble_web_search/src/register.py has a cosmetic dead-code line; sources/nimble_web_search/README.md has a stale test count in the verification section.

Important Files Changed

Filename	Overview
sources/nimble_web_search/src/register.py	Core implementation: typed config, retry loop with non-transient short-circuit, HTML-escaped XML rendering, content truncation with small-limit guard. One unreachable dead-code return on the last line of the retry loop.
sources/nimble_web_search/tests/test_nimble_register.py	32 credential-free tests covering config validation, stub behavior, retry logic, non-transient short-circuit, truncation edge cases, HTML escaping, and field passthrough. Coverage is thorough.
sources/nimble_web_search/README.md	Documentation is accurate and well-structured; the verification checklist has a stale test count (21 vs the actual 32).
sources/nimble_web_search/pyproject.toml	Package metadata correct; langchain-nimble>=3.0.0,<4.0.0 dependency matches Dockerfile pinned version; entry-point wires nimble_web_search NAT plugin.
deploy/Dockerfile	Adds editable install of nimble_web_search and pins langchain-nimble==3.0.0 nimble-python==0.18.0 so the plugin resolves in --no-dev container builds.
docs/source/customization/configuration-reference.md	New nimble_web_search section documents all config fields, search_depth options, and focus modes accurately.

Sequence Diagram

sequenceDiagram
    participant Agent
    participant nimble_web_search
    participant NimbleSearchRetriever
    participant NimbleAPI

    Agent->>nimble_web_search: question (str)
    Note over nimble_web_search: Truncate query to 400 chars
    loop attempt in range(max_retries)
        nimble_web_search->>NimbleSearchRetriever: ainvoke(question)
        NimbleSearchRetriever->>NimbleAPI: HTTP search request
        alt Success
            NimbleAPI-->>NimbleSearchRetriever: docs[]
            NimbleSearchRetriever-->>nimble_web_search: docs[]
            Note over nimble_web_search: Render XML Document blocks
            nimble_web_search-->>Agent: formatted result string
        else Empty results (ValueError)
            nimble_web_search-->>Agent: no results message (no retry)
        else 401 Unauthorized
            nimble_web_search-->>Agent: friendly key error message (no retry)
        else 403 Forbidden
            nimble_web_search-->>Agent: friendly entitlement message (no retry)
        else Transient error
            Note over nimble_web_search: sleep(2^attempt), retry
        end
    end

Loading

_{Reviews (5): Last reviewed commit: "fix(nimble_web_search): add typed focus ..." | Re-trigger Greptile}