feat: add Brave and TinyFish web search providers#252
Open
JithendraNara wants to merge 5 commits into
Open
Conversation
Signed-off-by: Jithendra Nara <jnara01@indianatech.net>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds two new NAT data source plugins for web search providers (Brave Search API and TinyFish Search API), including registration logic, packaging, tests, and docs updates.
Changes:
- Introduces
brave_web_searchandtinyfish_web_searchsource packages with NAT registration functions and configs. - Adds unit tests covering default config behavior, formatting, retries, truncation, and missing-key stubs.
- Updates workspace/dev dependencies, Docker build, env examples, and documentation to include both new providers.
Reviewed changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| sources/tinyfish_web_search/src/register.py | Implements TinyFish NAT tool config + registration and request/formatting logic |
| sources/tinyfish_web_search/tests/test_tinyfish_register.py | Adds tests for TinyFish config + tool behavior |
| sources/tinyfish_web_search/src/init.py | Exposes TinyFish tool entrypoint |
| sources/tinyfish_web_search/pyproject.toml | Packages TinyFish plugin and registers NAT entry point |
| sources/tinyfish_web_search/README.md | Documents TinyFish configuration |
| sources/brave_web_search/src/register.py | Implements Brave NAT tool config + registration and request/formatting logic |
| sources/brave_web_search/tests/test_brave_register.py | Adds tests for Brave config + tool behavior |
| sources/brave_web_search/src/init.py | Exposes Brave tool entrypoint |
| sources/brave_web_search/pyproject.toml | Packages Brave plugin and registers NAT entry point |
| sources/brave_web_search/README.md | Documents Brave configuration |
| pyproject.toml | Adds Brave/TinyFish packages to workspace/dev deps |
| docs/source/resources/troubleshooting.md | Adds troubleshooting rows for Brave/TinyFish keys |
| docs/source/resources/faq.md | Lists Brave/TinyFish as available web search tools |
| docs/source/get-started/quick-start.md | Adds env var hints for Brave/TinyFish |
| docs/source/get-started/installation.md | Adds install/key docs for Brave/TinyFish |
| docs/source/extending/adding-a-tool.md | Adds Brave/TinyFish to tool list table |
| docs/source/extending/adding-a-data-source.md | Adds Brave/TinyFish to data source list table |
| docs/source/deployment/kubernetes.md | Documents Brave/TinyFish env vars |
| docs/source/deployment/docker-compose.md | Documents Brave/TinyFish env vars |
| docs/source/deployment/docker-build.md | Adds Brave/TinyFish to Docker build install list |
| docs/source/customization/tools-and-sources.md | Updates backend-API-key examples to include Brave/TinyFish |
| docs/source/customization/configuration-reference.md | Adds config reference sections for Brave/TinyFish |
| deploy/Dockerfile | Installs Brave/TinyFish source packages in the image |
| deploy/.env.example | Adds optional env vars for Brave/TinyFish |
| README.md | Updates install instructions and API key table for new providers |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+86
to
+93
| def _render_document(result: dict, max_content_length: int | None) -> str: | ||
| url = result.get("url", "") or "" | ||
| title = result.get("title", "") or "" | ||
| site_name = result.get("site_name") or "" | ||
| snippet = result.get("snippet") or result.get("description") or "" | ||
| body_parts = [str(part) for part in (site_name, snippet) if part] | ||
| body = _truncate_content("\n".join(body_parts), max_content_length) | ||
| return f'<Document href="{url}">\n<title>\n{title}\n</title>\n{body}\n</Document>' |
Comment on lines
+94
to
+105
| def _render_document(result: dict, max_content_length: int | None) -> str: | ||
| url = result.get("url", "") or "" | ||
| title = result.get("title", "") or "" | ||
| snippets: list[str] = [] | ||
| description = result.get("description") or result.get("snippet") or "" | ||
| if description: | ||
| snippets.append(str(description)) | ||
| extra_snippets = result.get("extra_snippets") or [] | ||
| if isinstance(extra_snippets, list): | ||
| snippets.extend(str(snippet) for snippet in extra_snippets if snippet) | ||
| body = _truncate_content("\n".join(snippets), max_content_length) | ||
| return f'<Document href="{url}">\n<title>\n{title}\n</title>\n{body}\n</Document>' |
Comment on lines
+81
to
+83
| if max_content_length and len(content) > max_content_length: | ||
| return content[: max_content_length - 3] + "..." | ||
| return content |
Comment on lines
+88
to
+91
| def _truncate_content(content: str, max_content_length: int | None) -> str: | ||
| if max_content_length and len(content) > max_content_length: | ||
| return content[: max_content_length - 3] + "..." | ||
| return content |
Comment on lines
+197
to
+204
| if "401" in error_msg or "Unauthorized" in error_msg: | ||
| return ( | ||
| "Error: Brave web search failed due to invalid API key (401 Unauthorized).\n" | ||
| "Please check your BRAVE_API_KEY and ensure it is valid.\n" | ||
| ) | ||
| if "429" in error_msg: | ||
| return "Error: Brave web search failed because the Brave Search API rate limit was exceeded.\n" | ||
| return f"Error: Brave web search failed - {error_msg}" |
Contributor
Greptile SummaryThis PR adds two new web search source packages —
Confidence Score: 5/5Both new packages are self-contained additions that follow established patterns in the codebase; no existing code paths are modified. The changes add two independent source packages. Config validation is properly constrained, the truncation and escaping helpers are correct, and both packages have well-exercised mock-based test suites. The only findings are a post-loop dead-code return and a cosmetic retry-latency issue on empty results — neither affects correctness for users who receive valid API responses. No files require special attention; the two register.py files contain the noted minor issues but are otherwise safe. Important Files Changed
Sequence DiagramsequenceDiagram
participant App
participant Register as brave/tinyfish_web_search (register)
participant Env as os.environ
participant API as Brave / TinyFish API
App->>Register: tool_config (api_key, max_results, ...)
Register->>Env: get API key (env var or config fallback)
alt API key missing
Register-->>App: yield stub FunctionInfo (error message)
else API key present
Register-->>App: yield live FunctionInfo
App->>Register: call _search(question)
loop up to max_retries
Register->>API: "GET /search?q=...&count=..."
alt HTTP error
API-->>Register: HTTPError (401 / 429 / other)
else Success
API-->>Register: JSON payload
Register->>Register: slice results[:max_results]
Register->>Register: _truncate_content + html.escape
Register-->>App: formatted Document XML
end
end
end
Reviews (4): Last reviewed commit: "chore: merge upstream develop" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Signed-off-by: Jithendra Nara <jnara01@indianatech.net>
Signed-off-by: Jithendra Nara <jnara01@indianatech.net>
Author
|
Addressed the bot review feedback in
Re-ran:
|
Signed-off-by: Jithendra Nara <jnara01@indianatech.net>
Signed-off-by: Jithendra Nara <jnara01@indianatech.net> # Conflicts: # .secrets.baseline
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
brave_web_searchsource package for Brave Search API web resultstinyfish_web_searchsource package for TinyFish Search API web resultsTesting
uv run ruff check sources/brave_web_search sources/tinyfish_web_searchuv run pytest sources/brave_web_search/tests sources/tinyfish_web_search/testsuv run pytest sources/exa_web_search/tests sources/brave_web_search/tests sources/tinyfish_web_search/testsuv lock --checkuv run python -m py_compile sources/brave_web_search/src/register.py sources/tinyfish_web_search/src/register.pygit diff --checkNotes