Implement kb management#1
Conversation
There was a problem hiding this comment.
Pull Request Overview
This pull request introduces comprehensive Knowledge Base (KB) management capabilities to the OpenWebUI Python SDK and CLI, while also refactoring API response handling for better reliability and consistency.
- Knowledge Base SDK module with full CRUD operations for KBs and files, including batch directory uploads with
.kbignoresupport - New CLI commands under
owui kbfor creating, listing, uploading to, and managing knowledge bases and their files - Centralized API response handling utility that eliminates duplicated error logic across different API modules
Reviewed Changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| openwebui/api/knowledge.py | Implements comprehensive KnowledgeBaseAPI with file upload, directory batch upload, and KB management |
| openwebui/cli/main.py | Adds complete kb command group with create, list, upload, update, and delete operations |
| openwebui/utils/api_utils.py | Centralizes API response handling and error parsing across all API modules |
| openwebui/utils/kbignore_parser.py | Implements .kbignore file parsing using pathspec for gitignore-style pattern matching |
| tests/sdk/test_sdk_knowledge.py | Comprehensive unit tests for the KnowledgeBaseAPI functionality |
| tests/cli/test_cli_knowledge.py | CLI tests covering all knowledge base command scenarios |
| tests/integration/test_kb_sdk_integration.py | End-to-end integration tests for KB workflows |
| pyproject.toml | Adds new dependencies tqdm and pathspec for progress bars and ignore file support |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| # Attempt to decode as JSON if content-type suggests it | ||
| if "application/json" in response.headers.get("content-type", ""): | ||
| return json.loads(response.content.decode("utf-8", errors="ignore")) | ||
| return json.loads(response.content.decode("utf-8", errors="ignore")) # Fallback to raw string |
There was a problem hiding this comment.
The comment says 'Fallback to raw string' but the code is still attempting JSON parsing. This could raise a JSONDecodeError that isn't handled, contradicting the fallback intention.
| return json.loads(response.content.decode("utf-8", errors="ignore")) # Fallback to raw string | |
| return response.content.decode("utf-8", errors="ignore") # Fallback to raw string |
| try: | ||
| # Filter out any unexpected keys that are not part of KnowledgeResponse | ||
| # See TODO #1 above for context | ||
| kb_data = {k: v for k, v in kb_data.items() if k in models.KnowledgeResponse.__dict__.keys()} |
There was a problem hiding this comment.
Using __dict__.keys() on a class may not include all valid model attributes, especially for generated models that might use descriptors or properties. This could filter out valid fields. Consider using a more robust method to determine valid attributes.
| kb_data = {k: v for k, v in kb_data.items() if k in models.KnowledgeResponse.__dict__.keys()} | |
| kb_data = {k: v for k, v in kb_data.items() if k in models.KnowledgeResponse.__annotations__.keys()} |
| # This is where the RuntimeError usually happens if the loop is considered closed too soon. | ||
| # Adding a small delay can sometimes help with race conditions, but ideal fix | ||
| # is to ensure the cleanup is ordered correctly by pytest_asyncio. | ||
| await asyncio.sleep(1) |
There was a problem hiding this comment.
Adding a sleep before closing the client is a workaround that suggests a race condition in the cleanup logic. This could mask timing-related issues and make tests unreliable. Consider addressing the root cause instead of using artificial delays.
| await asyncio.sleep(1) |
| import asyncio # Assuming asyncio is available | ||
|
|
There was a problem hiding this comment.
These imports should be moved to the top of the file with other imports rather than being imported inside a function. This follows Python best practices and makes dependencies clear.
| import asyncio # Assuming asyncio is available |
This pull request introduces Knowledge Base (KB) management to the OpenWebUI Python SDK and CLI, bringing powerful new features for handling files, directories, and knowledge bases. It also refactors API response handling for greater reliability, and test coverage.
🚀 What's New
1. Knowledge Base SDK & CLI
openwebui.api.knowledge.KnowledgeBaseAPIfor KB creation, file upload, batch directory upload (with.kbignoresupport), listing, updating, and deletion.owui kb):kb create <name> --description <desc>kb list-kbskb list-files <kb-id> [--search "query"]kb upload-file <file> --kb-id <id>kb upload-dir <dir> --kb-id <id>kb update-file <file-id> <new-file>kb delete-file <file-id>kb delete-all-files <kb-id> [--yes].kbignorerules on directory upload, usingpathspecfor gitignore-style pattern matching.2. Refactored API Response Handling
openwebui/utils/api_utils.py:handle_api_response3. Docs & Usage
.kbignorehandling, and batch management.4. Test Coverage
tests/sdk/test_sdk_knowledge.pytests/cli/test_cli_knowledge.pytests/integration/test_kb_sdk_integration.pytests/conftest.pyto provide KB creation & cleanup.5. Project Metadata
tqdm,pathspecfor progress bars and ignore file support.🔧 Refactoring & Improvements