Add HuggingFace API search for models and datasets#9
Conversation
Implemented comprehensive search feature for HuggingFace models and datasets: - Created HuggingFaceService for API integration - Search models with sorting and filtering - Search datasets with sorting and filtering - Fetch detailed model/dataset information - Comprehensive error handling and input validation - Created HFSearchDialog component - Maximized overlay dialog for search results - Display search results with key metadata (downloads, likes, tags) - View detailed information including descriptions and model cards - Direct links to HuggingFace website for each result - Support for both models and datasets - Integrated search into Configure page - Added "Search Models" button next to Model ID input - Added "Search Datasets" button next to Calibration Dataset input - Users can now browse and select from HuggingFace Hub instead of manual typing - All API calls use the huggingface_hub library - Type-safe with proper error handling and logging - Validated all input parameters to ensure correct API usage
There was a problem hiding this comment.
Pull Request Overview
This PR adds HuggingFace Hub integration to MSQuant, enabling users to search and select models and datasets directly from the UI instead of manually typing repository IDs.
- Adds a new
HuggingFaceServiceclass for searching models/datasets via the HuggingFace Hub API - Creates a
HFSearchDialogUI component with search functionality, sorting, and detail views - Integrates search dialogs into the configure page with "Search Models" and "Search Datasets" buttons
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/msquant/services/huggingface.py | New service implementing HuggingFace Hub API wrapper for searching models/datasets and fetching details |
| src/msquant/services/init.py | Exports new HuggingFaceService and HFSearchResult classes |
| src/msquant/app/components/hf_search_dialog.py | New UI component providing interactive search dialog for HuggingFace models and datasets |
| src/msquant/app/components/init.py | Exports new HFSearchDialog component |
| src/msquant/app/pages/configure.py | Integrates search dialogs into model and dataset input fields |
| src/msquant/app/main.py | Initializes HuggingFaceService and passes it to configure page |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| description = card_data.get('description', '') or card_data.get('model-index', {}).get('results', [{}])[0].get('task', {}).get('name', '') | ||
| elif hasattr(card_data, 'to_dict'): |
There was a problem hiding this comment.
This line is excessively complex with deeply nested .get() calls, making it difficult to read and maintain. Consider breaking this into multiple lines with intermediate variables to improve clarity. For example, extract the model-index fallback logic into a separate variable or helper method.
| description = card_data.get('description', '') or card_data.get('model-index', {}).get('results', [{}])[0].get('task', {}).get('name', '') | |
| elif hasattr(card_data, 'to_dict'): | |
| # Try to get description directly | |
| description = card_data.get('description', '') | |
| # Fallback: try to get task name from model-index | |
| if not description: | |
| model_index = card_data.get('model-index', {}) | |
| results = model_index.get('results', [{}]) | |
| if isinstance(results, list) and results: | |
| first_result = results[0] | |
| task = first_result.get('task', {}) | |
| description = task.get('name', '') |
| if not isinstance(query, str): | ||
| raise ValueError(f"query must be a string, got {type(query)}") |
There was a problem hiding this comment.
The input validation logic for parameters (query, limit, sort, direction) is duplicated between search_models and search_datasets methods (lines 61-68 and 121-128). Consider extracting this validation into a private helper method to reduce code duplication.
| def _model_to_search_result(self, model: ModelInfo) -> HFSearchResult: | ||
| """Convert ModelInfo to HFSearchResult.""" | ||
| return HFSearchResult( | ||
| id=model.id, | ||
| author=getattr(model, 'author', None), | ||
| description=None, # Description not available in list results | ||
| downloads=getattr(model, 'downloads', 0), | ||
| likes=getattr(model, 'likes', 0), | ||
| tags=getattr(model, 'tags', []) or [], | ||
| last_modified=str(getattr(model, 'lastModified', '')), | ||
| created_at=str(getattr(model, 'created_at', '')), | ||
| library_name=getattr(model, 'library_name', None), | ||
| pipeline_tag=getattr(model, 'pipeline_tag', None), | ||
| ) |
There was a problem hiding this comment.
The pattern getattr(model, 'tags', []) or [] on line 273 is redundant - if the default value [] is returned by getattr, the or [] has no effect. Consider simplifying to getattr(model, 'tags', []). The same issue exists in the _dataset_to_search_result method on line 288.
| with ui.row().classes('gap-4'): | ||
| ui.button( | ||
| 'Select This', | ||
| on_click=lambda: [self._select_item(result), details_dialog.close()] |
There was a problem hiding this comment.
Using a list comprehension in a lambda for side effects (line 284) is not idiomatic Python and can be confusing. Consider defining a proper function or using a lambda with a simple function call that performs both actions.
|
|
||
| def __init__(self): | ||
| """Initialize the HuggingFace service.""" | ||
| self._cache: Dict[str, Any] = {} |
There was a problem hiding this comment.
The _cache attribute is initialized but never used anywhere in the HuggingFaceService class. Consider removing it or implementing caching functionality if it was intended to be used.
| self._cache: Dict[str, Any] = {} |
|
Automated review 🤖 Summary of ChangesThis PR introduces a HuggingFace API integration for searching models and datasets within the MSQuant application. It adds a new Key Changes & Positives
Potential Issues & Recommendations
Language/Framework Checks
Security & Privacy
Build/CI & Ops
Tests
Approval RecommendationRequest changes
|
Fixed all pyright type checking errors identified in CI: hf_search_dialog.py: - Removed Optional types from UI elements (they're always initialized) - Removed async from _perform_search and _show_details (no await statements) - Added proper type casting for sort (str) and direction (int) values - Fixed lambda event handlers to accept event parameter properly - Added missing type annotations for sort_select and direction_select huggingface.py: - Added Literal type import for proper typing - Fixed direction parameter to match huggingface_hub API expectations - Convert direction parameter: -1 maps to Literal[-1], 1 maps to None - This matches the HuggingFace Hub API signature requirements All 20 type errors are now resolved.
|
Automated review 🤖 Summary of ChangesThis PR introduces a HuggingFace API integration for searching models and datasets within the MSQuant application. It adds a new Key Changes & Positives
Potential Issues & Recommendations
Language/Framework Checks
Security & Privacy
Build/CI & Ops
Tests
Approval RecommendationApprove with caveats
|
Implemented comprehensive search feature for HuggingFace models and datasets:
Created HuggingFaceService for API integration
Created HFSearchDialog component
Integrated search into Configure page
All API calls use the huggingface_hub library
Type-safe with proper error handling and logging
Validated all input parameters to ensure correct API usage