Skip to content

Add test infrastructure and fixtures for University Agent unit tests#17

Open
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
devin/1760304728-test-infrastructure-university-agent
Open

Add test infrastructure and fixtures for University Agent unit tests#17
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
devin/1760304728-test-infrastructure-university-agent

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 12, 2025

Copy link
Copy Markdown

Description

This PR implements comprehensive unit test coverage for the University Agent components as part of Tasks 5.1, 5.4, and 5.5. It adds test infrastructure with reusable fixtures and creates 59 new unit tests covering the UniversityPlugin and GeminiChatCompletionFactory classes.

Link to Devin run: https://app.devin.ai/sessions/1bb810da59f64d8e8a22acab3c5602fc
Requested by: Shawn Azman (@ShawnAzman)

Changes

Test Infrastructure (Task 5.5)

  • Created tests/conftest.py with 7 pytest fixtures:
    • mock_app_config and mock_app_config_with_api_key for configuration mocking
    • mock_requests_get and mock_requests_exception for HTTP request mocking
    • mock_google_ai_chat_completion for Google Gemini API mocking
    • university_api_responses and university_search_responses for loading test data
  • Created tests/fixtures/university_api_responses.json with mock API response data (279 lines)
  • Created tests/fixtures/university_search_responses.json with search scenario data

UniversityPlugin Tests (Task 5.1) - 37 tests

  • Success path tests (8 tests): Verify both plugin methods (search_universities and get_universities_by_country) with various scenarios including successful responses, empty results, and result limits
  • Data parsing tests (3 tests): Validate JSON-to-Pydantic model conversion, nullable field handling, and multiple domains/pages
  • Search result format tests (3 tests): Verify UniversitySearchResult structure and format
  • Error handling tests (9 tests): Network failures, timeouts, HTTP errors, and generic exceptions
  • Input validation tests (6 tests): Empty strings, special characters, very long strings, Unicode
  • Malformed response tests (8 tests): Invalid JSON, missing fields, unexpected data types, null responses

GeminiChatCompletionFactory Tests (Task 5.4) - 22 tests

  • Initialization tests (4 tests): Valid API key, API key storage, missing API key, parent class initialization
  • Configuration tests (1 test): Verify get_configs() returns empty list
  • Valid model tests (4 tests): Parametrized tests for all 4 supported Gemini models (gemini-1.5-flash, gemini-1.5-pro, gemini-1.0-pro, gemini-2.0-flash-lite)
  • Error handling tests (3 tests): Invalid model name handling for all three factory methods
  • Model type tests (4 tests): Verify ModelType.GOOGLE returned for all supported models
  • Structured output tests (4 tests): Verify all models support structured output
  • Factory method tests (2 tests): Default and custom parameter handling for create_chat_completion()

Type of Change

  • New feature (test coverage)
  • Documentation (test docstrings)

Important Review Points

⚠️ Mock data accuracy: The fixture data uses "state-province" (hyphenated) which maps to state_province (underscored) in the Pydantic model. Verify this matches the actual universities.hipolabs.com API format.

⚠️ Result limit verification: Tests verify that search_universities() returns max 10 results and get_universities_by_country() returns max 20 results. Confirm these limits are implemented in the actual UniversityPlugin code (lines 59 and 105 in custom_plugins.py per task description).

⚠️ No integration tests: All HTTP requests and API calls are mocked. The plugin has never been tested against the real universities API or Google Gemini API.

⚠️ GoogleAIChatCompletion mocking: All Gemini API calls are properly mocked using unittest.mock.patch() to prevent actual API requests during testing.

Test Results

37 passed for UniversityPlugin
22 passed for GeminiChatCompletionFactory
Total: 59 new unit tests, all passing
Ruff linting: All checks passed

Additional Comments

  • This PR combines multiple tasks (5.1, 5.4, 5.5) to provide complete test coverage in a single changeset
  • All tests use mocks and fixtures to avoid external dependencies
  • Test infrastructure is designed for reuse across future test files
  • Comprehensive error handling coverage ensures robustness of both plugins

- Created tests/unit/ directory for unit tests
- Created tests/fixtures/ directory with mock data files
- Added university_api_responses.json with realistic API response scenarios
- Added university_search_responses.json with search test scenarios
- Created conftest.py with shared pytest fixtures:
  - mock_app_config: Mock AppConfig for testing components
  - mock_app_config_with_api_key: Pre-configured with test API key
  - mock_requests_get: Mock requests.get() for HTTP testing
  - university_api_responses: Loads mock API response data
  - university_search_responses: Loads search scenario data
  - mock_google_ai_chat_completion: Mock GoogleAIChatCompletion
  - mock_requests_exception: Mock requests exception for error testing

This infrastructure supports upcoming unit tests for:
- UniversityPlugin class
- GeminiChatCompletionFactory class
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

- Created comprehensive test suite for UniversityPlugin with 14 test cases
- Test search_universities() method with various scenarios (successful, empty, limit)
- Test get_universities_by_country() method with various scenarios
- Test university data parsing and transformation from JSON to Pydantic models
- Test UniversitySearchResult response format validation
- All HTTP requests are properly mocked using pytest fixtures from Task 5.5
- Tests validate 10 result limit for search_universities and 20 for get_universities_by_country
- Tests verify proper handling of nullable state_province field
- All tests pass and lint checks pass
- Add TestUniversityPluginErrorHandling class with 8 tests:
  - Test network failures (RequestException) for both methods
  - Test generic exceptions for both methods
  - Test timeout scenarios for both methods
  - Test HTTP errors (404, 500) for both methods

- Add TestUniversityPluginInputValidation class with 6 tests:
  - Test empty string inputs
  - Test special characters in inputs
  - Test very long query strings
  - Test unicode characters

- Add TestUniversityPluginMalformedResponses class with 7 tests:
  - Test invalid JSON responses
  - Test missing required fields
  - Test unexpected data types
  - Test non-list API responses
  - Test null responses
  - Test partial data handling

All tests use mocked HTTP requests and validate error handling
behavior according to the plugin implementation. Total: 21 new tests.
…zation

- Create tests/unit/test_gemini_completion_factory.py with 5 test cases
- Test successful initialization with valid API key
- Test API key storage and retrieval from AppConfig
- Test behavior with missing/None API key
- Test parent class initialization is called correctly
- Test get_configs() returns empty list
- All tests pass and ruff linting passes
- Uses fixtures from conftest.py (mock_app_config, mock_app_config_with_api_key)
- Added 17 new tests covering model validation and factory methods
- Tests cover all 4 supported Gemini models (1.5-flash, 1.5-pro, 1.0-pro, 2.0-flash-lite)
- Verify get_chat_completion_for_model_name() with valid/invalid models
- Test get_model_type_for_name() returns ModelType.GOOGLE
- Test model_supports_structured_output() returns True for all models
- Test create_chat_completion() with default and custom parameters
- All GoogleAIChatCompletion API calls are mocked to prevent actual API requests
- Total 22 tests (5 existing + 17 new), all passing with clean ruff lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant