test: Implement comprehensive E2E tests for University Agent system

[devin-ai-integration]

Description

This PR implements comprehensive end-to-end tests for the University Agent system using Robot Framework, as requested in Task 6. The implementation focuses on API-level testing with two main test suites: a UAT suite for testing live services (primary focus) and an E2E suite with automated service management using MockGeminiFactory.

Link to Devin run: https://app.devin.ai/sessions/54b4786af8a9421a80b3e9ac6472a4c5
Requested by: @thepollari (jussi.pollari@merck.com)

Changes

• Added Robot Framework dependencies to pyproject.toml:

  • robotframework>=7.0.0 - Core test framework
  • robotframework-requests>=0.9.7 - HTTP API testing
  • robotframework-jsonlibrary>=0.5.0 - JSON validation

• Created UAT test suite (tests/e2e/uat_live_system.robot) - PRIMARY FOCUS:

  • Tests against already-running services with real API keys
  • 7 comprehensive test cases covering API accessibility, university search, multi-turn conversations, response validation, and error handling
  • Uses tags for selective test execution (smoke, api, search, error)

• Created E2E test suite (tests/e2e/university_agent_system.robot):

  • Automated service startup/shutdown with MockGeminiFactory
  • 4 basic test cases for service management and API structure validation
  • Suitable for CI/CD without API key requirements

• Implemented MockGeminiFactory (tests/e2e/mock_gemini_factory.py):

  • Follows existing MockChatCompletionFactory pattern
  • Supports Gemini model types for predictable testing
  • Integrates via TA_CUSTOM_CHAT_COMPLETION_FACTORY_* environment variables

• Added service management keywords (tests/e2e/keywords/service_management.robot):

  • Simple process control for University Agent and Streamlit UI
  • Health check validation and process cleanup
  • Uses relative paths for portability

• Created comprehensive documentation (tests/e2e/README.md):

  • Setup instructions and environment configuration
  • Usage examples for both test suites
  • Troubleshooting guides and CI/CD integration examples

Type of Change

• New feature
• Documentation

Critical Review Areas

⚠️ Important: This PR adds test infrastructure that hasn't been fully validated end-to-end. Please focus on:

1. Path Resolution: Verify ${CURDIR} relative paths in Robot Framework files resolve correctly when executed
2. MockGeminiFactory Integration: Confirm the mock factory actually loads and integrates with the chat completion system
3. API Response Format Assumptions: Validate that tests assume correct University Agent response structure (output_raw, token_usage, etc.)
4. Service Management Robustness: Test that process startup/shutdown keywords work reliably with proper cleanup
5. Environment Variable Configuration: Ensure the environment setup instructions work across different deployment scenarios

Testing Status

• ✅ Robot Framework syntax validation passed (--dryrun)
• ✅ Python linting passed (ruff check)
• ✅ All existing tests pass (162 tests)
• ⚠️ End-to-end execution not yet validated - requires manual testing with live services

Next Steps

1. Test UAT suite against manually started University Agent services
2. Validate E2E suite with MockGeminiFactory integration
3. Verify service management keywords work reliably
4. Add to CI/CD pipeline once validated

Additional Comments

The implementation prioritizes API-level testing over complex browser automation as requested. The UAT suite serves as the primary validation tool for real user workflows, while the E2E suite provides automated testing suitable for CI/CD environments.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring