Skip to content

Conversation

@SL-Mar
Copy link
Owner

@SL-Mar SL-Mar commented Jan 26, 2026

Summary

This PR significantly improves the project's CI/CD infrastructure and testing coverage by enhancing the GitHub Actions workflow, adding comprehensive integration tests, and creating contributor documentation.

Key Changes

CI/CD Pipeline Improvements

  • Enhanced test reporting: Added --cov-report=term and --cov-fail-under=50 to pytest to enforce minimum coverage threshold and display coverage in terminal
  • Upgraded codecov action: Updated from v3 to v4 with explicit token configuration for more reliable coverage reporting
  • Stricter coverage enforcement: Changed fail_ci_if_error from false to true to fail builds when coverage upload fails
  • Improved security scanning: Modified pip-audit to install dev dependencies and use --strict flag for stricter vulnerability checking
  • New integration test job: Added dedicated integration job that runs after lint and type-check, with proper dependencies and markers for selective test execution

Comprehensive Integration Tests

  • New test file: tests/test_integration.py with 561 lines of integration tests covering:
    • CLI smoke tests for all major commands (search, download, summarize, generate, validate, backtest, auto, library, evolve, config-show)
    • Search command integration tests with mocked APIs
    • Generate command tests with mocked LLM responses
    • Validate command tests for both valid and invalid code
    • Autonomous mode (auto) command tests
    • Library builder command tests
    • Evolution (evolve) command tests
    • End-to-end workflow tests combining multiple commands
    • Error handling tests for missing API keys, network errors, and invalid inputs
  • Proper test organization: Uses pytest markers (@pytest.mark.integration) for selective test execution
  • Comprehensive mocking: All external service calls are properly mocked to ensure tests are isolated and fast

Documentation

  • New CONTRIBUTING.md: Comprehensive contributor guide including:
    • Code of conduct reference
    • Development setup instructions with virtual environment setup
    • Testing guidelines with examples
    • Code quality tools and commands
    • Branch naming conventions and commit message format
    • Pull request process and review guidelines
    • Coding standards and security best practices
    • Documentation requirements
    • Test organization and markers

Implementation Details

  • Integration tests use Click's CliRunner for testing CLI commands without actual execution
  • All external dependencies (APIs, LLMs, file systems) are mocked using unittest.mock
  • Tests are organized into logical test classes for better maintainability
  • Fixtures provide reusable test setup (CLI runner, mock environment)
  • Error handling tests verify graceful degradation and helpful error messages
  • End-to-end workflow tests demonstrate realistic usage patterns

Benefits

  • Improved code quality: Enforced minimum coverage threshold prevents regression
  • Better reliability: Integration tests catch CLI-level issues before production
  • Easier onboarding: CONTRIBUTING.md provides clear guidelines for new contributors
  • Stronger security: Stricter pip-audit checks catch more vulnerabilities
  • Better visibility: Terminal coverage reports provide immediate feedback to developers

- Add comprehensive CLI integration tests with smoke tests and mocked workflows
- Fix pip-audit to fail on vulnerabilities (remove || true)
- Add coverage threshold (50%) and update codecov action
- Add separate integration test job in CI
- Add CONTRIBUTING.md with development setup and coding standards
- Add requirements-lock.txt with pinned versions for reproducible builds
- Add optional extras for LLM providers (openai, anthropic, mistral, all-llm)
- Add http_utils.py with retry logic (exponential backoff) and response caching
- Update article_tools.py to use new HTTP utilities for better reliability
- Simplify code of conduct section in CONTRIBUTING.md
@SL-Mar SL-Mar merged commit efe1c27 into main Jan 26, 2026
3 of 15 checks passed
@SL-Mar SL-Mar deleted the claude/assess-repo-quality-GgRDF branch January 26, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants