Skip to content

Add CI integration testing for backend API commands #1

Description

@Rome-1

Context

The CLI has two test layers:

  • Unit tests (node/tests/*.test.ts): Run via pnpm test / vitest in CI. Test local logic (config-manager, command-interceptor, pattern-engine). No API calls.
  • Integration tests (python/test_rafter_cli.sh, node/tests/test_rafter_cli.sh): Manual shell scripts that hit the live Rafter API. Not run in CI.

The integration tests previously had a hardcoded API key (now removed—see security review). They need to be wired into CI properly.

Requirements

1. CI integration test job

  • Add test-integration job to publish.yml (or a separate workflow)
  • Use RAFTER_API_KEY from GitHub Actions secrets
  • Run both Node and Python integration scripts
  • Gate on unit tests passing first

2. Python test gate

  • publish-python currently has no test prerequisite (unlike Node which has test-node)
  • Add a test-python job and make publish-python depend on it

3. Test improvements

  • Add timeout to integration tests (prevent hanging on API issues)
  • Add explicit pass/fail exit codes
  • Consider converting shell scripts to proper test frameworks (vitest for Node, pytest for Python) with API mocking for unit-level and real calls for integration-level

References

  • Security review: hardcoded key finding (C-0) in Projects/Rafter/Agent-Security-CLI/Security-Review-2026-02-09.md
  • Current CI: .github/workflows/publish.yml, .github/workflows/validate-release.yml

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions