feat(test): add test infrastructure and fix test suite (Phase T0) - #12
Merged
Merged
Conversation
Rewrite run-tests.sh to use per-package isolated venvs (uv sync + uv run pytest) instead of a single shared venv. This eliminates dependency conflicts across the 35 instrumentation packages and matches the upstream CI approach. Test runner enhancements: - Add --all, --cassettes, --python flags - Per-package venv isolation via uv sync + uv run pytest - Platform and Python version-aware skip logic - Smart install failure handling (skip vs error) - Exclude sample-app from discovery - Guard against uncommitted uv.lock changes CI pipeline (fr-pr.yaml): - Add all-tests job running --all (UT + FR + VCR) - Both jobs now test Python 3.10, 3.11, 3.12, 3.13 - Add uv install step (astral-sh/setup-uv) Test fixes across packages: - traceloop-sdk: fix endpoint detection for FR rebranding, fix span postprocess callback fixture isolation - bedrock: fix streaming wrapper attribute name, scope metric assertions to bedrock-specific metrics only - openai: use SpanAttributes constants instead of hardcoded strings for reasoning attributes - anthropic: add safety handler cleanup fixture - google-genai: add session-scoped instrumentor with re-instrument guard, make token metric assertion conditional, pin google-genai test dep - llamaindex: pin llama-index and llama-index-core test deps - lancedb: fix version constraint, add stale table cleanup - fortifyroot: add pytest-asyncio test dependency New files: - .env.test.example: environment variable template for cassette recording - scripts/templates/conftest_vcr.py: standardized VCR conftest template Validated: 0 test failures on macOS (31 pass, 4 skip) and Ubuntu 22.04 (34 pass, 1 skip).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
run-tests.shto use per-package isolated venvs (uv sync+uv run pytest) instead of a shared venv, eliminating cross-package dependency conflicts--all,--cassettes,--pythonflags and platform/version-aware skip logicfr-pr.yamlCI withall-testsjob across Python 3.10–3.13.env.test.examplefor future test phasesTest results
Test plan
./scripts/run-tests.sh --allpasses on macOS (0 failures)./scripts/run-tests.sh --frpasses on macOS (0 failures)