From 2e59f88b7b8a8b0d874680c6766a413cedac8380 Mon Sep 17 00:00:00 2001 From: Sohan Kshirsagar Date: Tue, 24 Mar 2026 15:24:10 -0700 Subject: [PATCH 1/5] add bug hunting skill --- .claude/commands/bug-hunt.md | 430 +++++++++++++++++++++++++++++++++++ .gitignore | 3 - 2 files changed, 430 insertions(+), 3 deletions(-) create mode 100644 .claude/commands/bug-hunt.md diff --git a/.claude/commands/bug-hunt.md b/.claude/commands/bug-hunt.md new file mode 100644 index 0000000..8dd8854 --- /dev/null +++ b/.claude/commands/bug-hunt.md @@ -0,0 +1,430 @@ +# Instrumentation Bug Hunting + +## Arguments + +$ARGUMENTS - The library name to test (e.g., aiohttp, django, fastapi, flask, grpc, httpx, psycopg, psycopg2, redis, requests, sqlalchemy, urllib, urllib3) + +## Library-to-GitHub-Repo Mapping + +Use this mapping to clone the package source code for analysis: + +| Library | GitHub Repo | Notes | +| ---------- | ------------------------------------------------- | ---------------------------------------------- | +| aiohttp | https://github.com/aio-libs/aiohttp | | +| django | https://github.com/django/django | | +| fastapi | https://github.com/tiangolo/fastapi | | +| flask | https://github.com/pallets/flask | | +| grpc | https://github.com/grpc/grpc | Focus on `src/python/grpcio/` | +| httpx | https://github.com/encode/httpx | | +| psycopg | https://github.com/psycopg/psycopg | Monorepo — focus on `psycopg/` | +| psycopg2 | https://github.com/psycopg/psycopg2 | | +| redis | https://github.com/redis/redis-py | | +| requests | https://github.com/psf/requests | | +| sqlalchemy | https://github.com/sqlalchemy/sqlalchemy | | +| urllib | N/A | Built-in Python stdlib — no repo to clone | +| urllib3 | https://github.com/urllib3/urllib3 | | + +## E2E Test Paths + +Each library has a single e2e-tests directory (no ESM/CJS variants): + +| Library | E2E test path | +| ---------- | -------------------------------------------------------| +| aiohttp | `drift/instrumentation/aiohttp/e2e-tests/` | +| django | `drift/instrumentation/django/e2e-tests/` | +| fastapi | `drift/instrumentation/fastapi/e2e-tests/` | +| flask | `drift/instrumentation/flask/e2e-tests/` | +| grpc | `drift/instrumentation/grpc/e2e-tests/` | +| httpx | `drift/instrumentation/httpx/e2e-tests/` | +| psycopg | `drift/instrumentation/psycopg/e2e-tests/` | +| psycopg2 | `drift/instrumentation/psycopg2/e2e-tests/` | +| redis | `drift/instrumentation/redis/e2e-tests/` | +| requests | `drift/instrumentation/requests/e2e-tests/` | +| sqlalchemy | `drift/instrumentation/sqlalchemy/e2e-tests/` | +| urllib | `drift/instrumentation/urllib/e2e-tests/` | +| urllib3 | `drift/instrumentation/urllib3/e2e-tests/` | + +--- + +## Phase 0: Environment Setup + +### 0.1 Validate the library argument + +The library must be one of: aiohttp, django, fastapi, flask, grpc, httpx, psycopg, psycopg2, redis, requests, sqlalchemy, urllib, urllib3. + +If the argument is invalid, list the valid options and stop. + +### 0.2 Docker Setup (Claude Code Web only) + +Check if Docker is running. If not, start it: + +```bash +dockerd --storage-driver=vfs &>/tmp/dockerd.log & +# Wait for Docker to be ready +for i in $(seq 1 30); do + docker info &>/dev/null 2>&1 && break + sleep 1 +done +docker info &>/dev/null 2>&1 || { echo "Docker failed to start. Check /tmp/dockerd.log"; exit 1; } +``` + +If Docker is already running, skip this step. + +### 0.3 Build the base Docker image + +This is required before running any e2e tests: + +```bash +cd +docker build -t python-e2e-base:latest -f drift/instrumentation/e2e_common/Dockerfile.base . +``` + +### 0.4 Clone the package source code (for analysis only) + +If the library has a GitHub repo (see mapping above), clone it for reference: + +```bash +git clone --depth 1 /tmp/-source +``` + +This is read-only reference material — you will NOT modify this repo. + +### 0.5 Create a working branch + +Skip this step if you are already on a dedicated branch (e.g., in Claude Code Web where each session has its own branch). + +```bash +git checkout -b bug-hunt/-$(date +%Y-%m-%d) +``` + +--- + +## Phase 1: Develop Understanding + +### 1.1 Analyze the Instrumentation Code + +Read the instrumentation code at: + +``` +drift/instrumentation/$ARGUMENTS/instrumentation.py +``` + +Also check for any additional files in the same directory: + +``` +drift/instrumentation/$ARGUMENTS/ +``` + +Identify: + +- Which functions from the package are patched/instrumented +- The patching strategy (what gets wrapped, when, and how) +- Any helper modules or utilities used + +### 1.2 Analyze Existing E2E Tests + +Review the test files: + +- `drift/instrumentation/$ARGUMENTS/e2e-tests/src/app.py` — all test endpoints (Flask/FastAPI/Django app) +- `drift/instrumentation/$ARGUMENTS/e2e-tests/src/test_requests.py` — which endpoints are called +- `drift/instrumentation/$ARGUMENTS/e2e-tests/entrypoint.py` — test orchestration and setup + +Understand what functionality is already tested and identify coverage gaps. + +### 1.3 Analyze the Package Source Code + +If you cloned the package source, read it to understand: + +- The package's entry points and full API surface +- Functions that are currently patched vs functions that exist but aren't patched +- Alternative call patterns, overloads, and edge cases + +--- + +## Phase 2: Identify Potential Gaps + +Reason about potential issues in the instrumentation. Consider: + +- **Untested parameters**: Parameter combinations not covered by existing tests +- **Alternative call patterns**: Can patched functions be invoked differently (sync vs async, context managers, generators)? +- **Missing patches**: Functions that should be instrumented but aren't +- **Edge cases**: None values, empty results, large payloads, streaming responses, connection errors +- **Context managers**: Are `with` statements and async context managers handled? +- **Async variations**: For async libraries (aiohttp, httpx, fastapi), are all async patterns covered? +- **ORM/wrapper usage**: Libraries like SQLAlchemy that wrap database drivers — are those call paths instrumented? +- **Real-world usage patterns**: How is the package typically used in production? + +Produce a prioritized list of potential bugs to investigate. + +--- + +## Phase 3: Initialize Bug Tracking Document + +Create `BUG_TRACKING.md` in the e2e test directory: + +```bash +# Path: drift/instrumentation/$ARGUMENTS/e2e-tests/BUG_TRACKING.md +``` + +```markdown +# $ARGUMENTS Instrumentation Bug Tracking + +Generated: + +## Summary + +- Total tests attempted: 0 +- Confirmed bugs: 0 +- No bugs found: 0 +- Skipped tests: 0 + +--- + +## Test Results + +(Tests will be documented below as they are completed) +``` + +--- + +## Phase 4: Write Tests and Verify Issues + +For each potential bug, follow this workflow: + +### 4.1 Initial Setup (Once) + +Navigate to the e2e test directory: + +```bash +cd drift/instrumentation/$ARGUMENTS/e2e-tests/ +``` + +Build and start Docker containers: + +```bash +docker compose build +docker compose run --rm -d --name bug-hunt-app app /bin/bash -c "sleep infinity" +``` + +This starts the container in the background so you can exec into it. + +### 4.2 Test Each Potential Bug (Repeat for each) + +#### A. Clean Previous Test Data + +```bash +docker exec bug-hunt-app rm -rf .tusk/traces/* .tusk/logs/* +``` + +#### B. Write New Test Endpoint + +Add a new endpoint to `src/app.py` that exercises the potential bug. Also add the corresponding request to `src/test_requests.py`. + +Example for Flask: + +```python +@app.route("/test/my-new-test", methods=["GET"]) +def my_new_test(): + # Your test code here + return jsonify({"success": True}) +``` + +#### C. Test in DISABLED Mode (No Instrumentation) + +Start server without instrumentation: + +```bash +docker exec -e TUSK_DRIFT_MODE=DISABLED bug-hunt-app python src/app.py & +sleep 5 +``` + +Hit the endpoint: + +```bash +docker exec bug-hunt-app curl -s http://localhost:8000/test/my-new-test +``` + +**Verify**: Response is correct and endpoint works. + +Stop the server: + +```bash +docker exec bug-hunt-app pkill -f "python src/app.py" || true +sleep 2 +``` + +**If the endpoint fails in DISABLED mode**: + +- Update `BUG_TRACKING.md` with status: "Skipped - Failed in DISABLED mode" +- Fix the test code or move on to next potential bug + +#### D. Test in RECORD Mode (With Instrumentation) + +Clean traces and logs: + +```bash +docker exec bug-hunt-app rm -rf .tusk/traces/* .tusk/logs/* +``` + +Start server in RECORD mode: + +```bash +docker exec -e TUSK_DRIFT_MODE=RECORD bug-hunt-app python src/app.py & +sleep 5 +``` + +Hit the endpoint: + +```bash +docker exec bug-hunt-app curl -s http://localhost:8000/test/my-new-test +``` + +Wait for spans to export: + +```bash +sleep 3 +``` + +Stop the server: + +```bash +docker exec bug-hunt-app pkill -f "python src/app.py" || true +sleep 2 +``` + +**Check for issues:** + +1. **Endpoint returns error or wrong response vs DISABLED mode**: + - BUG FOUND: Instrumentation breaks functionality + - Update `BUG_TRACKING.md`: Status "Confirmed Bug - RECORD mode failure", Failure Point "RECORD" + - Keep the endpoint, move to next + +2. **No traces created** (`docker exec bug-hunt-app ls .tusk/traces/`): + - BUG FOUND: Instrumentation failed to capture traffic + - Update `BUG_TRACKING.md`: Status "Confirmed Bug - No traces captured", Failure Point "RECORD" + - Keep the endpoint, move to next + +#### E. Test in REPLAY Mode + +Run the Tusk CLI to replay: + +```bash +docker exec -e TUSK_ANALYTICS_DISABLED=1 bug-hunt-app tusk drift run --print --output-format "json" --enable-service-logs +``` + +**Check for issues:** + +1. **Test fails** (`"passed": false` in JSON output): + - BUG FOUND: Replay doesn't match recording + - Update `BUG_TRACKING.md`: Status "Confirmed Bug - REPLAY mismatch", Failure Point "REPLAY" + +2. **No logs created** (`docker exec bug-hunt-app ls .tusk/logs/`): + - BUG FOUND: Replay failed to produce logs + - Update `BUG_TRACKING.md`: Status "Confirmed Bug - No replay logs", Failure Point "REPLAY" + +3. **Logs contain socket warnings**: + + ```bash + docker exec bug-hunt-app cat .tusk/logs/*.log | grep -i "TCP connect() called from inbound request context" + ``` + + - BUG FOUND: Unpatched dependency detected + - Update `BUG_TRACKING.md`: Status "Confirmed Bug - Unpatched dependency", Failure Point "REPLAY" + +#### F. No Bug Found + +If all modes pass with no issues: + +- Update `BUG_TRACKING.md`: Status "No Bug - Test passed all modes" +- **Remove the test endpoint** from `src/app.py` and `src/test_requests.py` +- Move to next potential bug + +--- + +## Phase 5: Bug Tracking Documentation Format + +After each test, append to `BUG_TRACKING.md`: + +```markdown +### Test N: [Brief description] + +**Status**: [Confirmed Bug | No Bug | Skipped] + +**Endpoint**: `/test/endpoint-name` + +**Failure Point**: [DISABLED | RECORD | REPLAY | N/A] + +**Description**: +[What this test was trying to uncover] + +**Expected Behavior**: +[What should happen] + +**Actual Behavior**: +[What actually happened] + +**Error Logs**: +``` + +[Relevant error messages, stack traces, or warnings] + +``` + +**Additional Notes**: +[Observations, potential root causes, context] + +--- +``` + +**Important**: Update `BUG_TRACKING.md` immediately after each test — do not batch updates. + +--- + +## Phase 6: Cleanup and Commit + +After testing all potential bugs: + +```bash +docker stop bug-hunt-app || true +docker compose down -v +``` + +Clean up cloned package source: + +```bash +rm -rf /tmp/*-source +``` + +**Final state of the e2e test files:** + +- `src/app.py` should contain ONLY the original endpoints + new endpoints that expose confirmed bugs +- `src/test_requests.py` should be updated to include requests to bug-exposing endpoints +- `BUG_TRACKING.md` should have accurate summary counts and all test results + +Commit the changes: + +```bash +git add drift/instrumentation/$ARGUMENTS/e2e-tests/ +git commit -m "bug-hunt($ARGUMENTS): add e2e tests exposing instrumentation bugs + +Found N confirmed bugs in $ARGUMENTS instrumentation. +See BUG_TRACKING.md for details" +``` + +Push the branch (skip if in Claude Code Web where the session handles this): + +```bash +git push origin bug-hunt/$ARGUMENTS-$(date +%Y-%m-%d) +``` + +--- + +## Success Criteria + +1. Created `BUG_TRACKING.md` before starting any tests +2. Tested all identified potential bugs +3. Updated `BUG_TRACKING.md` after each individual test +4. Only bug-exposing endpoints remain in test files +5. Removed test endpoints that didn't expose bugs +6. Accurate summary counts in `BUG_TRACKING.md` +7. Changes committed and pushed to a `bug-hunt/` branch diff --git a/.gitignore b/.gitignore index fd458bb..1c1425a 100644 --- a/.gitignore +++ b/.gitignore @@ -223,9 +223,6 @@ __marimo__/ # macOS .DS_Store -# Bug tracking -**/BUG_TRACKING.md - # Coverage coverage.lcov coverage.xml From f5d2b1ef321726d98482682dd62010f8c1d2a231 Mon Sep 17 00:00:00 2001 From: Sohan Kshirsagar Date: Tue, 24 Mar 2026 15:47:22 -0700 Subject: [PATCH 2/5] update skill --- .claude/commands/bug-hunt.md | 43 ++++++++++++++++++++++++++---------- 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/.claude/commands/bug-hunt.md b/.claude/commands/bug-hunt.md index 8dd8854..fa5b751 100644 --- a/.claude/commands/bug-hunt.md +++ b/.claude/commands/bug-hunt.md @@ -2,7 +2,17 @@ ## Arguments -$ARGUMENTS - The library name to test (e.g., aiohttp, django, fastapi, flask, grpc, httpx, psycopg, psycopg2, redis, requests, sqlalchemy, urllib, urllib3) +$ARGUMENTS - The library name, optionally followed by focus context. + +**Format**: ` [focus on ]` + +**Examples**: + +- `/bug-hunt redis` — broad bug hunting across all redis functionality +- `/bug-hunt redis focus on pub sub interactions` — prioritize pub/sub patterns +- `/bug-hunt psycopg2 focus on async cursors and connection pooling` — prioritize those areas + +**Parsing**: The first word is always the library name. Everything after it is the optional focus context. ## Library-to-GitHub-Repo Mapping @@ -48,11 +58,15 @@ Each library has a single e2e-tests directory (no ESM/CJS variants): ## Phase 0: Environment Setup -### 0.1 Validate the library argument +### 0.1 Parse and validate the arguments + +Extract the library name (first word) and optional focus context (remaining words) from the arguments. The library must be one of: aiohttp, django, fastapi, flask, grpc, httpx, psycopg, psycopg2, redis, requests, sqlalchemy, urllib, urllib3. -If the argument is invalid, list the valid options and stop. +If the library is invalid, list the valid options and stop. + +If focus context is provided, it will guide Phases 1 and 2 to prioritize that area of the library's functionality. ### 0.2 Docker Setup (Claude Code Web only) @@ -101,18 +115,20 @@ git checkout -b bug-hunt/-$(date +%Y-%m-%d) ## Phase 1: Develop Understanding +**If focus context was provided**, prioritize your analysis around that area. For example, if the focus is "pub sub interactions", concentrate on pub/sub-related code paths in the instrumentation, tests, and package source. + ### 1.1 Analyze the Instrumentation Code Read the instrumentation code at: ``` -drift/instrumentation/$ARGUMENTS/instrumentation.py +drift/instrumentation//instrumentation.py ``` Also check for any additional files in the same directory: ``` -drift/instrumentation/$ARGUMENTS/ +drift/instrumentation// ``` Identify: @@ -120,17 +136,20 @@ Identify: - Which functions from the package are patched/instrumented - The patching strategy (what gets wrapped, when, and how) - Any helper modules or utilities used +- **If focus context provided**: Which patches relate to the focus area, and what's missing? ### 1.2 Analyze Existing E2E Tests Review the test files: -- `drift/instrumentation/$ARGUMENTS/e2e-tests/src/app.py` — all test endpoints (Flask/FastAPI/Django app) -- `drift/instrumentation/$ARGUMENTS/e2e-tests/src/test_requests.py` — which endpoints are called -- `drift/instrumentation/$ARGUMENTS/e2e-tests/entrypoint.py` — test orchestration and setup +- `drift/instrumentation//e2e-tests/src/app.py` — all test endpoints (Flask/FastAPI/Django app) +- `drift/instrumentation//e2e-tests/src/test_requests.py` — which endpoints are called +- `drift/instrumentation//e2e-tests/entrypoint.py` — test orchestration and setup Understand what functionality is already tested and identify coverage gaps. +- **If focus context provided**: What tests already exist for the focus area? What's missing? + ### 1.3 Analyze the Package Source Code If you cloned the package source, read it to understand: @@ -138,11 +157,14 @@ If you cloned the package source, read it to understand: - The package's entry points and full API surface - Functions that are currently patched vs functions that exist but aren't patched - Alternative call patterns, overloads, and edge cases +- **If focus context provided**: Deep-dive into the focus area's API surface and usage patterns --- ## Phase 2: Identify Potential Gaps +**If focus context was provided**, prioritize bugs related to that area. You may still note other potential issues, but test the focus area first. + Reason about potential issues in the instrumentation. Consider: - **Untested parameters**: Parameter combinations not covered by existing tests @@ -405,10 +427,7 @@ Commit the changes: ```bash git add drift/instrumentation/$ARGUMENTS/e2e-tests/ -git commit -m "bug-hunt($ARGUMENTS): add e2e tests exposing instrumentation bugs - -Found N confirmed bugs in $ARGUMENTS instrumentation. -See BUG_TRACKING.md for details" +git commit -m "bug-hunt($ARGUMENTS): add e2e tests exposing instrumentation bugs" ``` Push the branch (skip if in Claude Code Web where the session handles this): From adeac9a1270adb1fa2c5d8448438ea5fcd6cd4cc Mon Sep 17 00:00:00 2001 From: Sohan Kshirsagar Date: Tue, 24 Mar 2026 16:14:49 -0700 Subject: [PATCH 3/5] fix substitution in bug-hunt slash command --- .claude/commands/bug-hunt.md | 14 +++++++------- docs/environment-variables.md | 8 ++++---- docs/quickstart.md | 2 +- drift/instrumentation/README-e2e-tests.md | 2 +- drift/instrumentation/e2e_common/base_runner.py | 10 +++++----- 5 files changed, 18 insertions(+), 18 deletions(-) diff --git a/.claude/commands/bug-hunt.md b/.claude/commands/bug-hunt.md index fa5b751..7b1e61c 100644 --- a/.claude/commands/bug-hunt.md +++ b/.claude/commands/bug-hunt.md @@ -12,7 +12,7 @@ $ARGUMENTS - The library name, optionally followed by focus context. - `/bug-hunt redis focus on pub sub interactions` — prioritize pub/sub patterns - `/bug-hunt psycopg2 focus on async cursors and connection pooling` — prioritize those areas -**Parsing**: The first word is always the library name. Everything after it is the optional focus context. +**Parsing**: The first word of `$ARGUMENTS` is always the library name. Everything after it is the optional focus context. All references to `` below mean this parsed first word — NOT the raw `$ARGUMENTS` string. ## Library-to-GitHub-Repo Mapping @@ -185,11 +185,11 @@ Produce a prioritized list of potential bugs to investigate. Create `BUG_TRACKING.md` in the e2e test directory: ```bash -# Path: drift/instrumentation/$ARGUMENTS/e2e-tests/BUG_TRACKING.md +# Path: drift/instrumentation//e2e-tests/BUG_TRACKING.md ``` ```markdown -# $ARGUMENTS Instrumentation Bug Tracking +# Instrumentation Bug Tracking Generated: @@ -218,7 +218,7 @@ For each potential bug, follow this workflow: Navigate to the e2e test directory: ```bash -cd drift/instrumentation/$ARGUMENTS/e2e-tests/ +cd drift/instrumentation//e2e-tests/ ``` Build and start Docker containers: @@ -426,14 +426,14 @@ rm -rf /tmp/*-source Commit the changes: ```bash -git add drift/instrumentation/$ARGUMENTS/e2e-tests/ -git commit -m "bug-hunt($ARGUMENTS): add e2e tests exposing instrumentation bugs" +git add drift/instrumentation//e2e-tests/ +git commit -m "bug-hunt(): add e2e tests exposing instrumentation bugs" ``` Push the branch (skip if in Claude Code Web where the session handles this): ```bash -git push origin bug-hunt/$ARGUMENTS-$(date +%Y-%m-%d) +git push origin bug-hunt/-$(date +%Y-%m-%d) ``` --- diff --git a/docs/environment-variables.md b/docs/environment-variables.md index a535a66..7e1e365 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -11,7 +11,7 @@ The `TUSK_DRIFT_MODE` environment variable controls how the SDK operates in your | Mode | Description | When to Use | | ---------- | ---------------------------------------------------- | -------------------------------------------------------------------------------------------- | | `RECORD` | Records traces for all instrumented operations | Set this in environments where you want to capture API traces (e.g., staging, production) | -| `REPLAY` | Replays previously recorded traces | Automatically set by the Tusk CLI when running `tusk run` - you should NOT set this manually | +| `REPLAY` | Replays previously recorded traces | Automatically set by the Tusk CLI when running `tusk drift run` - you should NOT set this manually | | `DISABLED` | Disables all instrumentation and recording | Use when you want to completely disable Tusk with no performance impact | | Unset | Same as `DISABLED` - no instrumentation or recording | Default state when the variable is not set | @@ -25,7 +25,7 @@ The `TUSK_DRIFT_MODE` environment variable controls how the SDK operates in your **Replaying Traces:** -- `TUSK_DRIFT_MODE` is automatically set to `REPLAY` by the Tusk CLI when you run `tusk run` +- `TUSK_DRIFT_MODE` is automatically set to `REPLAY` by the Tusk CLI when you run `tusk drift run` - **Do NOT** manually set `TUSK_DRIFT_MODE=REPLAY` in your application startup commands - The start command specified in your `.tusk/config.yaml` should NOT cause `TUSK_DRIFT_MODE` to be set to anything - the CLI handles this automatically @@ -61,7 +61,7 @@ start_command: "python app.py" # Do NOT include TUSK_DRIFT_MODE here ```bash # The CLI automatically sets TUSK_DRIFT_MODE=REPLAY -tusk run +tusk drift run ``` **Disabling Tusk:** @@ -99,7 +99,7 @@ Your Tusk Drift API key, required when using Tusk Cloud for storing and managing - Can be set as an environment variable: ```bash - TUSK_API_KEY=your-api-key-here tusk run + TUSK_API_KEY=your-api-key-here tusk drift run ``` - Or use the Tusk CLI login command (recommended): diff --git a/docs/quickstart.md b/docs/quickstart.md index 3dd7fd7..e25aff5 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -56,7 +56,7 @@ Need to install the Tusk CLI? See [CLI installation guide](https://github.com/Us Replay the recorded test: ```bash -tusk run +tusk drift run ``` You should see an output similar to: diff --git a/drift/instrumentation/README-e2e-tests.md b/drift/instrumentation/README-e2e-tests.md index 098369a..44544f8 100644 --- a/drift/instrumentation/README-e2e-tests.md +++ b/drift/instrumentation/README-e2e-tests.md @@ -184,7 +184,7 @@ The e2e tests follow a **Docker entrypoint-driven architecture** where the Pytho │ - Stop app, verify traces created │ │ │ │ Phase 3: Run Tests │ -│ - Execute `tusk run` CLI │ +│ - Execute `tusk drift run` CLI │ │ - Parse JSON test results │ │ │ │ Phase 4: Check Instrumentation Warnings│ diff --git a/drift/instrumentation/e2e_common/base_runner.py b/drift/instrumentation/e2e_common/base_runner.py index d13974f..9ed2fab 100644 --- a/drift/instrumentation/e2e_common/base_runner.py +++ b/drift/instrumentation/e2e_common/base_runner.py @@ -296,12 +296,12 @@ def run_tests(self): check=False, ) - # Debug: show what tusk run returned - self.log(f"tusk run exit code: {result.returncode}", Colors.YELLOW) + # Debug: show what tusk drift run returned + self.log(f"tusk drift run exit code: {result.returncode}", Colors.YELLOW) if result.stdout: - self.log(f"tusk run stdout:\n{result.stdout}", Colors.YELLOW) + self.log(f"tusk drift run stdout:\n{result.stdout}", Colors.YELLOW) if result.stderr: - self.log(f"tusk run stderr:\n{result.stderr}", Colors.YELLOW) + self.log(f"tusk drift run stderr:\n{result.stderr}", Colors.YELLOW) # Parse JSON results self.parse_test_results(result.stdout) @@ -429,7 +429,7 @@ def check_socket_instrumentation_warnings(self): else: self.log("✓ No socket instrumentation warnings found.", Colors.GREEN) - # Verify trace files exist (double-check after tusk run) + # Verify trace files exist (double-check after tusk drift run) trace_files = list(traces_dir.glob("*.jsonl")) if traces_dir.exists() else [] if trace_files: self.log(f"✓ Found {len(trace_files)} trace file(s).", Colors.GREEN) From 838ae93b03cce3960d1c60900cc7e5ab5cda4983 Mon Sep 17 00:00:00 2001 From: Sohan Kshirsagar Date: Tue, 24 Mar 2026 16:23:43 -0700 Subject: [PATCH 4/5] convert to skill --- .claude/{commands/bug-hunt.md => skills/bug-hunt/SKILL.md} | 6 ++++++ 1 file changed, 6 insertions(+) rename .claude/{commands/bug-hunt.md => skills/bug-hunt/SKILL.md} (98%) diff --git a/.claude/commands/bug-hunt.md b/.claude/skills/bug-hunt/SKILL.md similarity index 98% rename from .claude/commands/bug-hunt.md rename to .claude/skills/bug-hunt/SKILL.md index 7b1e61c..b516c8a 100644 --- a/.claude/commands/bug-hunt.md +++ b/.claude/skills/bug-hunt/SKILL.md @@ -1,3 +1,9 @@ +--- +name: bug-hunt +description: Hunt for instrumentation bugs by analyzing code gaps and running e2e tests through DISABLED/RECORD/REPLAY cycle +disable-model-invocation: true +--- + # Instrumentation Bug Hunting ## Arguments From 1e3f40d08edf7594d1ea726b414c3b8c1b5713b2 Mon Sep 17 00:00:00 2001 From: Sohan Kshirsagar Date: Tue, 24 Mar 2026 16:30:37 -0700 Subject: [PATCH 5/5] address pr comments --- .claude/skills/bug-hunt/SKILL.md | 2 +- drift/instrumentation/e2e_common/base_runner.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.claude/skills/bug-hunt/SKILL.md b/.claude/skills/bug-hunt/SKILL.md index b516c8a..3768e8b 100644 --- a/.claude/skills/bug-hunt/SKILL.md +++ b/.claude/skills/bug-hunt/SKILL.md @@ -337,7 +337,7 @@ sleep 2 Run the Tusk CLI to replay: ```bash -docker exec -e TUSK_ANALYTICS_DISABLED=1 bug-hunt-app tusk drift run --print --output-format "json" --enable-service-logs +docker exec -e TUSK_ANALYTICS_DISABLED=1 -e TUSK_REQUIRE_INBOUND_REPLAY_SPAN=1 bug-hunt-app tusk drift run --print --output-format "json" --enable-service-logs ``` **Check for issues:** diff --git a/drift/instrumentation/e2e_common/base_runner.py b/drift/instrumentation/e2e_common/base_runner.py index 9ed2fab..4f1b64a 100644 --- a/drift/instrumentation/e2e_common/base_runner.py +++ b/drift/instrumentation/e2e_common/base_runner.py @@ -291,7 +291,7 @@ def run_tests(self): } result = self.run_command( - ["tusk", "run", "--print", "--output-format", "json", "--enable-service-logs"], + ["tusk", "drift", "run", "--print", "--output-format", "json", "--enable-service-logs"], env=env, check=False, )