Release 0.6.2 — probe-adapter + Copilot profile#6
Conversation
Bundle a copilot.toml profile for the GitHub Copilot CLI (GA since 2026-02): -i interactive launch, --allow-all-tools/--allow-all-paths bypass, prompt loads SKILL.md directly (no native skill discovery), skills shared in .agents/skills with codex/gemini. Add the copilot-settings-json hook dialect: handlers are stored directly in the event list (no 'hooks' wrapper), configs are versioned, and timeouts are seconds. merge_hooks now dedupes both the wrapped and bare handler shapes so re-runs stay idempotent across all dialects. Hooks register under VS Code-compatible PascalCase names so Copilot emits the snake_case payloads the shared relay reads. Marked pending live E2E verification in docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…data Add `bmad-auto probe-adapter` (alias `collect-adapter-data`): a self-service command that pulls and sanitizes everything a maintainer needs to finalize a generic-adapter CLI profile — hook payload shape, transcript location/format, and the token-usage schema for a usage_parser. - default zero-launch SCAN; opt-in `--probe` live capture in an ephemeral mkdtemp workspace (rmtree'd in a finally) - single audited PII chokepoint (sanitize.py): counts/keys pass through, leaf strings kept only if identifier-shaped, paths/prose/emails redacted - packaged full-payload capture hook gated on BMAD_AUTO_PROBE_CAPTURE_DIR - registers via the existing install.merge_hooks (broadened HOOK_MARKER to match either bmad-auto hook script, keeping merges idempotent) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- new docs/adapter-authoring-guide.md: finalizing a CLI profile with probe-adapter (scan vs probe, PII model, walkthrough, copilot example) - add probe-adapter to the README + FEATURES command references - add the missing copilot row to the README profile table - link the guide from docs/README.md and CONTRIBUTING.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WalkthroughAdds Changesprobe-adapter Command + Copilot Profile Release
Sequence Diagram(s)sequenceDiagram
actor User
participant cli as cmd_probe (cli.py)
participant probe as probe.py
participant sanitize as sanitize.py
participant tmux
participant ProbeHook as bmad_auto_probe_hook.py
User->>cli: bmad-auto probe-adapter <cli> [--probe]
cli->>probe: scan(cli, profile, hints)
probe->>probe: run_version_help → FlagFinding
probe->>probe: discover_transcript → TranscriptFinding
probe->>probe: infer_token_schema → TokenSchema
probe-->>cli: ProfileFinding (SCAN)
User->>cli: bmad-auto probe-adapter <cli> --probe
cli->>probe: probe(cli, profile, hints)
probe->>tmux: new-session, set BMAD_AUTO_PROBE_CAPTURE_DIR
tmux->>ProbeHook: Stop / SessionStart events fired
ProbeHook->>ProbeHook: atomic write *.signal.json + *.payload.json
probe->>probe: _collect_captures → EventCapture[]
probe->>sanitize: scrub_event_payload(raw_payload)
sanitize-->>probe: scrubbed payload
probe->>probe: discover_transcript + infer_token_schema
probe->>tmux: kill-session, rm temp dir
probe-->>cli: ProfileFinding (PROBE)
cli->>User: Markdown report (+ optional JSON block)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🤖 Augment PR SummarySummary: This PR releases Changes:
Technical Notes: Probe mode uses an ephemeral workspace + tmux launch to capture real hook payloads; scan mode avoids launching the CLI and relies on on-disk conventions. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
| hook_src = resources.files("automator.data").joinpath(PROBE_HOOK_NAME) | ||
| hook_path = tmpdir / PROBE_HOOK_NAME | ||
| hook_path.write_text(hook_src.read_text(encoding="utf-8"), encoding="utf-8") | ||
| registrations = { |
There was a problem hiding this comment.
src/automator/probe.py:525 builds registrations so the capture hook receives {canonical} as argv, but later code treats argv_event as the native event key (breaking non-identity maps like Gemini AfterAgent → Stop and yielding ?/mislabeling in the report). Other locations where this applies: src/automator/data/bmad_auto_probe_hook.py:44.
Severity: medium
Other Locations
src/automator/data/bmad_auto_probe_hook.py:44
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| missing = "tmux" if not shutil.which("tmux") else binary | ||
| finding.warnings.append(f"{missing} not on PATH — cannot probe; falling back to scan") | ||
| scanned = scan(cli=cli, profile=profile, project=project, hints=hints) | ||
| scanned.mode = "probe" |
There was a problem hiding this comment.
src/automator/probe.py:511 falls back to scan() when tmux/binary is missing but then forces mode = "probe", which makes render_markdown() take the probe branch and hide declared events / show “no hook payloads captured”. This can confuse users who actually got a scan result due to missing prerequisites.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| """ | ||
| paths: set[str] = set() | ||
| scanned = 0 | ||
| for entry in _jsonl_entries(path): |
There was a problem hiding this comment.
src/automator/probe.py:307 iterates transcripts via _jsonl_entries(path) only, but discover_transcript() can select a *.json transcript (e.g., via --session-dir fallback), which will silently scan 0 entries and omit schema/candidates. That makes the “token usage schema” section inaccurate for JSON transcripts.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
|
||
| # CLI flags | ||
| out.append("## CLI flags") | ||
| out.append(_fmt_kv("launch_args / bypass_args", "see profile (rendered verbatim below)")) |
There was a problem hiding this comment.
src/automator/probe.py:649 says launch/bypass args are “rendered verbatim below”, but the report never actually prints the profile’s launch_args / bypass_args. This mismatch can mislead users reading the report output.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
|
||
| if args.out: | ||
| out_path = Path(args.out) | ||
| out_path.write_text(report, encoding="utf-8") |
There was a problem hiding this comment.
src/automator/cli.py:930 writes the --out report with Path.write_text() but doesn’t handle OSError (missing parent dir, permissions, etc.), so probe-adapter may crash with a traceback instead of returning a clean failure. That’s especially painful since the report content is otherwise already assembled and paste-safe.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/automator/cli.py`:
- Around line 903-912: The success message printed on line 907 (ok: unknown
profile...) will be displayed even when --probe is subsequently used and will
fail, creating contradictory output. Gate this print statement to only execute
when args.probe is False, so the "ok" message is not printed in scenarios where
the --probe check will immediately fail due to the unknown profile. This keeps
the CLI output consistent by avoiding success messages that are followed by
guaranteed failure.
In `@src/automator/install.py`:
- Around line 31-35: The HOOK_MARKER constant uses a broad substring match of
"bmad_auto" which can incorrectly identify unrelated commands as already-managed
hooks. Replace the substring-based marker approach with explicit matching for
the exact managed hook script basenames: bmad_auto_hook.py and
bmad_auto_probe_hook.py. Update all locations where HOOK_MARKER is used
(including lines 81-85 referenced in the comment) to check for these specific
script names instead of a substring match, ensuring the deduplication logic only
applies to the actual managed hook scripts and remains idempotent.
In `@src/automator/probe.py`:
- Around line 525-528: The registrations dictionary is passing the canonical
event name in the probe hook command string, but _collect_captures() expects to
receive the native event name as argv_event to perform the canonical lookup via
events_map.get(native). In the f-string that builds the command for each
registration entry, replace the use of canonical with native as the argument
passed to the hook_path command.
- Around line 602-605: The warning message being appended to finding.warnings in
this code block directly embeds the raw tmpdir path without sanitization, which
can leak sensitive filesystem information. Apply the appropriate path
sanitization function to the tmpdir variable before including it in the f-string
message passed to the append method. This ensures the warning message follows
the same sanitization contract as other parts of the report.
- Around line 507-512: When the tmux or binary check fails in the conditional
block starting with if not shutil.which("tmux"), a warning is appended to the
finding object but then discarded because the function returns the scanned
object instead. To preserve the downgrade reason, transfer the warning message
from finding.warnings to scanned.warnings before returning scanned, ensuring
users see why probe mode fell back to scan mode.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d5ada323-5d87-4db6-9c40-58eff1f438fb
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (22)
.claude-plugin/marketplace.jsonCHANGELOG.mdCONTRIBUTING.mdREADME.mddocs/FEATURES.mddocs/README.mddocs/adapter-authoring-guide.mdmodule.yamlpyproject.tomlsrc/automator/__init__.pysrc/automator/adapters/profile.pysrc/automator/cli.pysrc/automator/data/bmad_auto_probe_hook.pysrc/automator/data/profiles/copilot.tomlsrc/automator/data/skills/bmad-auto-setup/assets/module.yamlsrc/automator/install.pysrc/automator/probe.pysrc/automator/sanitize.pytests/test_install.pytests/test_probe.pytests/test_probe_hook.pytests/test_sanitize.py
| except ProfileError as e: | ||
| if not args.binary: | ||
| print(f"FAIL: {e}", file=sys.stderr) | ||
| return 1 | ||
| print(f" ok: unknown profile {args.cli!r}; reduced report from --binary {args.binary}") | ||
|
|
||
| if args.probe: | ||
| if profile is None: | ||
| print("FAIL: --probe needs a known profile (its hook dialect/events)", file=sys.stderr) | ||
| return 1 |
There was a problem hiding this comment.
Avoid printing a success message right before guaranteed --probe failure.
With unknown profile + --binary + --probe, Line 907 prints an “ok” message before Line 911 fails. Gate that message to non-probe mode to keep CLI output consistent.
Proposed fix
except ProfileError as e:
if not args.binary:
print(f"FAIL: {e}", file=sys.stderr)
return 1
- print(f" ok: unknown profile {args.cli!r}; reduced report from --binary {args.binary}")
+ if not args.probe:
+ print(f" ok: unknown profile {args.cli!r}; reduced report from --binary {args.binary}")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| except ProfileError as e: | |
| if not args.binary: | |
| print(f"FAIL: {e}", file=sys.stderr) | |
| return 1 | |
| print(f" ok: unknown profile {args.cli!r}; reduced report from --binary {args.binary}") | |
| if args.probe: | |
| if profile is None: | |
| print("FAIL: --probe needs a known profile (its hook dialect/events)", file=sys.stderr) | |
| return 1 | |
| except ProfileError as e: | |
| if not args.binary: | |
| print(f"FAIL: {e}", file=sys.stderr) | |
| return 1 | |
| if not args.probe: | |
| print(f" ok: unknown profile {args.cli!r}; reduced report from --binary {args.binary}") | |
| if args.probe: | |
| if profile is None: | |
| print("FAIL: --probe needs a known profile (its hook dialect/events)", file=sys.stderr) | |
| return 1 |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/automator/cli.py` around lines 903 - 912, The success message printed on
line 907 (ok: unknown profile...) will be displayed even when --probe is
subsequently used and will fail, creating contradictory output. Gate this print
statement to only execute when args.probe is False, so the "ok" message is not
printed in scenarios where the --probe check will immediately fail due to the
unknown profile. This keeps the CLI output consistent by avoiding success
messages that are followed by guaranteed failure.
| # Dedup marker: matches any bmad-auto-managed hook command — both the signal | ||
| # relay (bmad_auto_hook.py) and the probe-adapter capture hook | ||
| # (bmad_auto_probe_hook.py) — so merge_hooks stays idempotent for either. | ||
| HOOK_MARKER = "bmad_auto" | ||
| GEMINI_HOOK_TIMEOUT_MS = 60_000 |
There was a problem hiding this comment.
Narrow the dedupe marker to exact managed hook script names.
The current substring marker is too broad and can treat unrelated commands as already-managed, which can skip required hook registration for an event. Match explicit script basenames (relay/probe) instead of any bmad_auto substring.
Suggested fix
-HOOK_MARKER = "bmad_auto"
+HOOK_MARKERS = ("bmad_auto_hook.py", "bmad_auto_probe_hook.py")
@@
- already = any(
- HOOK_MARKER in handler.get("command", "")
+ already = any(
+ any(marker in handler.get("command", "") for marker in HOOK_MARKERS)
for entry in matchers
if isinstance(entry, dict)
for handler in (entry, *entry.get("hooks", []))
if isinstance(handler, dict)
)Also applies to: 81-85
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/automator/install.py` around lines 31 - 35, The HOOK_MARKER constant uses
a broad substring match of "bmad_auto" which can incorrectly identify unrelated
commands as already-managed hooks. Replace the substring-based marker approach
with explicit matching for the exact managed hook script basenames:
bmad_auto_hook.py and bmad_auto_probe_hook.py. Update all locations where
HOOK_MARKER is used (including lines 81-85 referenced in the comment) to check
for these specific script names instead of a substring match, ensuring the
deduplication logic only applies to the actual managed hook scripts and remains
idempotent.
| if not shutil.which("tmux") or not shutil.which(binary): | ||
| missing = "tmux" if not shutil.which("tmux") else binary | ||
| finding.warnings.append(f"{missing} not on PATH — cannot probe; falling back to scan") | ||
| scanned = scan(cli=cli, profile=profile, project=project, hints=hints) | ||
| scanned.mode = "probe" | ||
| return scanned |
There was a problem hiding this comment.
Preserve the downgrade reason when --probe falls back to scan.
The tmux/binary-missing warning is appended to finding and then discarded by returning scanned, so users can lose the actual reason probe mode didn’t run.
Proposed fix
if not shutil.which("tmux") or not shutil.which(binary):
missing = "tmux" if not shutil.which("tmux") else binary
- finding.warnings.append(f"{missing} not on PATH — cannot probe; falling back to scan")
+ fallback_warning = f"{missing} not on PATH — cannot probe; falling back to scan"
scanned = scan(cli=cli, profile=profile, project=project, hints=hints)
scanned.mode = "probe"
+ scanned.warnings.insert(0, fallback_warning)
return scanned📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if not shutil.which("tmux") or not shutil.which(binary): | |
| missing = "tmux" if not shutil.which("tmux") else binary | |
| finding.warnings.append(f"{missing} not on PATH — cannot probe; falling back to scan") | |
| scanned = scan(cli=cli, profile=profile, project=project, hints=hints) | |
| scanned.mode = "probe" | |
| return scanned | |
| if not shutil.which("tmux") or not shutil.which(binary): | |
| missing = "tmux" if not shutil.which("tmux") else binary | |
| fallback_warning = f"{missing} not on PATH — cannot probe; falling back to scan" | |
| scanned = scan(cli=cli, profile=profile, project=project, hints=hints) | |
| scanned.mode = "probe" | |
| scanned.warnings.insert(0, fallback_warning) | |
| return scanned |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/automator/probe.py` around lines 507 - 512, When the tmux or binary check
fails in the conditional block starting with if not shutil.which("tmux"), a
warning is appended to the finding object but then discarded because the
function returns the scanned object instead. To preserve the downgrade reason,
transfer the warning message from finding.warnings to scanned.warnings before
returning scanned, ensuring users see why probe mode fell back to scan mode.
| registrations = { | ||
| native: f"python3 {shlex.quote(str(hook_path))} {canonical}" | ||
| for native, canonical in profile.hooks.events.items() | ||
| } |
There was a problem hiding this comment.
Pass the native event name into the probe hook command.
_collect_captures() treats argv_event as the native key and resolves canonical via events_map.get(native), but registration currently injects canonical into argv. This breaks native→canonical pairing when names differ.
Proposed fix
- registrations = {
- native: f"python3 {shlex.quote(str(hook_path))} {canonical}"
- for native, canonical in profile.hooks.events.items()
- }
+ registrations = {
+ native: f"python3 {shlex.quote(str(hook_path))} {shlex.quote(native)}"
+ for native in profile.hooks.events
+ }Also applies to: 472-477
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/automator/probe.py` around lines 525 - 528, The registrations dictionary
is passing the canonical event name in the probe hook command string, but
_collect_captures() expects to receive the native event name as argv_event to
perform the canonical lookup via events_map.get(native). In the f-string that
builds the command for each registration entry, replace the use of canonical
with native as the argument passed to the hook_path command.
| finding.warnings.append( | ||
| f"--keep-temp: RAW probe data retained at {tmpdir} — DO NOT SHARE; " | ||
| "delete it after inspection" | ||
| ) |
There was a problem hiding this comment.
Sanitize the --keep-temp path before adding it to warnings.
This warning currently embeds a raw filesystem path in the report, bypassing the sanitizer contract and potentially leaking home-directory data.
Proposed fix
if keep_temp:
+ safe_tmpdir = sanitize.redact_home(str(tmpdir))
finding.warnings.append(
- f"--keep-temp: RAW probe data retained at {tmpdir} — DO NOT SHARE; "
+ f"--keep-temp: RAW probe data retained at {safe_tmpdir} — DO NOT SHARE; "
"delete it after inspection"
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| finding.warnings.append( | |
| f"--keep-temp: RAW probe data retained at {tmpdir} — DO NOT SHARE; " | |
| "delete it after inspection" | |
| ) | |
| if keep_temp: | |
| safe_tmpdir = sanitize.redact_home(str(tmpdir)) | |
| finding.warnings.append( | |
| f"--keep-temp: RAW probe data retained at {safe_tmpdir} — DO NOT SHARE; " | |
| "delete it after inspection" | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/automator/probe.py` around lines 602 - 605, The warning message being
appended to finding.warnings in this code block directly embeds the raw tmpdir
path without sanitization, which can leak sensitive filesystem information.
Apply the appropriate path sanitization function to the tmpdir variable before
including it in the f-string message passed to the append method. This ensures
the warning message follows the same sanitization contract as other parts of the
report.
1 participant
0.6.2
Added
bmad-auto probe-adapter(aliascollect-adapter-data) — self-service command that collects + sanitizes everything needed to finalize a CLI adapter profile (hook payload shape, transcript location/format, token-usage schema) through one audited PII sanitizer. Default zero-launch scan; opt-in--probelive capture.copilotprofile (Copilot CLI ≥ 2026-02):-ilaunch, VS Code-compatibleStophook,--allow-all-tools. Pending live E2E and ausage_parser.Docs
probe-adapteradded to both command references.Release mechanics
uv.lock; CHANGELOG curated. TUI unchanged → screenshots/demo not regenerated. On merge tomain, the Release workflow auto-creates thev0.6.2tag + GitHub release.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
bmad-auto probe-adaptercommand to collect and sanitize CLI adapter profile data, with optional live probe mode for capturing hook behaviorDocumentation