Commit 0ff99fd
committed
feat(astgrep): binary auto-provisioning Phase 1 — download + SHA-256 verify + cache (closes #68 Phase 1)
Adds scripts/astgrep_runner.py — the binary lifecycle layer for ast-grep
(sg), an optional accelerator for rule pattern matching. Phase 1 of
issue #68.
## Contract
1. detect_platform() — normalize host machine+os into a release id
(x86_64-linux, aarch64-linux, x86_64-darwin, arm64-darwin,
x86_64-windows). Returns None for unsupported platforms.
2. provision() — download the right ast-grep binary from GitHub
releases, verify its SHA-256 against the hard-coded expected hash,
cache at ~/.codelens/ast-grep/<version>/<platform>/. Idempotent —
cache hit is a no-op. force=True re-downloads.
3. get_astgrep_path() — public API: returns the absolute path to the
cached binary, or None. NEVER raises — network failure, SHA
mismatch, unsupported platform all return None so callers fall
back to the native Semgrep-YAML matcher (rule_matcher.match_source).
4. is_available() — quick boolean check that does NOT trigger a
download (used by `codelens doctor` for status reporting).
## Safety guards
- **SHA-256 verification is mandatory.** If the downloaded binary's
hash does not match the expected value, the file is deleted and
provision() returns ok=False. The tampered binary is NEVER executed.
- **Placeholder hashes refuse to provision.** Until ASTGREP_RELEASES
is populated with real SHA-256 hashes captured from the GitHub
release page, ast-grep is treated as unavailable. This is a
deliberate safety guard: cannot verify SHA without the real hash.
- **tarfile.extract uses filter="data"** (PEP 706) to prevent path-
traversal attacks from malicious tar archives.
## What's NOT in Phase 1
- Rule format bridge (ast-grep YAML rule parsing) — Phase 2
- Rule pack porting from UBS — Phase 3
- Integration with rule_matcher.match_source (callsite: when
get_astgrep_path() returns non-None, route certain patterns to sg
for ~3x speedup; fall back to native matcher otherwise) — Phase 2
The module is purely additive: no existing file is modified. When
get_astgrep_path() returns None (current state — placeholder hashes),
all existing CodeLens behavior is unchanged.
## Tests (tests/test_astgrep_runner.py — 31 tests)
- Platform detection (5 tests): known/unknown platforms, machine
normalization (amd64→x86_64, aarch64)
- SHA-256 (3 tests): matches hashlib, empty file, 200KB streamed
- Cache layout (3 tests): directory structure, sg vs sg.exe naming
- Graceful fallback (7 tests): unsupported platform, placeholder hash,
network failure (URLError), SHA mismatch deletes binary, happy path,
cache idempotency, force=True re-downloads
- Public API (6 tests): get_astgrep_path/is_available never raise
- Extract helpers (3 tests): tar.gz + zip + missing binary
- Module invariants (4 tests): ASTGREP_RELEASES structure, version
string, cache root under ~/.codelens/
Tests use monkeypatch + mock to avoid hitting the network. The
download path is exercised only via mocked _download, so tests run
in CI / sandboxes without internet access.
Run: python -m pytest tests/test_astgrep_runner.py -v
Result: 31 passed in 0.14s
Closes #68 (Phase 1 only — Phase 2 + 3 are follow-up work).1 parent a875428 commit 0ff99fd
2 files changed
Lines changed: 1080 additions & 0 deletions
0 commit comments