Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
# Lint + import sorting — blocking. Rules come from pyproject.toml.
# packages/ holds the agami-core library.
- name: ruff check
run: uvx ruff@0.15.19 check plugins packages tests
run: uvx ruff@0.15.19 check plugins packages tests dev.py dev

# Format check is informational for now: the tree has a large unformatted
# backlog (~74 files). Flip `continue-on-error` off after a dedicated
Expand All @@ -49,6 +49,17 @@ jobs:
--with-editable "packages/agami-core[model,server]"
pytest tests/ -q --cov=plugins --cov=packages/agami-core/src --cov-report=term-missing

hygiene:
name: hygiene (conflict markers)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# Its own job rather than a step on `gitleaks`, so a failure here reads as what it
# is instead of arriving under a secret-scan heading. Pre-commit runs the same
# script locally; this is the copy that cannot be skipped with `--no-verify`.
- name: no merge-conflict markers
run: python3 dev/check_conflict_markers.py

gitleaks:
name: gitleaks (secret scan)
runs-on: ubuntu-latest
Expand Down
18 changes: 18 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,24 @@ repos:
hooks:
- id: gitleaks

# Merge-conflict markers — a marker that survives a resolution is invisible to review
# and renders broken in a public repo. Deliberately NOT pre-commit-hooks'
# `check-merge-conflict`: that one only inspects files while a merge is in progress, so a
# marker already committed on a branch you later merge walks straight past it. This scans
# the tracked tree unconditionally. CI re-runs it as the unbypassable gate.
- repo: local
hooks:
- id: conflict-markers
name: no merge-conflict markers
entry: dev/check_conflict_markers.py
# `language: script` runs the file via its shebang and lets pre-commit resolve the
# interpreter per-platform. Neither bare name is portable: `python3` is absent on
# Windows (the official installer ships `python.exe` + the `py` launcher), and
# `python` is absent on Debian/Ubuntu without `python-is-python3`.
language: script
always_run: true
pass_filenames: false

# Test suite — runs at PRE-PUSH (not on every commit) so commits stay fast but nothing
# red leaves your machine. Same deps + command CI uses.
- repo: local
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ demand by `uvx`, the same on macOS / Linux / Windows. From the repo root:

```bash
uv run dev.py setup # once: wire the local pre-commit hooks
uv run dev.py check # ruff + tests + gitleaks — the same checks CI gates on
uv run dev.py check # ruff + tests + gitleaks + conflict markers — the same checks CI gates on
uv run dev.py cover # did the lines I changed get tested? (patch coverage)
```

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ tiny cross-platform task runner (`dev.py`) wraps it all:

```bash
uv run dev.py setup # once: wire the pre-commit hooks (ruff + gitleaks on commit, tests on push)
uv run dev.py check # the whole gate locally — ruff + tests + gitleaks (same as CI)
uv run dev.py check # the whole gate locally — ruff + tests + gitleaks + conflict markers (same as CI)
uv run dev.py cover # did the lines I changed get tested? (patch coverage)
```

Expand Down
14 changes: 11 additions & 3 deletions dev.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

Tasks:
setup wire the local pre-commit hooks (ruff + gitleaks on commit; tests on push)
check the full local gate: ruff lint + tests + gitleaks (what CI runs)
check the full local gate: ruff lint + tests + gitleaks + conflict markers (what CI runs)
test just the test suite
lint just ruff (lint + format check)
fmt apply ruff's auto-formatter to the tree
Expand All @@ -30,7 +30,9 @@
# The suite imports the agami-core library, so install it editable with the [model]
# extra (pydantic/pyyaml/sqlglot). DB drivers are omitted on purpose — those tests skip without a DB.
TEST_DEPS = ["--with", "pytest-cov", "--with-editable", "packages/agami-core[model,server]"]
TARGETS = ["plugins", "packages", "tests", "dev.py"]
# `dev` (the directory) as well as `dev.py` (the file) — the helper scripts under dev/ were
# unlinted until the conflict-marker gate landed there.
TARGETS = ["plugins", "packages", "tests", "dev.py", "dev"]

_ROOT = Path(__file__).resolve().parent
# The plugin's runtime scripts import a small stdlib-only slice of the agami-core library. The
Expand Down Expand Up @@ -73,6 +75,11 @@ def secrets() -> int:
return run(["uvx", "pre-commit", "run", "gitleaks", "--all-files"])


def conflicts() -> int:
"""Fail on a merge-conflict marker that survived into a tracked file (CI's `hygiene` job)."""
return run([sys.executable, str(_ROOT / "dev" / "check_conflict_markers.py")])


def sync_lib() -> int:
"""Regenerate plugins/agami/lib/ from packages/agami-core/src (the vendored closure the scripts import)."""
for rel in _VENDORED:
Expand All @@ -98,10 +105,11 @@ def _lib_drift() -> int:


def check() -> int:
"""ruff lint + tests + gitleaks + the vendored-lib drift check — the same checks CI gates on."""
"""ruff + tests + gitleaks + conflict markers + vendored-lib drift — the same checks CI gates on."""
rc = lint()
rc |= test()
rc |= secrets()
rc |= conflicts()
rc |= _lib_drift()
print("\n✓ all checks passed" if rc == 0 else "\n✗ some checks failed")
return rc
Expand Down
158 changes: 158 additions & 0 deletions dev/check_conflict_markers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""Fail if a merge-conflict marker survived into a tracked file.

Why this exists as its own check rather than pre-commit's `check-merge-conflict`:
that hook only inspects files **while a merge is in progress**, so a marker that is
committed and only noticed later — or one that arrives on a branch you merged — walks
straight past it. This runs over the tracked tree unconditionally, in CI, where it
cannot be skipped with `--no-verify`.

The matching rule is deliberately asymmetric, because two of the four markers a
conflict can leave behind are ambiguous:

open / close (a `<` or `>` run) never occur in legitimate content -> always an error.
base-with-label (`|` run + text) likewise -> always an error.
a bare `=` run is also a Markdown setext heading underline (`Title`
on one line, the rule under it), so flagging it
unconditionally would false-positive on prose.
a bare `|` run is also a degenerate Markdown table row of empty cells.

The two bare forms are reported only when the same file also carries an unambiguous
marker. That asymmetry is the point: a real conflict always leaves at least one
unambiguous marker, so the ambiguous ones become *attributable* rather than guessed at.

Three states, never two: markers -> fail; clean -> pass; unreadable -> fail loudly.
A file we could not read is *unknown*, not clean, and a gate must never collapse those
two into the same answer.
"""

from __future__ import annotations

import os
import re
import subprocess
import sys

# Marker runs are 7 characters by default, but `.gitattributes` can widen them via
# `conflict-marker-size`, so match a run of 7-or-more. The trailing label ("HEAD",
# "origin/main") is optional: git always writes one, hand-resolved conflicts often don't.
UNAMBIGUOUS = (
re.compile(rb"^<{7,}(?: .*)?$"), # ours
re.compile(rb"^>{7,}(?: .*)?$"), # theirs
re.compile(rb"^\|{7,} .+$"), # diff3/zdiff3 common ancestor, WITH a label
)
AMBIGUOUS = (
re.compile(rb"^={7,}$"), # also a Markdown setext heading underline
re.compile(rb"^\|{7,}$"), # also a Markdown table row of empty cells
)

# Suffixes we skip outright. This is a fast path, NOT the binary defence — the
# authoritative test is the NUL-byte probe in `scan()`, so an unlisted binary type is
# still handled correctly instead of being silently mis-scanned.
SKIP_SUFFIXES = (
".png", ".jpg", ".jpeg", ".gif", ".ico", ".bmp", ".webp", ".pdf",
".woff", ".woff2", ".ttf", ".otf", ".eot", ".zip", ".gz", ".bz2", ".xz",
".7z", ".jar", ".class", ".so", ".dylib", ".dll", ".exe", ".pyc", ".whl",
".parquet", ".xlsx", ".docx", ".pptx", ".db", ".sqlite", ".mp4", ".mov",
)

BINARY_SNIFF_BYTES = 8192


def _git_bytes(*args: str) -> bytes:
"""Run a git command, surfacing git's own message rather than a bare exit code."""
proc = subprocess.run(["git", *args], capture_output=True)
if proc.returncode != 0:
detail = proc.stderr.decode("utf-8", "replace").strip() or f"exit {proc.returncode}"
sys.exit(f"✗ git {' '.join(args)} failed: {detail}")
return proc.stdout


def repo_root() -> str:
"""The repo's top level, so a run from a subdirectory still scans the whole tree.

`git ls-files` is cwd-relative: invoked from a clean subdirectory it lists only that
subtree, and the gate would print a reassuring pass having scanned almost nothing.
"""
return _git_bytes("rev-parse", "--show-toplevel").decode("utf-8", "replace").strip()


def tracked_files() -> list[str]:
"""Every file git tracks, so an untracked scratch file cannot fail the build.

`-z` is load-bearing: without it `core.quotePath` (on by default) C-quotes any path
holding a non-ASCII byte, a control character, a quote or a backslash — turning
`café.py` into the literal `"caf\\303\\251.py"`, which is not an openable path. The
file would then be skipped silently, and a non-ASCII *directory* would hide its
entire subtree.
"""
raw = _git_bytes("ls-files", "-z")
paths = [os.fsdecode(p) for p in raw.split(b"\0") if p]
# An unmerged index lists a path once per stage; collapse while preserving order.
unique = list(dict.fromkeys(paths))
return [p for p in unique if not p.lower().endswith(SKIP_SUFFIXES)]


def scan(path: str) -> tuple[list[tuple[int, str]], str | None]:
"""Return (findings, error) for one file — exactly one of the two is meaningful.

Matching runs on BYTES: the markers are pure ASCII, so decoding buys nothing and
costs correctness. A single latin-1 or UTF-16 byte anywhere in an otherwise ordinary
text file makes a whole-file strict decode raise, which would discard the scan.
"""
try:
with open(path, "rb") as fh:
data = fh.read()
except OSError as exc:
return [], f"{type(exc).__name__}: {exc.strerror or exc}"
if b"\0" in data[:BINARY_SNIFF_BYTES]:
return [], None # genuinely binary — no text to match
lines = [line.rstrip(b"\r") for line in data.split(b"\n")]
hard = [
(n, line) for n, line in enumerate(lines, 1) if any(p.match(line) for p in UNAMBIGUOUS)
]
if not hard:
# No unambiguous marker, so a bare `=` run here is a setext heading, not a conflict.
return [], None
soft = [(n, line) for n, line in enumerate(lines, 1) if any(p.match(line) for p in AMBIGUOUS)]
return [(n, line.decode("utf-8", "replace")) for n, line in sorted(hard + soft)], None


def main() -> int:
os.chdir(repo_root())
paths = tracked_files()
if not paths:
print("✗ no tracked files to scan — refusing to report a pass", file=sys.stderr)
return 1

findings: list[tuple[str, int, str]] = []
unreadable: list[tuple[str, str]] = []
for path in paths:
hits, error = scan(path)
if error:
unreadable.append((path, error))
findings.extend((path, n, line) for n, line in hits)

if not findings and not unreadable:
print(f"✓ no merge-conflict markers ({len(paths)} tracked files scanned)")
return 0
Comment on lines +121 to +138

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added — tests/test_conflict_marker_gate.py, 24 cases, all green.

It covers the asymmetric rule you name, including the case that turned out to matter most: the bare separator is scoped per file, so a conflict in a.py must not indict a setext heading in b.md. Worth noting no tracked file in this repo currently contains a line of only =, so deleting that guard would have kept CI green until someone added a doc with a setext heading, at which point the gate fails the whole repo on innocent prose.

Note the bootstrap problem: a fixture with markers at column 0 would be caught by this very gate. The tests build markers at runtime ("<" * 7) and drive the script as a subprocess against a throwaway git repo, so nothing is written that the gate would later scan.


if findings:
print("✗ merge-conflict markers found in tracked files:\n", file=sys.stderr)
for path, n, line in findings:
print(f" {path}:{n}: {line[:80]}", file=sys.stderr)
print(
"\nResolve the conflict and remove every marker before committing"
" — a diff3/zdiff3 conflict leaves four, not three.",
file=sys.stderr,
)
if unreadable:
# Unknown, not clean. Fail rather than certify a file we never actually read.
print("\n✗ tracked files could not be read, so they were not checked:\n", file=sys.stderr)
for path, error in unreadable:
print(f" {path}: {error}", file=sys.stderr)
return 1


if __name__ == "__main__":
raise SystemExit(main())
Loading
Loading