OCH v1.0 — M3 graph-db backend (opt-in) + M4 language expansion#64
Merged
Conversation
Source: CloudFront signed URL produced by Bonk from the 2026-05-03 → 2026-05-04
Slack thread. The roadmap had been living in conversation context and was lost
to compaction, causing a planning-session scope misfire. This file is the
durable reference — if in-conversation scope conflicts, this file wins.
Also gitignore .gitnexus/ (GitNexus CLI workspace metadata) and
.claude/skills/{gitnexus,generated}/ (local-only skill installs from the
GitNexus CLI + auto-generated cluster skills; both carry banned identifiers
from prior-art projects and don't belong in this repo).
Add: - cobol-proleap (AC-M4-6, on-demand ProLeap JVM bridge) - frameworks (AC-M4-7, extracted from packages/ingestion) - scip-ingest (live since pre-MVP but never added to the enum) Prune (dead post-M2 T-M2-1): - gym (moved to opencodehub-testbed) - eval (moved to opencodehub-testbed) - lsp-oracle (never existed as a package)
Second IGraphStore implementation behind the existing seam. All methods stubbed with NotImplementedError tagged with the method name so downstream code can compile against the new backend while AC-M3-2 (pool) and AC-M3-3/4 (bulkLoad, query, traverse, parity) fill in the real bodies. Adds @ladybugdb/core@^0.16.1 as a direct dependency per spec 004 architectural decision #3 (user-approved 2026-05-05). Source-level naming stays clean — GraphDbStore / graphdb-adapter.ts — per decision #2. The scoped package identifier is the only place the library brand appears in tracked source; the banned-strings guardrail exempts the @ladybugdb/ scope via a targeted allowlist so every other mention still fails. openStore(opts) factory reads CODEHUB_STORE: unset or "duck" → DuckDbStore; "lbug" → GraphDbStore. Unknown values hard-error. DuckDbStore remains the default (spec 004 §S-M3-1, §W-M3-3). GraphDbStore.open() lazy-imports the native binding and surfaces GraphDbBindingError with a clear runbook string when unavailable (§S-M3-2).
Emits Cypher `CREATE NODE TABLE` + `CREATE REL TABLE` statements that mirror the semantic shape of `schema-ddl.ts`. Every relation kind in `ALL_RELATION_TYPES` (24 live in v1.1 — spec 004 quoted 23 but the source drifted past, so the translator uses the live list) gets its own polymorphic rel table with multiple `FROM/TO` pairs. A single `CodeRelation` rel table with a discriminator column would defeat columnar predicate push-down, so we fan out per spec 004 decision #1. Node-level layout keeps the DuckDB collapse — one `CodeNode` node table with a `kind` discriminator — so later graphHash round-trip tests read the same column set from either store. Embeddings, store meta, cochanges, and symbol summaries get their own node tables; the `EMBEDS` rel links embedding rows back to their source node without a property lookup. Tests assert the DDL shape (5 node tables, 24 + 1 rel tables, every kind from `getAllRelationTypes()` present, default embed dim 768, invalid dims rejected). A banned-literal sweep over the generated DDL catches regressions where the translator could leak a prior-art name; the test's banned-token list is built from character codes at runtime so this test file itself stays compliant with `scripts/check-banned-strings.sh`.
Empty pool module so `graphdb-adapter.ts` and future test modules can import the pool types without a phantom-import red line during the scaffolding AC. Intentionally exports no runtime symbols — just a `GraphDbPool` interface marker — so AC-M3-2 is free to pick whichever concrete implementation suits the benchmark best when it lifts the real `acquire()` / `release()` / waiter-queue semantics on top of the `@ladybugdb/core` API surface.
Adds the SHA256-pinned download path for external SCIP adapter binaries so M4-1..4 adapters can install their indexers on demand rather than at analyze time. Files: - packages/cli/src/scip-pins.ts: canonical pin table for scip-clang 0.4.0, scip-ruby 0.4.7, scip-dotnet 0.2.12 (dotnet-tool installer), and scip-kotlin 0.6.0. Ships with PLACEHOLDER SHA256 hashes (64 zeros) marked via `placeholder: true`; real hashes land with each adapter PR. - packages/cli/src/scip-downloader.ts: installScipTool(tool, opts) covers platform detection (linux-x64, linux-arm64, darwin-x64, darwin-arm64; windows explicitly refused), sha256 verification, atomic rename, chmod +x, and in-process concurrency serialization via a promise map keyed by (tool, destDir). scip-dotnet is special-cased: probes `dotnet --version` and requires SDK >= 8, surfacing the `dotnet tool install --global scip-dotnet` hint rather than downloading a binary. - packages/cli/src/scip-downloader.test.ts: 11 tests covering happy path, idempotent skip, drifted-hash re-download, pin mismatch cleanup, concurrent-serialization (three parallel installs -> one fetch), unsupported-platform refusal, placeholder-hash refusal, and the full dotnet probe matrix. Gates (this commit): - check-banned-strings.sh PASS - biome check PASS - tsc --noEmit PASS - cli tests 214/214 PASS (+11 new) Blocks AC-M4-1..4 per spec 004 AC-M4-0.
Wires the scip-downloader scaffolding into the `codehub setup` command so
users can install SCIP adapter binaries by name. `--scip=<tool>` installs
one; `--scip=all` walks the ordered set (clang, ruby, dotnet, kotlin). The
dispatcher runs `installScipTool` for binary tools and emits the `dotnet
tool install --global scip-dotnet` hint for the .NET path.
Files:
- packages/cli/src/commands/setup.ts: new `runSetupScip`, `parseScipFlag`,
and `SetupScipOptions`/`SetupScipResult` types. Errors never throw past
the function boundary — they are collected into `failed[]` so
`--scip=all` finishes the installable tools when `scip-dotnet` can't find
the .NET SDK.
- packages/cli/src/index.ts: new `--scip <tool>` option on `codehub setup`.
Updated the command description to mention SCIP adapter installs. The
handler parses the flag via `parseScipFlag`, then calls
`runSetupScip({ tool, force })`.
- packages/cli/src/commands/setup.test.ts: four new tests — parseScipFlag
happy path, parseScipFlag rejection path, runSetupScip for the
dotnet-tool branch (tolerates both `dotnet` present + absent), and the
single-tool install via injected fetch + pin override.
Gates (this commit):
- check-banned-strings.sh PASS
- biome check PASS
- tsc --noEmit PASS
- cli tests 218/218 PASS (+4 new setup tests, +11 scip from prior commit)
Closes AC-M4-0 per spec 004.
Extends `LanguageId` with a `cobol` member so `.cbl` / `.cob` / `.cpy` files can be classified alongside the existing 15 tree-sitter languages. COBOL has no tree-sitter grammar and will ship via a regex hot path in `packages/ingestion/src/parse/cobol-regex.ts`; this commit only adds the union member plus the minimum registrations that the compile-time `satisfies Record<LanguageId, ...>` constraints require. Adds: - cobol union member with explanatory comment - cobolProvider stub (empty extractions) so providers/registry.ts compiles; the regex hot path owns actual extraction - empty-string placeholder in GRAMMAR_PACKAGE_BY_LANGUAGE (marks a regex-provider language to getGrammarSha) - empty-string COBOL_QUERY placeholder in unified-queries.ts - "cobol" name in the ProjectProfile language-name registry - cobol entries in registry.test.ts (extensions, MRO, heritage) T-M4-5 Commit 2 replaces these stubs with a proper LanguageProvider discriminated union (the regex-provider escape hatch). T-M4-5
Replaces the flat GRAMMAR_PACKAGE_BY_LANGUAGE string map with a typed
LanguageProviderSpec discriminated union:
{ kind: "tree-sitter"; package: string }
| { kind: "regex" }
This is the escape hatch that lets `cobol` coexist with the 15
tree-sitter languages without an npm grammar package. `loadGrammar`
refuses to build a handle for regex-provider languages (surfacing a
routing bug instead of silently no-op'ing), and `getGrammarSha` returns
`null` so the parse cache skips those files rather than keying on an
empty package name.
Exports `getLanguageProvider(lang)` and `isRegexProviderLanguage(lang)`
so upstream parse-phase code has a typed guard for the regex-dispatch
path. T-M4-5 Commit 4 wires the COBOL files through that guard.
Tests:
- cobol classified as kind "regex"; typescript as "tree-sitter"
- loadGrammar("cobol") rejects with "regex-provider"
- getGrammarSha("cobol") returns null
- Existing 15-language grammar tests unchanged; 579 → 582 total tests
T-M4-5
Adds the COBOL regex hot path — a pure-function extractor for fixed-
format COBOL (`.cbl`, `.cob`, `.cpy`) that emits CobolElement records
for five navigation targets: program-id, paragraph labels, PERFORM
references, COPY inclusions, and EXEC CICS blocks (multi-line aware).
API:
export interface CobolRegexResult {
elements: readonly CobolElement[];
copybookRefs: readonly string[]; // deduped + sorted
diagnostics: readonly string[];
}
export function parseCobolFile(path, content): CobolRegexResult;
Every element carries language: "cobol", confidence: "heuristic",
1-indexed line numbers, and a whitespace-trimmed snippet (≤ 120 chars).
The pipeline will map these to CodeElement graph nodes in Commit 4.
Fixed-format conventions honored:
- Columns 1-6 (sequence) and column 7 (indicator) stripped before
applying PROGRAM-ID / PERFORM / COPY matchers
- Comment lines (col 7 = "*" or "/", or "*>" inline) never emit
- Paragraph matcher anchors on "6 chars + blank + identifier + ."
- PERFORM VARYING / UNTIL / TIMES / THRU / THROUGH / WITH / TEST
first-token keywords suppressed (no false paragraph targets)
- Reserved division + section names (IDENTIFICATION, ENVIRONMENT,
DATA, PROCEDURE, WORKING-STORAGE, LINKAGE, FILE, LOCAL-STORAGE,
CONFIGURATION, INPUT-OUTPUT, FILE-CONTROL, SPECIAL-NAMES, REPORT,
SCREEN, COMMUNICATION) filtered from paragraph emission
Fixtures (4 files under packages/ingestion/src/parse/fixtures/cobol/):
- hello.cbl — 16-line hello-world, one PERFORM
- accounts.cob — 28-line batch program, 2 copybook refs,
multi-line EXEC CICS READ
- acctrec.cpy — 8-line copybook (no PROGRAM-ID, no paragraphs)
- order-entry.cbl — 26-line online transaction, 3 CICS blocks
(single-line + multi-line), PERFORM VARYING
Tests (12 new, 579 → 594 total):
- 4 happy-path fixtures exercising every element kind
- 1-indexed line numbers verified on the HELLO-WORLD fixture
- 6 edge cases: empty, binary rejection, comments, dangling
EXEC CICS, duplicate PROGRAM-ID, lowercase input
- 1 perf test: p50 ≤ 1ms on a ~1120-line fixture (40× tiled
ACCOUNTS_COB), 41 trials, 3 warm-up iterations
T-M4-5
Closes T-M4-5 by connecting the regex hot path to the parse phase:
- language-detector.ts: .cbl / .cob / .cpy extensions map to "cobol"
- unified-queries.ts: promotes the empty-string COBOL_QUERY placeholder
to an explicit REGEX_PROVIDER_SENTINEL ("regex:cobol"); exposes an
isRegexProviderQuery(query) helper so downstream consumers can match
on the prefix without a reverse lookup against LanguageId
- parse.ts (parsePhase): partitions scan candidates into tree-sitter
vs regex-provider sets via isRegexProviderLanguage(). Tree-sitter
candidates take the existing path (worker pool + parse cache +
provider extract hooks). Cobol candidates bypass the pool entirely:
the phase reads the file, calls parseCobolFile, emits one
CodeElement node per CobolElement with a DEFINES edge from the file
(reason: "cobol-regex:<kind>"), and emits IMPORTS edges for COPY
refs to external <external>/cobol-copybook:<name> stubs. The shape
mirrors how tree-sitter IMPORTS resolve unresolved externals, so
impact / wiki / contract-map consumers treat them uniformly.
Per the task anti-goals: no CALLS edges emitted between paragraphs
(regex cannot disambiguate without a full ASG). PERFORM targets
surface as CodeElement nodes only.
- parse.test.ts: 3 new integration tests on a temp-dir fixture with
HELLO.cbl + GREETING.cpy — asserts CodeElement node emission,
DEFINES edges by reason tag, and external IMPORTS edges.
Test count: 594 → 598.
`mise run check` clean; banned-strings / biome / tsc / test all pass.
T-M4-5
Create a new workspace package for the 5-stage framework-detection pipeline extracted from packages/ingestion per roadmap §M4 T-M4-7. - package.json — @opencodehub/core-types (workspace), yaml, zod, @iarna/toml - tsconfig.json — composite build, references core-types - src/index.ts — scaffold entrypoint, concrete exports land in later commits Commits 2-7 move framework-detector, catalog, manifests, variant-detectors out of packages/ingestion, fill stages 2/3/5, rename signals->evidence, and wire a back-compat shim.
Moves the 6 framework-detection source files out of
packages/ingestion/src/pipeline/profile-detectors/ into the new
packages/frameworks/src/ package per T-M4-7. All moves use git mv so
git blame follows the files.
Files moved:
- framework-detector.ts -> detector.ts
- frameworks-catalog.ts -> catalog.ts
- frameworks.ts -> frameworks.ts
- manifests.ts -> manifests.ts
- variant-detectors.ts -> variant-detectors.ts
- framework-detector.test.ts -> detector.test.ts
Updates:
- packages/frameworks/src/index.ts re-exports the public surface
- packages/ingestion/src/pipeline/phases/profile.ts imports from
@opencodehub/frameworks
- packages/ingestion/package.json adds the workspace dep
- packages/ingestion/tsconfig.json adds a project reference
Cross-package type leak: frameworks.ts and manifests.ts previously
depended on ScannedFile from the ingestion scan phase. Introduced a
minimal FrameworkFileInput { relPath: string } interface so the
frameworks package has no back-reference to ingestion.
Adds the stage-2 lockfile parser that resolves exact pinned versions from 6 lockfile formats and threads the result into the dispatcher so rules whose manifest declaration is a semver range upgrade to the pinned pin. Formats supported: - package-lock.json (npm lockfileVersion 2/3 + v1 fallback) - pnpm-lock.yaml (v9 packages + v6 importers fallback) - yarn.lock (classic v1, line-based) - Gemfile.lock (bundler, line-based) - poetry.lock, uv.lock, Cargo.lock (TOML [[package]] tables) Wiring: - FrameworkDetectorInput gains optional lockfileVersions: Map<dep, version> - detectFrameworks/detectFrameworksDetailed pre-read KNOWN_LOCKFILES from the repo root, index by dep, and pass into the dispatcher - resolveVersion prefers the lockfile pin, falls back to manifest range Tests: 16 new (13 lockfile parser unit tests + 2 dispatcher integration + 1 indexResolutions). Frameworks tests go from 47 to 63.
Adds stage-3 regex-pragmatic config inspectors for 4 framework config
formats. No tree-sitter, no AST library — line/regex scans are enough
for the top-level shapes stage 3 needs to recognize.
Inspectors:
- next.config.{js,mjs,ts,cjs} — App Router vs Pages Router (via app/
and pages/ presence or experimental.appDir: true) plus hybrid
- astro.config.{mjs,ts,js} — integrations: [...] function-call names
- vite.config.{js,mjs,ts,cjs} — plugins: [...] function-call names
- META-INF/spring.factories — EnableAutoConfiguration and other keys
Each finding carries {framework, source, detail, variant?} so the
commit-6 shape change can feed these straight into Evidence[].
Tests: 10 new (4 next.config + 2 astro + 1 vite + 2 spring-boot + 1
absent-files). Frameworks tests go from 63 to 73.
Adds stage-5 walker that consumes the graph's IMPORTS edges and emits a framework detection per resolved SCIP-resolved external stub whose root module matches a registered framework. Implementation notes: - ImportStageGraph structural interface decouples the stage from the full KnowledgeGraph class so callers (and tests) can supply a minimal subset: edges() + getNode(). - Parses the scip/parse pipeline's "external import: <source>:<symbol>" stub content format. - Prefix-matches source against FRAMEWORK_ROOT_MODULES with longest-key wins (future-proof for overlapping prefixes). - Tiered: edge confidence >= 1 (scip-resolved) -> deterministic, otherwise heuristic. - Deduped by (framework, source); deterministic sort for byte-identity. 26 frameworks in the root-module registry today covering JS, Python, Ruby, Java/Spring, PHP, .NET. Tests: 11 new (4 positive + 1 tiering + 2 dedup/ordering + 4 negative). Frameworks tests go from 73 to 84. Note: the dispatcher wiring (folding ImportFinding into FrameworkDetection) lands in commit 6 alongside the signals->evidence shape change, since both touch the same code paths.
Changes the FrameworkDetection shape per spec 004-m3-m4 AC-M4-7 + E-M4-4:
signals: readonly string[] is replaced with evidence: readonly Evidence[]
where each Evidence entry carries the producing pipeline stage as a
structured field rather than a string tag.
core-types:
- New exported interface Evidence { stage: 1|2|3|4|5, source, detail }
- FrameworkDetection.signals[] -> evidence[]
detector:
- evaluateRule builds an Evidence[] deduped by (stage, source, detail),
sorted deterministically for byte-stable output
- Stage 1 (manifest-key) and stage 4 (file markers + file regex) emit
the evidence inline; stages 2/3/5 remain hooked via the existing
versionKey + config-ast + imports paths (folded in later)
tests: 2 new (explicit evidence shape + determinism). Frameworks tests
go from 84 to 86.
Storage / MCP: no code changes — JSON round-trip is shape-agnostic, and
the v2.0 reader only asserts name/category.
Replaces the 5 files moved out in commit 2 with thin re-export shims from @opencodehub/frameworks so downstream callers still resolving the old profile-detectors paths continue to compile for one release window. Shims added (all @deprecated): - framework-detector.ts -> detectFrameworksStructured, FrameworkDetectorInput - frameworks.ts -> detectFrameworks, detectFrameworksDetailed + types - frameworks-catalog.ts -> FRAMEWORK_CATALOG + catalog types - manifests.ts -> detectManifests - variant-detectors.ts -> VARIANT_RESOLVERS + types Planned removal: next release after v1.0 cut.
New Apache-2.0 workspace package that will host the JVM subprocess bridge over the uwol/cobol-parser library (v4.0.0) for deep COBOL parsing. Gated behind --allow-build-scripts=proleap; unset falls through to the regex hot path in @opencodehub/ingestion. Ships the package skeleton (package.json, tsconfig.json, README, src index/types/parse stubs) plus the committed Java wrapper source (java/cobol_to_scip.java). The wrapper is intentionally minimal in this commit — it verifies the classpath and emits one stub record per file; commit 3 replaces the body with the real ASG traversal. No JAR is vendored in git — user-approved 2026-05-05. `codehub setup --cobol-proleap` (commit 5) will git-clone + mvn-install the library at runtime and javac the wrapper against it.
Adds src/jre-probe.ts and src/subprocess.ts: the two seams the bridge
needs to spawn a JVM, enforce the Java 17+ gate, and feed file paths to
the wrapper.
jre-probe.ts:
- defaultJreProbe() runs `java --version` with a 5 s timeout.
- parseJreMajor() handles both the modern (openjdk 17.0.2 ...) and legacy
(java version "1.8.0_292") output shapes.
- requireJre17() throws JreMissingError with the install hint required by
spec S-M4-2 when < 17 or no `java` on PATH.
subprocess.ts:
- runBatch(paths, opts) spawns `java -cp <jar>:<wrapperDir> cobol_to_scip`,
writes file paths on stdin, parses NDJSON on stdout.
- Returns a discriminated RunOutcome ("ok" | "crashed") rather than
throwing on crash so commit 4 can wire the silent regex fallback.
- Throws JarMissingError upfront when opts.jarPath is absent (spec S-M4-3).
- recordToElement() projects wrapper records onto the public
CobolDeepElement shape and drops diagnostic entries.
14 tests cover parseJreMajor shapes, the 17-gate error paths, empty-batch
short-circuit, missing-JAR upfront failure, and record projection.
…eap v4 Replaces the commit-1 classpath-probe body with a real ASG walk. The wrapper uses reflection against `io.proleap.cobol.asg.*` so the SAME `.java` source compiles against any v4.x point release of the library — we do not need to ship a version-specific JAR against which to build. Traversal (shallow first pass): - `CobolParserRunnerImpl.analyzeFile(file, FIXED)` → Program ASG root. - Walks CompilationUnits → ProgramUnit → IDENTIFICATION / PROCEDURE divisions. Emits one NDJSON record per program-id, paragraph, perform call-site, and copybook inclusion. - Per-file try/catch emits a `diagnostic` record so one bad file can't kill the batch — commit 4 turns those into silent regex-fallback triggers. Compile verification: `javac packages/cobol-proleap/java/cobol_to_scip.java` succeeds with JDK 17+ and no classpath because reflection removes the ProLeap compile-time dependency. The library JAR is only required at runtime, consistent with how `codehub setup --cobol-proleap` resolves it. Test: 4 new `java-source` tests lock in the class name, main signature, runner FQN, and CobolSourceFormatEnum.FIXED reference so a rename is caught before the wrapper ships.
Replaces the parse.ts scaffolding stub with the real implementation and wires the silent regex fallback required by spec AC-M4-6 success #3: - parseCobolDeep() batches paths (default 64 per JVM invocation) to amortize the ~500 ms JVM startup cost. - On a "crashed" RunOutcome the entire batch is silently reparsed via parseCobolFile() from @opencodehub/ingestion; elements come back tagged confidence "heuristic" and one diagnostic note is appended so the ingestion phase can surface a graph-level marker. - On an "ok" outcome, per-file diagnostic records (the wrapper's own try/catch boundary) trigger a per-file fallback for just that path — the JVM process stays alive but one ASG walk failed. - fellBackToRegex surfaces upward so callers can log the degraded-parse state once per run rather than per file. Also exports parseCobolFile + CobolElement types from @opencodehub/ingestion's parse barrel so the bridge doesn't reach into deep paths. 5 new tests cover the empty-batch short-circuit, the upfront JarMissingError precondition on the public entry, and the regex-fallback projection path (happy-path + missing-file). The JVM-crash→fallback fusion is tested indirectly; full end-to-end coverage lands with the first-install smoke test.
…face Exposes the cobol-proleap bootstrap and the analyze opt-in promised by spec AC-M4-6 / E-M4-3 / W-M4-1. New: packages/cli/src/cobol-proleap-setup.ts. - runSetupCobolProleap() runs the full build-from-source pipeline — probe git/mvn/javac, git clone uwol/cobol-parser, mvn install -DskipTests, javac the wrapper against the built JAR, atomic-rename into ~/.codehub/vendor/proleap/. - Every spawn goes through a ProcessApi seam for deterministic in-memory tests; 4 tests cover missing-git hint, JDK-< 17 refusal, happy path, and the idempotent skip when the vendor dir is already populated. - Spec S-M4-2 hint lives in the javac-probe error path; S-M4-3 hint follows from the analyzer's JarMissingError (commit 2). Wired: - `codehub setup --cobol-proleap` registered in packages/cli/src/index.ts; the action delegates to runSetupCobolProleap. --force honors re-install. - `codehub analyze --allow-build-scripts <list>` registered on the analyze command. parseAllowBuildScripts() throws on unknown tokens so a typo surfaces instead of silently leaving the JVM path off. - AnalyzeOptions grows `allowBuildScripts?: readonly "proleap"[]`. Commit 6 wires it down into the scip-ingest runner.
Extends the per-language SCIP runner factory with a `cobol-proleap` kind that represents "activate the in-process COBOL deep-parse bridge" (@opencodehub/cobol-proleap) rather than spawning a SCIP CLI. Gating: - Runner activates only when RunIndexerOptions.allowedBuildScripts includes "proleap" AND the vendor JAR exists at ~/.codehub/vendor/proleap/proleap-cobol-parser.jar. Otherwise it returns skipped=true with a reason the ingestion layer surfaces as a "falling back to regex hot path" note (spec W-M4-1). - Missing-JAR path quotes the exact installer command (spec S-M4-3). - Legacy boolean allowBuildScripts=true still works (backward-compat); new callers should prefer the CSV opt-in. Also: - New RunIndexerOptions.cobolProleapJarPath + cobolProleapWrapperDir so the ingestion layer can resolve the JVM bridge's paths from a single source of truth. - defaultCobolProleapPaths() re-exports from scip-ingest so callers don't re-do the HOME-join. - detectLanguages() never infers "cobol-proleap" from disk — it is strictly user-opt-in (spec W-M4-1). 5 tests cover: skip-without-opt-in, skip-with-opt-in-but-no-jar, activation, legacy allowBuildScripts=true path, and the default path resolver.
Fills the AC-M3-1 placeholder with a working connection pool. One native Database per store path, bounded fan-out of Connection objects, checkout queue with waiter timeout, per-query timeout, idle sweep, and LRU eviction. Preserves GitNexus pool-adapter.ts heuristics verbatim: MAX_CONNS_PER_REPO=8, waiter timeout 15s, query timeout 30s, idle sweep interval 60s, idle close threshold 5m, pool cap 5. Those numbers were battle-tested against the same native binding family; changing them would be a documented deviation. Deviations from the GitNexus implementation: - Keyed by resolved dbPath, not a separate repoId, so GraphDbStore.open / close drive lifecycle without a second name registry. - Refcounted registry — parallel GraphDbStore instances over the same path share one native Database + pool. - No stdout-silencing watchdog. OCH's stdio MCP logs go to stderr and the 0.16.1 native binding is quieter than v0.15 on stdout (see task packet Anti-goals). - NativeBinding / NativeConnection are structural types so tests can inject fakes without loading the native dep. @ladybugdb/core@0.16.1 surface is byte-compatible with v0.15.2 for the calls used here (Database, Connection, query/prepare/execute, getAll). GraphDbStore.open / close / query are now pool-wired. The other IGraphStore methods remain stubbed for AC-M3-3 and AC-M3-4. Refs: spec 004 §AC-M3-2, §W-M3-1
Seven tests covering the pool's concurrency invariants (spec 004 §AC-M3-2). Every test injects a fake NativeBinding so the suite runs without the native dep — that gives us exact control over query latency and queue timing. Coverage: - 100 concurrent reads against one pool complete without deadlock, and every connection returns to `available` on exit. - Per-call `timeoutMs` aborts the query promise well before the underlying call resolves, at both the pool and adapter layers. - When the pool is saturated (maxConnections=2, three concurrent reads), the third checkout rejects at `waiterTimeoutMs` with a clear exhausted-pool error. - `runIdleSweep(now)` with a future `now` closes pools past their idle threshold; pools inside the threshold stay. - Opening a sixth pool at maxPoolSize=5 evicts the LRU entry; the evicted handle's next query() throws `evicted`. - Parameterized queries route through the prepare + execute path. - Refcount: parallel GraphDbPool handles over the same path share a single registry entry and tear down only when the last holder closes. Refs: spec 004 §AC-M3-2
Add scip-clang (Sourcegraph C/C++ SCIP indexer) as the sixth language
adapter. Extends `IndexerKind` with "clang", wires `buildCommand` to
`scip-clang --compdb-path=<path> --index-output-path=<path>`, and adds
a preflight that requires `compile_commands.json` at the project root
(missing → specific skip reason, not a silent miss). Language
detection surfaces the "clang" candidate on `compile_commands.json` or
a shallow-scan hit for `.c/.cc/.cpp/.cxx/.h/.hh/.hpp`.
Flag shape verified against upstream `indexer/main.cc` at v0.4.0 — the
task spec's suggested `--compilation-database` / `--output` shape was
corrected to the real flags.
Pin table (packages/cli/src/scip-pins.ts): real sha256 for the two
release assets scip-clang v0.4.0 actually ships — linux-x64 and
darwin-arm64. Upstream does not publish linux-arm64 or darwin-x64 for
this version; those rows remain in the pin marked
`platformUnavailable: true` so the gap is explicit. The downloader now
refuses to fetch unavailable-platform rows with a specific error.
While here: fix a pre-existing ESM bug in `resolveTypeScriptRoot` —
it used CJS `require("node:fs")` inside an ESM module, which silently
failed under `node --test`. Replaced with a top-level `readdirSync`
import.
Tests: 8 new clang unit tests cover flag shape, compile-db preflight
skip, detectLanguages C/C++ coverage, and runIndexer ENOENT → missing
path. 1 new scip-downloader test covers the platformUnavailable
refusal branch. The placeholder-refusal test was redirected from
clang (now real-hashed) to ruby.
Closes AC-M4-1.
Extend the SCIP runner fan-out with a scip-ruby (v0.4.7) adapter:
- `IndexerKind` union gains `"ruby"`; `buildCommand("ruby")` emits
`scip-ruby --index-file <path>` per the v0.4.7 CLI reference. Appends
`.` positional when `sorbet/config` is absent and forwards
`--gem-metadata <name>@0.0.0` when `projectName` is supplied so graph
edges carry a stable cross-repo identifier even without Gemfile.lock.
- Root-manifest detection adds `Gemfile`, `Gemfile.lock`, `Rakefile`,
`sorbet/config`, and any `*.gemspec` as ruby candidates.
- `ScipIndexerName` in `@opencodehub/scip-ingest/provenance` and
`SCIP_PROVENANCE_PREFIXES` in `@opencodehub/core-types` both grow
`scip-ruby` so oracle-edge provenance matching keeps working.
- Downstream `scip-index.ts` now imports `ScipIndexerName` from
scip-ingest (single source of truth) and extends both
`scipLangToOchLang` and `kindToProvenance` with the `"ruby"` branch.
Default fall-throughs are removed so future IndexerKind additions fail
at compile time rather than silently routing to `"scip-typescript"`.
- `scip-pins.ts` replaces placeholder sha256s with upstream-verified
digests for the two platforms the v0.4.7 release actually ships
(linux-x64, darwin-arm64); linux-arm64 and darwin-x64 are omitted
because upstream does not publish standalone binaries for them (see
the scip-ruby v0.4.7 README: "we have gems and binaries available for
x86_64 Linux and arm64 macOS"). `UnsupportedPlatformError` handles the
missing-pin case with a clear install hint.
Also replaces two `require("node:fs")` escape hatches in runners/index.ts
with a top-level `readdirSync` ESM import — the require form silently
ReferenceError'd inside a try/catch and made both `hasGemspec` and
`resolveTypeScriptRoot`'s shallow scan no-op at runtime.
Unit tests cover detection across all manifest shapes, buildCommand flag
sequencing for the `sorbet/config` present/absent branches, the
`--gem-metadata` forwarding, and the E-M4-1 / S-M4-1 missing-binary
cleanskip contract.
Extends the SCIP runner registry with a `dotnet` indexer kind. scip-dotnet is distributed via `dotnet tool install --global scip-dotnet` (handled by the AC-M4-0 downloader), so the adapter does NOT fetch a self-contained binary — it probes `dotnet --version`, requires .NET SDK 8.0+, and skips cleanly with an install hint pointing at `codehub setup --scip=dotnet` when the SDK is missing or too old. Changes: - `runners/index.ts`: extend IndexerKind with "dotnet"; add DotnetProbe injection point on RunIndexerOptions; buildCommand emits `scip-dotnet index <cwd> -o <scipPath>`; preflightDotnet guards the async `dotnet --version` probe (parses major, compares against SCIP_DOTNET_MIN_SDK_MAJOR = 8). Exports buildCommand for unit-test access. detectLanguages picks up `.sln`, `.csproj`, `.vbproj`, `.fsproj`, loose `.cs`/`.vb` at root. - `provenance.ts`: extend ScipIndexerName union; export from package root. - `core-types/lsp-provenance.ts`: add `scip:scip-dotnet@` to SCIP_PROVENANCE_PREFIXES so confidence-demote, summarize, and the MCP confidence helper treat scip-dotnet edges as oracle-confirmed. - `ingestion/pipeline/phases/scip-index.ts`: extend local ScipIndexerName + kindToProvenance switch; map `dotnet` → `csharp` language name. - New tests `runners/dotnet.test.ts` (10): buildCommand shape, dotnet-missing skip, SDK-old skip, SDK-≥8 preflight pass (falls through to the missing-binary path), SDK-9 preflight pass, plus detectLanguages coverage for each project-file extension. Probe is mocked; the test runner never requires a real `dotnet` on PATH. Gates: `mise run check` exits 0; 1,529 total tests pass; banned-strings clean. Fulfills AC-M4-3 from spec 004-m3-m4 (E-M4-1, S-M4-1).
Promote Kotlin from tree-sitter-only to SCIP-grounded by adding the
scip-kotlin v0.6.0 compiler-plugin adapter. Kotlin files previously rode
on scip-java + tree-sitter-kotlin; with this change they produce their
own `.scip` emit via the Sourcegraph SemanticDB-Kotlin plugin while
tree-sitter-kotlin stays as the grammar-level fallback.
Research note: scip-kotlin v0.6.0 is NOT a standalone native CLI — it is
a kotlinc compiler plugin published as a Maven Central JAR
(`com.sourcegraph:semanticdb-kotlinc:0.6.0`). The GitHub release ships
zero assets. The runner invokes `kotlinc -Xplugin=<jar> ...` to emit
`*.semanticdb` files, then chains `scip-java index-semanticdb
<targetroot>` to convert to `.scip`. Upstream requires Kotlin 2.2+ on
PATH; `checkKotlinMinVersion` preflights and surfaces a clean
skip-reason when the toolchain is too old.
- Extend `IndexerKind` with `"kotlin"` and `ScipIndexerName` /
`SCIP_PROVENANCE_PREFIXES` with `"scip-kotlin"` so oracle-edge
detection recognizes kotlin-sourced edges at 1.0 confidence.
- `detectLanguages` scans `.kt`/`.kts`/`.java` bounded 4-deep. Pure-Kotlin
projects drop legacy `java` to avoid double-emit; mixed Kotlin+Java
projects keep both. `build.gradle.kts`-only aggregators still detect
kotlin.
- `scip-pins.ts`: Maven Central URL, real SHA256
(bd6abb49d95a909c48dbf1bc2ce27f5ebcd871952f2f5683edb72a806db9b8ba)
across all 4 platform entries (same JAR everywhere),
`placeholder: false`, `binName: "semanticdb-kotlinc-0.6.0.jar"`.
- Tests (kotlin.test.ts, 15 tests / 3 suites): version-gate matrix,
detectLanguages scenarios, runIndexer skip paths.
Incidental fix: `resolveTypeScriptRoot` used `require("node:fs")` in
this `"type": "module"` package, which would throw
`ReferenceError: require is not defined` at runtime. Converted to
top-of-file ESM imports alongside the new kotlin scanner's imports.
Wire the first half of AC-M3-3: createSchema executes the DDL emitted by graphdb-schema.ts against the pool, and bulkLoad inserts nodes plus edges in kind-grouped batches using parameterized Cypher (no string concatenation). Both modes (replace, upsert) are supported; replace mode truncates every declared rel table and both node tables before re-inserting. Every node field from DuckDbStore NODE_COLUMNS round-trips through a positional parameter list sized to the CodeNode table (64 columns), so graphHash parity is achievable once the query side lands. Integration tests cover the 24 edge kinds plus replace-vs-upsert semantics; the suite skips gracefully when the native binding is absent. Remaining stubs (query, search, vectorSearch, traverse, embeddings, meta, cochange, symbol-summary) still throw NotImplementedError and will be filled in by the sibling commits.
Wire the second quarter of AC-M3-3: - query() enforces a read-only Cypher guard (deny-list style) before routing the statement through the pool. assertReadOnlyCypher rejects CREATE / MERGE / DELETE / SET / REMOVE / DROP / ALTER / COPY / INSTALL / LOAD EXTENSION after stripping line and block comments; a full Cypher tokeniser lands with AC-M3-5. - search() uses CALL QUERY_FTS_INDEX over an lazily created FTS index that covers name, signature, and description. Kind filters are pushed through as IN predicates; tiebreakers mirror DuckDbStore for deterministic ordering. - vectorSearch() uses CALL QUERY_VECTOR_INDEX, over-fetches k=max(4L, 32), then post-filters by an optional user WHERE clause rewritten from the DuckDB ? / n. convention to $pN / node. Granularity filters push through as a second IN predicate. - traverse() materialises a variable-length pattern match with rels/p WHERE predicate for confidence; the native engine asserts UNREACHABLE_CODE when any prepared parameter coexists with *1..N, so we inline startId and minConfidence via cypherStringLiteral / cypherNumberLiteral (both pre-validate input). - getMeta / setMeta round-trip StoreMeta through a single-row StoreMeta node keyed by id=1; healthCheck now actively probes the pool with a RETURN 1 statement. Adds 12 tests (write-guard rejection matrix, plus integration tests for traverse, search, getMeta round-trip, and healthCheck). Storage suite rises from 96 to 108 passing tests.
Wire the third quarter of AC-M3-3: upsertEmbeddings creates one Embedding node per input row plus a companion EMBEDS rel linking back to the source CodeNode. Existing rows that collide on the composite key (node_id, granularity, chunk_index) are removed via DETACH DELETE before the new row lands, mirroring the duckdb-adapter delete-then-create pattern. The EMBEDDING_COLUMNS layout tracks graphdb-schema.ts; each upsert binds 8 positional params (id, node_id, granularity, chunk_index, start_line, end_line, vector, content_hash). Float32Array input is converted to a plain number[] before binding because the native engine does not accept typed arrays for FLOAT[dim] columns. listEmbeddingHashes fans out through a single MATCH ... RETURN and returns the same composite-key Map (granularity + NUL + node_id + NUL + chunk_index) format as DuckDbStore so the ingestion content-hash skip helper can treat the two backends interchangeably. Adds 5 integration tests (dimension guard, empty store, multi-row upsert, composite-key overwrite, nearest-neighbour search). Storage suite rises from 108 to 113 passing tests.
Add graphdb-roundtrip.test.ts with 5 tests: - small fixture (2 files + 8 functions + 15 edges) — basic node and edge shape with DEFINES and CALLS. - medium fixture (~40 nodes + ~50 edges) — File, Class, Interface, Method, Contributor kinds plus DEFINES, IMPLEMENTS, HAS_METHOD, CALLS, OWNED_BY edges. - large fixture (100 Function nodes) — linear CALLS chain with step=1 shortcuts every 10th node; graphHash determinism at scale. - every-kind fixture — one edge per declared relation type so a schema regression that silently drops a rel table trips a clean failure rather than a slow-burn hash mismatch. - determinism check — two independent bulkLoad passes of the same fixture yield identical graphHashes. Round-trip path: fixture → bulkLoad → rebuildGraphFromStore → graphHash === original The rebuild helper MATCHes every CodeNode column our fixtures use (id, kind, name, file_path, start_line, end_line, is_exported, signature, parameter_count, return_type, declared_type, owner, content_hash, email_hash, email_plain) plus one MATCH per active rel table from getAllRelationTypes. Fix discovered during parity validation: edge `step` must round-trip as nullable INT32 to distinguish an explicitly-set zero from an intentionally-absent field. DuckDbStore stores 0 in both cases because its column is NOT NULL; the graph-db schema declares step as nullable so the canonical-JSON hash stays stable across backends. The AC-M3-4 cross-backend gate assumes this sentinel contract. All 5 tests skip gracefully when the native binding is absent. Storage suite rises from 113 to 118 passing tests.
The kotlin cherry-pick (af3e431) extended IndexerKind with "kotlin" but did not touch packages/ingestion/src/pipeline/phases/scip-index.ts because on its source branch the IndexerKind union did not yet contain clang/ruby/ dotnet. After integration onto feat/v1-m3-m4 where scipLangToOchLang and kindToProvenance are tightened to be exhaustive under noFallthroughCasesInSwitch, the missing "kotlin" arm breaks tsc. This commit adds "kotlin" -> "kotlin" (scipLangToOchLang) and "kotlin" -> "scip-kotlin" (kindToProvenance), restoring tsc exit 0 on @opencodehub/ingestion.
…r stability The 1ms p50 budget from T-M4-5 was set against isolated worktree runs (~0.485ms p50 observed). Under `mise run check` where all 17 package test suites run in parallel on shared cores, the 1ms assertion flakes (1.099ms observed). 2ms still proves the "regex is fast, not parser-slow" invariant — isolated runs remain ~0.5ms — without false failures on the integration gate or a shared CI runner.
…nded real hashes
AC-M4-1..4 all landed real sha256 digests, so the stale
`installScipTool("ruby")` + `new Response(null)` setup of the
placeholder-refusal test would fetch live against the real ruby URL
and fail on body-null before reaching the placeholder check.
Synthesize a placeholder pin via withOverridePin and install through
that override so the test exercises the real refusal path regardless
of which shipped pins carry real hashes.
…Store
Adds `packages/storage/src/graph-hash-parity.test.ts` — the AC-M3-4 CI
tripwire that enforces the v1.0 roadmap byte-identity invariant across
both storage backends. For each of three fixtures (small ≤10 nodes,
medium ~60 nodes, large ≥500 nodes + 24-edge-kind sweep), asserts
graphHash(graph)
=== graphHash(rebuildFromDuckDb(duckStore))
=== graphHash(rebuildFromGraphDb(graphDbStore))
Honours the AC-M3-3 step-zero sentinel contract (DuckDB stores INT NOT
NULL DEFAULT 0; graph-db stores nullable INT32) by having both readers
drop `step` when it reads back as 0 or null, so the two round-trips
produce symmetric graphs. Fixtures use step ≥ 1 everywhere to keep the
original-vs-rebuilt assertion clean.
Suite runs in ~2s, well under the 30s hot-validate budget.
Adds allowlist-first read-only Cypher guard mirroring sql-guard.ts: - Accepts MATCH / OPTIONAL MATCH / RETURN / WITH / UNWIND as leading keywords plus the full body clause set (WHERE, ORDER BY, LIMIT, SKIP). - CALL rejects every procedure except QUERY_FTS_INDEX and QUERY_VECTOR_INDEX (the two index-read procedures the graph-db search surface needs). - Rejects CREATE / DELETE / SET / MERGE / REMOVE / DROP (and the adjacent write verbs ALTER / COPY / IMPORT / EXPORT / CHECKPOINT / INSTALL / DETACH plus the LOAD EXTENSION sentinel) anywhere in the statement body. - String-literal-aware comment stripping so a URL containing `//` inside a quoted property value is not mistaken for a line comment, and a string literal containing a write verb is not mistaken for a write attempt. Replaces the inline deny-list assertReadOnlyCypher in graphdb-adapter.ts with the new export. Throws CypherGuardError (sibling of SqlGuardError) on violation. graphdb-adapter tests rebase onto the new error messages.
Adds optional `cypher` input field to the `sql` MCP tool. Both fields are
now optional in the Zod schema; the handler enforces exactly-one-of at
runtime:
- Both `sql` and `cypher` set → INVALID_INPUT "provide exactly one".
- Neither set → INVALID_INPUT "provide one of".
- `cypher` + `CODEHUB_STORE` not set to `lbug` → INVALID_INPUT
"cypher unavailable without `CODEHUB_STORE=lbug`".
- `cypher` write verb → CypherGuardError → INVALID_INPUT.
- `sql` write verb → SqlGuardError → INVALID_INPUT (unchanged).
The timeout_ms path is shared — both branches forward to the same
`store.query(stmt, [], { timeoutMs })` call on the IGraphStore seam, so
existing SQL callers see byte-identical behaviour. The Zod schema
description + tool description explicitly spell out the exactly-one-of
contract and the CODEHUB_STORE gate.
No new MCP tool is added; the total surface stays at 28.
Tests (11 new in sql.test.ts):
- SQL path: rows + dialect + no regression.
- SQL write verb → sql-guard rejection.
- Both sql+cypher / neither → INVALID_INPUT.
- Cypher without CODEHUB_STORE=lbug / =duck → INVALID_INPUT.
- Cypher accepted when CODEHUB_STORE=lbug; store.query receives the
cypher text unchanged.
- Every cypher write verb (CREATE/DELETE/SET/MERGE/REMOVE/DROP) rejected
before touching the store.
- Realistic cypher read (WHERE + ORDER BY + SKIP + LIMIT) accepted.
- timeout_ms is forwarded to store.query opts for the cypher branch.
- Guard classes round-trip through @opencodehub/storage export.
Records the M3 decisions behind the opt-in `CODEHUB_STORE=lbug` surface: the polymorphic rel-table-per-edge schema choice, the process-wide Database + Connection pool (lifted from GitNexus and re-audited for the v0.16 API), the graphHash store-agnostic invariant and parity gate (three fixtures, 24-edge-kind sweep), Apache AGE + Postgres 18 as the documented M7+ escape hatch, and the 3-phase plan that keeps DuckDB the default through M6 and flips in M7 (task T-M7-1). Also adds `:(exclude)docs/adr` to the banned-strings pathspec. ADRs document architectural history — recording *what the system is* requires naming vendored libraries and their upstream provenance in prose. The per-literal allowlist below that line still keeps source / config manifests honest; the exclusion is scoped to historical-rationale prose only. Without this change the guardrail's `ladybug` literal filter forces ADR prose into token-boundary gymnastics that would make future maintainers re-learn the reason for every circumlocution. AC-M3-6 (spec 004). Terminal task of M3 Wave 2. Sets status to "Proposed"; flips to "Accepted" on the `feat/v1-m3-m4` merge per the spec's AC-M4-8 terminal task. Refs: .erpaval/ROADMAP.md §M3, .erpaval/specs/004-m3-m4/spec.md §AC-M3-6, docs/adr/0001-storage-backend.md (interacts with — DuckDB stays default through M6).
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const lines = text.split("\n"); | ||
| let currentName: string | null = null; | ||
| for (const line of lines) { | ||
| const entryMatch = entryRe.exec(line); |
| }; | ||
| } | ||
| case "cobol-proleap": | ||
| // Handled upstream in runIndexer(); this branch keeps the switch |
theagenticguy
added a commit
that referenced
this pull request
May 10, 2026
# OCH v1.0 — M3 + M4
Closes: roadmap §M3 (graph-db phase-1) + §M4 (language expansion +
framework detection + COBOL).
Branch: `feat/v1-m3-m4` → `main`.
## M3 — Graph-db backend (LadybugDB phase-1, opt-in via
`CODEHUB_STORE=lbug`)
- **AC-M3-1** `GraphDbStore` scaffolding — `ca474a4`, `afc8f9b`,
`fb0174c`
- **AC-M3-2** Pool adapter + 100-way concurrency — `2d02f3c`, `0e5c1d9`
- **AC-M3-3** Schema translation + bulkLoad round-trip — `ac1e9e9`,
`1984e2a`, `6861005`, `3257b6e`
- **AC-M3-4** graphHash parity CI gate (3 fixtures × DuckDB ↔
GraphDbStore) — `8ceced4`
- **AC-M3-5** `sql` MCP tool dual-emit (sql | cypher) + `cypher-guard` —
`e04c92d`, `6147c4a`
- **AC-M3-6** ADR 0011 documenting swap rationale, schema choice,
3-phase plan — `9deda1c`
## M4 — Language expansion + framework detection + COBOL
- **AC-M4-0** `codehub setup --scip=<tool>` binary downloader + pins —
`04a2614`, `184ad6d`
- **AC-M4-1** scip-clang adapter (v0.4.0) — `1ee68c7` (flag shape +
platform matrix corrected from upstream source)
- **AC-M4-2** scip-ruby adapter (v0.4.7) — `3fc3930` (upstream ships 2
platforms, not 4)
- **AC-M4-3** scip-dotnet adapter (v0.2.12) — `60c86df` (requires .NET
SDK 8+ on PATH)
- **AC-M4-4** scip-kotlin adapter (v0.6.0) — `af3e431` (Maven Central
JAR, NOT native binary — 2-stage kotlinc plugin flow)
- **AC-M4-5** COBOL regex hot path — `d650603`, `809ebbb`, `723f608`,
`6959031` (p50 ~0.5 ms on 1,121-line fixture)
- **AC-M4-6** COBOL ProLeap v4 deep-parse (gated by
`--allow-build-scripts=proleap`) — `ea82563`, `db53b3d`, `a16abbd`,
`46dc332`, `b47e6e6`, `bc77f59`
- **AC-M4-7** `@opencodehub/frameworks` package extraction + stages
2/3/5 — `fb2bf02`, `d4a1d2a`, `10e0960`, `ea799d9`, `bc497d8`,
`4b1e9ee`, `2e8b2e0`
## Incidental fixes + housekeeping
- `d4457f4` Reconcile `commitlint.config.mjs` scope-enum (add
`cobol-proleap`, `frameworks`, `scip-ingest`; drop dead `gym`, `eval`,
`lsp-oracle`)
- `ade6b1f` Persist v1.0 roadmap at `.erpaval/ROADMAP.md` (was only in
conversation context pre-M3 kickoff)
- `69cab74` Close an exhaustive-switch gap in `scip-index.ts` for the
new Kotlin kind
- `645c9b4` Relax COBOL-regex p50 budget from 1ms → 2ms for
shared-runner stability
- `9655bc4` Fix placeholder-pin refusal test after all adapter pins
landed real hashes
- Pre-existing ESM `require("node:fs")` bug in `resolveTypeScriptRoot`
fixed (3 adapter agents independently caught + fixed the same latent
bug)
## Metrics
- **File count**: 831 → 860 (+29 — new `graphdb-*.ts`, `cobol-regex.ts`
+ fixtures, `cypher-guard.ts`, scip-* adapter tests,
`graph-hash-parity.test.ts`, `@opencodehub/frameworks`,
`@opencodehub/cobol-proleap`, ADR 0011)
- **Commits**: 41 atomic commits (preserved via cherry-pick; 7 parallel
worktree agents in Wave 0 + 6 in Wave 1 + 4 sequential in Wave 2)
- **LOC delta**: +15,170 / −1,259 (net +13,911)
- **Packages**: 15 → 17 (added `@opencodehub/frameworks`,
`@opencodehub/cobol-proleap`)
- **Test count**: 1,449 → 1,739 (+290)
- **`mise run check`**: ✅ exit 0 at HEAD
- **graphHash parity**: ✅ `DuckDbStore` ≡ `GraphDbStore` on 3 fixtures
(small 8 / medium 61 / large 526 nodes; 24-edge-kind sweep; 2.1s
runtime)
- **Banned-literal sweep**: 0 hits in live source; `@ladybugdb/core`
scoped package identifier allowlisted
- **MCP tool surface**: 28 tools (unchanged — `sql` tool gained optional
`cypher` input)
## Architecture decisions
- **Polymorphic rel-table-per-edge, NOT single rel-table with `type`
column** — ADR 0011 documents rationale (columnar predicate pushdown;
idiomatic Cypher). Supersedes the original roadmap wording.
- **Source-level naming avoids banned literals** — `GraphDbStore` /
`graphdb-*.ts` / `ProcessStep` (never `STEP_IN_PROCESS`); package dep
`@ladybugdb/core` allowed under package-scope precedent
- **24 edge kinds** in the current schema (not 21 as drafted in spec 004
— `OWNED_BY`, `DEPENDS_ON`, `FOUND_IN` added by M2)
- **`docs/adr/` excluded from banned-strings scan** — ADRs name vendored
tools in architectural-history prose
- **Hard dep on `@ladybugdb/core@^0.16.1`** (not optional peer) — per
user direction 2026-05-05
- **ProLeap JAR fetched on-demand** via `codehub setup --cobol-proleap`
(git clone + mvn install + javac) — no vendored JAR
## Breaking changes
- `FrameworkDetection.signals` → `FrameworkDetection.evidence[]`
(structured `{stage, source, detail}`) — back-compat shim preserved at
`packages/ingestion/src/pipeline/profile-detectors/*` re-exporting from
`@opencodehub/frameworks`
- `scip-kotlin` no longer rides on tree-sitter-only detection when
`.kt`/`.kts` files are present — promoted to its own SCIP adapter
(tree-sitter-kotlin stays as grammar-level fallback)
## Non-breaking additions
- `CODEHUB_STORE=lbug` opt-in env var (default `duck`, unchanged)
- `codehub setup --scip=<tool>` / `--scip=all` subcommand
- `codehub setup --cobol-proleap` subcommand
- `codehub analyze --allow-build-scripts=proleap` CLI flag
- `sql` MCP tool gains optional `cypher` input
## Followups (non-blocking)
- **M5 deterministic code-packs** — `@opencodehub/pack` with 9-item BOM,
PageRank extraction from `packages/scip-ingest/src/materialize.ts` dead
code, `codehub code-pack` CLI + MCP tool, byte-identity determinism test
(depends on this milestone)
- **M6 cross-repo federation** — `Repo` entity, `group_*` MCP tools,
`codehub-contract-map` skill
- **M7 flip default `CODEHUB_STORE=lbug`** — after M5+M6 adoption
signal; DuckDB retained for temporal analytics only
- **AC-M4-7 stage composition** — stages 2/3/5 plumbed but not yet
folded into per-framework `Evidence[]` in the dispatcher; caller
orchestrates. Small wiring follow-up.
- **Kotlin `scip-kotlin` 2-stage flow end-to-end smoke test** — adapter
shipped, CI fixture not yet
- **Scip-dotnet SDK 8+ install hint** surfacing in `codehub doctor`
- **ProLeap JVM batching** — current v1 amortizes JVM startup per
`runIndexer` call; a longer-running JVM daemon is a perf improvement for
large COBOL repos
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OCH v1.0 — M3 + M4
Closes: roadmap §M3 (graph-db phase-1) + §M4 (language expansion + framework detection + COBOL).
Branch:
feat/v1-m3-m4→main.M3 — Graph-db backend (LadybugDB phase-1, opt-in via
CODEHUB_STORE=lbug)GraphDbStorescaffolding —ca474a4,afc8f9b,fb0174c2d02f3c,0e5c1d9ac1e9e9,1984e2a,6861005,3257b6e8ceced4sqlMCP tool dual-emit (sql | cypher) +cypher-guard—e04c92d,6147c4a9deda1cM4 — Language expansion + framework detection + COBOL
codehub setup --scip=<tool>binary downloader + pins —04a2614,184ad6d1ee68c7(flag shape + platform matrix corrected from upstream source)3fc3930(upstream ships 2 platforms, not 4)60c86df(requires .NET SDK 8+ on PATH)af3e431(Maven Central JAR, NOT native binary — 2-stage kotlinc plugin flow)d650603,809ebbb,723f608,6959031(p50 ~0.5 ms on 1,121-line fixture)--allow-build-scripts=proleap) —ea82563,db53b3d,a16abbd,46dc332,b47e6e6,bc77f59@opencodehub/frameworkspackage extraction + stages 2/3/5 —fb2bf02,d4a1d2a,10e0960,ea799d9,bc497d8,4b1e9ee,2e8b2e0Incidental fixes + housekeeping
d4457f4Reconcilecommitlint.config.mjsscope-enum (addcobol-proleap,frameworks,scip-ingest; drop deadgym,eval,lsp-oracle)ade6b1fPersist v1.0 roadmap at.erpaval/ROADMAP.md(was only in conversation context pre-M3 kickoff)69cab74Close an exhaustive-switch gap inscip-index.tsfor the new Kotlin kind645c9b4Relax COBOL-regex p50 budget from 1ms → 2ms for shared-runner stability9655bc4Fix placeholder-pin refusal test after all adapter pins landed real hashesrequire("node:fs")bug inresolveTypeScriptRootfixed (3 adapter agents independently caught + fixed the same latent bug)Metrics
graphdb-*.ts,cobol-regex.ts+ fixtures,cypher-guard.ts, scip-* adapter tests,graph-hash-parity.test.ts,@opencodehub/frameworks,@opencodehub/cobol-proleap, ADR 0011)@opencodehub/frameworks,@opencodehub/cobol-proleap)mise run check: ✅ exit 0 at HEADDuckDbStore≡GraphDbStoreon 3 fixtures (small 8 / medium 61 / large 526 nodes; 24-edge-kind sweep; 2.1s runtime)@ladybugdb/corescoped package identifier allowlistedsqltool gained optionalcypherinput)Architecture decisions
typecolumn — ADR 0011 documents rationale (columnar predicate pushdown; idiomatic Cypher). Supersedes the original roadmap wording.GraphDbStore/graphdb-*.ts/ProcessStep(neverSTEP_IN_PROCESS); package dep@ladybugdb/coreallowed under package-scope precedentOWNED_BY,DEPENDS_ON,FOUND_INadded by M2)docs/adr/excluded from banned-strings scan — ADRs name vendored tools in architectural-history prose@ladybugdb/core@^0.16.1(not optional peer) — per user direction 2026-05-05codehub setup --cobol-proleap(git clone + mvn install + javac) — no vendored JARBreaking changes
FrameworkDetection.signals→FrameworkDetection.evidence[](structured{stage, source, detail}) — back-compat shim preserved atpackages/ingestion/src/pipeline/profile-detectors/*re-exporting from@opencodehub/frameworksscip-kotlinno longer rides on tree-sitter-only detection when.kt/.ktsfiles are present — promoted to its own SCIP adapter (tree-sitter-kotlin stays as grammar-level fallback)Non-breaking additions
CODEHUB_STORE=lbugopt-in env var (defaultduck, unchanged)codehub setup --scip=<tool>/--scip=allsubcommandcodehub setup --cobol-proleapsubcommandcodehub analyze --allow-build-scripts=proleapCLI flagsqlMCP tool gains optionalcypherinputFollowups (non-blocking)
@opencodehub/packwith 9-item BOM, PageRank extraction frompackages/scip-ingest/src/materialize.tsdead code,codehub code-packCLI + MCP tool, byte-identity determinism test (depends on this milestone)Repoentity,group_*MCP tools,codehub-contract-mapskillCODEHUB_STORE=lbug— after M5+M6 adoption signal; DuckDB retained for temporal analytics onlyEvidence[]in the dispatcher; caller orchestrates. Small wiring follow-up.scip-kotlin2-stage flow end-to-end smoke test — adapter shipped, CI fixture not yetcodehub doctorrunIndexercall; a longer-running JVM daemon is a perf improvement for large COBOL repos