Skip to content

Integration/v1 adr alignment#349

Merged
ohdearquant merged 105 commits into
mainfrom
integration/v1-adr-alignment
May 25, 2026
Merged

Integration/v1 adr alignment#349
ohdearquant merged 105 commits into
mainfrom
integration/v1-adr-alignment

Conversation

@ohdearquant
Copy link
Copy Markdown
Owner

This pull request introduces new database migrations (V5–V9) and updates the entity store schema and logic to support an optional entity_type field and new indexing. It also adds lifecycle and backend routing columns to the graph_edges table, with robust migration idempotency and expanded test coverage. Additionally, the CI workflow is updated to run on integration branches.

Database migrations and schema changes:

  • Adds migration V5 to introduce an optional entity_type column and a composite index (namespace, kind, entity_type) to the entities table, with idempotency handling if the column already exists. [1] [2] [3] [4]
  • Adds migration V9 to rebuild the graph_edges table with new lifecycle columns (updated_at, deleted_at) and a target_backend column for backend routing, including a partial index on target_backend. [1] [2]

Entity store and query logic:

  • Updates all entity store code and queries to handle the new entity_type field, including reading, writing, and filtering by entity_type. [1] [2] [3] [4] [5] [6] [7]

Migration and test improvements:

  • Ensures migrations are idempotent by checking for the existence of columns before applying changes, and expands tests to verify all new migrations, columns, and indexes are correctly applied and tracked. [1] [2] [3] [4] [5]

CI workflow:

  • Updates the CI workflow to also run on integration/** branches in addition to main.

ohdearquant and others added 30 commits May 23, 2026 22:23
…h ADR-001/008

Closes #311.

F001/F002/F006: EntityKind enum extended from 6 to 8 closed base kinds —
Artifact and Service added. ENTITY_KINDS in KgPack updated to match.
Module comment updated from "6 kinds" to "8 closed base kinds".

F003/F004: entity_type persisted as a first-class SQL column (not a
property). V5 migration adds the column and a composite index. Entity
struct, storage layer, and all runtime APIs (create_entity, list_entities,
hybrid_search) accept and propagate entity_type.

F005: Local EntityKind enum removed from khive-pack-kg/vocab.rs.
KgPack now re-exports khive_types::EntityKind so a single canonical enum
covers the whole workspace.

F046: NodePattern gains an entity_type field. The GQL and SPARQL parsers
lift entity_type out of the properties map into the dedicated field. The
SQL compiler emits entity_type as a direct column condition (not
json_extract) in both fixed-length and variable-length paths.
NODE_COLUMNS whitelist updated; full-variable SELECT projections include
entity_type.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
F128: rename Fold methods initial→init, step→reduce; add Send+Sync supertrait
F129: remove chrono timing from FoldOutcome; simplify to {state, entries_processed}
F130: change Objective::select return Vec<Selection<&T>>; update all call sites
F131: add Send+Sync to Anchor trait; change credit weights f32→f64
F132: add Send+Sync to Selector<T> trait
F133: add ComposePipeline<T> struct to lib.rs
F134: add khive-types dependency (pragmatic: keep chrono/serde/uuid for context.rs)

147 unit tests pass, clippy clean, fmt applied.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Objective::select now returns Vec<Selection<&T>> (F130). The runtime
registry wraps it with ObjectiveResult<Selection<&T>> as its public
contract; apply the same into_iter().next().ok_or_else adapter used
in khive-fold's registry at all three call sites.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g primitives

(closes #312)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ip, filter, parser lift, and SQL compilation

(closes #311)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ignment

(closes #312)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Align edge behavior across ADR-002, ADR-004, ADR-009, ADR-029, ADR-038:

- F007: EdgeRelation (15 variants), EdgeCategory (8), symmetric/directed
  classification, EdgeRelation::ALL slice, FromStr / Display impls
- F008: Edge struct with lifecycle columns (namespace, updated_at, deleted_at,
  target_backend); upsert_edge / upsert_edges storage methods; V5 migration
  adds updated_at (NOT NULL), deleted_at, target_backend to graph_edges
- F009: parse_relation error message built dynamically from EdgeRelation::ALL
- F010: base_entity_rule_allows() with 42-entry ADR-002 allowlist; enforcement
  in validate_edge_relation_endpoints; test fixtures updated to valid pairs
- F011: pack EDGE_RULES remain additive per ADR-031; no base-rule duplication
- F012: canonical_edge_endpoints() for symmetric relations (CompetesWith,
  ComposedWith) — canonicalizes to (min,max) UUID order before upsert
- F013: metadata: Option<Value> on link(); validate_edge_metadata rejects
  unknown keys; infer_dependency_kind / merge_dependency_kind auto-inject
  dependency_kind for DependsOn edges
- F205/D1: BulkLinkEntry + updated LinkParams; handle_link dispatches to
  link_many (atomic) or per-entry link() loop (non-atomic)
- D2: LinkSpec struct, build_edge, link_many (atomic upsert_edges batch)
- curation.rs: EdgeRow extended with updated_at/deleted_at/target_backend to
  match V5 schema; merge_entity_sql INSERT updated accordingly

(closes #313)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Enforce ADR-002 allowlist on supersedes relation endpoints (concept→concept,
  document→document, dataset→dataset, project→project, person→person, org→org
  allowed; cross-kind entity pairs rejected)
- Add 1000-entry limit on bulk link requests
- Deduplicate bulk link entries by (source, target, relation) after symmetric
  endpoint canonicalization; track skipped count
- Return structured {attempted, created, skipped, failed, edges, errors}
  response for both atomic and non-atomic bulk link paths
- Tests: f010_supersedes_same_kind_entity_allowed,
  f010_supersedes_cross_kind_entity_rejected,
  bulk_link_over_1000_entries_returns_error,
  bulk_link_dedup_and_response_shape

(closes #313)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace closed NoteKind enum with pack-owned String kinds, make
salience/decay_factor genuinely nullable via V7 table rebuild, add
NoteStatus::Deleted for soft-delete, tombstone merged entities instead
of hard-deleting, and route update/delete/merge dispatch by public kind.

18 findings (6 CRIT, 10 MAJ, 2 MIN) across ADR-004, ADR-005, ADR-013,
ADR-014, ADR-039. Schema changes: V5 (note status), V6 (entity
tombstone columns), V7 (notes table rebuild for nullable metrics).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ter-05)

Aligns ADR-005 (storage capability traits) and ADR-011 (multi-vector) per
cluster-05 spec. Adds SparseStore capability, multi-vector storage shape, and
removes quota policy from storage layer per ADR boundaries.

(closes #315)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…acts

- khive-types: replace flat Event with typed EventKind (23 variants),
  EventPayload enum, AggregateRef, and all Proposal/Rerank payload structs;
  add EventBuilder; export all new surface in lib.rs
- khive-storage: new Event struct (kind, payload: Value,
  payload_schema_version, profile_state_version, session_id, aggregate);
  add EventObservation, EventView, ReferentKind, ObservationRole; drop
  namespaces from EventFilter; add kinds/session_id/observed/selected/
  payload_proposal_id filter fields; export in lib.rs
- khive-db: atomic insert_event_with_observations (single transaction);
  query_events ORDER BY created_at DESC, id DESC (F115); namespace always
  enforced (F113 via NamespaceToken in operations layer); new EVENTS_DDL
  with event_observations table + 10 indexes; migration V5 with idempotent
  build_v5_event_observability_sql
- khive-runtime: NamespaceToken (ADR-007 bridge); list_events signature
  → (token: &NamespaceToken, filter, page); DispatchHook::on_dispatch →
  &EventView; export EventObservation/EventView/ObservationRole/ReferentKind
- khive-pack-brain/khive-pack-kg: update to 5-arg Event::new, with_payload,
  DispatchHook &EventView, and removed namespaces filter field (transitive)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ish MIN sentinel from NEG_INF

Per Ocean's "no legacy/deprecation in early khive" rule + the formal model in
lean-proofs/Score/DeterministicScore.lean:

1. Delete QuantKey entirely (not deprecate). The Lean proof has no QuantKey; it
   was a hot-loop sort optimization outside the deterministic scoring contract.
   Removed crates/khive-score/src/quantkey.rs, the mod decl in lib.rs, and
   already-absent re-exports in khive-fold.

2. Align constants with the Lean proof:
   - NEG_INF was i64::MIN; the proof has NEG_INF = i64::MIN + 1.
   - Add MIN (= i64::MIN) as reserved unreachable sentinel, never produced by
     public arithmetic or float conversion.
   - Saturating add/sub/mul/div + from_f64 now clamp to [NEG_INF, MAX], matching
     the proof's RuntimeValid predicate.

3. Update tests to match the proven invariants:
   - neg_inf_is_i64_min → neg_inf_is_i64_min_plus_one
   - min_sentinel_is_i64_min (new)
   - min_sentinel_distinct_from_neg_inf (new)
   - neg_infinity_maps_to_neg_inf
   - underflow_clamps_to_neg_inf_not_min (new)

4. Update ops.rs weighted_sum clamp to use DeterministicScore::NEG_INF/MAX
   instead of raw i64::MIN/MAX.

5. ADR-006 updates:
   - Saturation/sentinel section now distinguishes MIN vs NEG_INF and references
     the Lean proof.
   - "QuantKey deprecation" → "QuantKey removal" (no deprecation period).
   - Canonical implementation section: khive-score is self-contained for now;
     ruvector-core migration is deferred until upstream ships our contributions.

Tests: 41 passed in khive-score, full workspace cargo check green.

(closes #315 — superseded inline by the merged contract; reopen if more is needed)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…re vs Lean proof, QuantKey removed, Fold init/reduce, FoldOutcome deterministic
…ist (ADR-001)

c01 added entity_type as a first-class node column; the GQL compiler now emits
it in the 'Valid: ...' error list. Update the contract assertion to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The v1 ADR alignment work uses an integration/v1-adr-alignment branch as a
staging area before main. CI must run on PRs targeting integration/* the same
way it runs on main, otherwise the staging branch has no test gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mn extension

Adds entity_type as first-class field on Entity, storage, DB schema, GQL AST/SQL.
Updates contract test for new valid-column list.
Adds CI trigger on integration/* branches.

Known follow-ups (tracked):
- #345 EntityTypeRegistry implementation (currently stored as unchecked string)
- #346 cluster-24 migration ledger collision (V5 used here + #338 + ledger)
… fix ADR-002 contract tests

Conflict resolution:
- migrations.rs: c01's V5 (add_entity_type_to_entities) already in integration;
  c03's edge-lifecycle migration becomes V6 (next available, no ledger collision).
  Tests updated to expect version=6 and V1..V6 count.
- All test prose conflicts (V1..V5 vs V1 through V5): kept integration's wording.

ADR-002 contract test fixes (codex blocker on #338):
- tests/contract_test.py:439 — concept→concept relation changed from depends_on
  to enables. ADR-002 restricts depends_on to Project/Service/Artifact endpoints.
- tests/contract_test.py:679 — same fix in test_merge_semantics.

cargo check --workspace passes.
…DR-002)

ADR-002 §endpoint allowlist restricts depends_on to Project/Service/Artifact
endpoints. Two contract tests (test_edge_cascade_hard_delete and
test_merge_semantics) used depends_on between Concept entities, which the
runtime now rejects. enables is the valid concept-to-concept replacement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nt contract

- crates/khive-runtime/src/operations.rs: 10 internal test callers of
  create_entity updated to pass None for the entity_type arg added by c01.
- tests/smoke_test.py: introduced_by direction reversed to concept→document
  per ADR-002 (LoRA introduced_by paper, not paper introduced_by LoRA).
  Neighbor assertion updated: LoRA now has 1 inbound + 1 outbound (not 2 inbound).
- tests/contract_test.py: depends_on between Concept entities replaced with
  enables (valid concept-to-concept) — same pattern in test_edge_cascade and
  test_merge_semantics.

make ci passes locally end-to-end (workspace tests, smoke tests, GTD + memory pack
smoke tests, contract tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…5/soft-delete SQL)

- V9 migration: renumber V6 → V9, add V6/V7/V8 reserved no-op slots per
  ADR-015 ledger (F052/CRIT)
- F010: fix supersedes allowlist to match ADR-002 exactly (Concept/Document/
  Artifact/Service/Dataset); remove Project/Person/Org; add 7 regression tests
- F011: migrate pack metadata end-to-end from VerbDef/VERBS to HandlerDef/
  HANDLERS/Visibility across khive-types, khive-runtime, pack-kg, pack-gtd,
  pack-memory, pack-brain, and all test packs; keep deprecated VerbDef alias
- F012: enforce symmetric-relation direction normalization in
  neighbors_with_query() so competes_with/composed_with always returns Both
  direction; add regression test
- F161: document and test ADR-009 target_backend invariant at runtime write
  path; add f161_link_always_writes_null_target_backend and link_many tests
- F205: guard bulk link edges key behind verbose=true in both atomic and
  non-atomic paths; update integration tests for correct response shape
- Soft-delete SQL: add deleted_at IS NULL to all graph_edges aliases in
  fixed-length pattern JOIN and both CTE steps (seed + recurse)
- File GitHub issue #347 for ADR-038 write_keys preflight (out of scope)

All gates pass: fmt, check, clippy -D warnings, test --workspace.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- types/lib.rs: export NoteKind + HandlerDef/Visibility (c03), keep NoteStatus (c04)
- vocab.rs: add EntityKind enum (c04) alongside NoteKind (c03)
- handlers.rs: merge import lists (LinkSpec from c03, ContentMergeStrategy/NotePatch from c04); drop orphaned dead-code block from c03 in handle_delete
- stores/entity.rs: all SQL includes entity_type (c03) + merged_into/merge_event_id (c04)
- migrations.rs: keep c03 V5-V9; renumber c04 V5→V10/V6→V11/V7→V12 with idempotency checks
- curation.rs: SQL SELECTs include entity_type + merged_into/merge_event_id; drop misplaced c03 edge-rewire block (c04's dry_run-guarded version is authoritative)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…_edge arity

NoteKind lives in khive-pack-kg/src/vocab.rs, not khive-types/src/note.rs.
delete_edge requires a bool hard arg — pass p.hard.unwrap_or(false) in Edge branch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…e cols)

- Update DupPack test to use HANDLERS/handlers() (VerbDef→HandlerDef rename)
- Fix create_entity test calls: add entity_type arg (7-arg signature from c03)
- Fix link test call: add metadata arg (6-arg signature from c03)
- Update edge rewire SQL to include updated_at/deleted_at/target_backend cols
  (V9 migration made updated_at NOT NULL)
- Add #[allow(dead_code)] on EdgeRow structs (fields needed for column position)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add into_id == from_id guard on merge_entity and merge_note (data loss bug)
- Add regression test merge_entity_self_merge_rejected
- Update note-merge EdgeRow to include lifecycle cols from V9 migration
- Update note-merge edge INSERT to include updated_at/deleted_at/target_backend

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tale docstrings

- Replace silent Namespace::local() fallback with InvalidInput error in VerbRegistry::dispatch
- Update delete_note/delete_entity docstrings: 'returns false' -> 'Err(NamespaceMismatch)'

(closes #317)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Seal Namespace: remove Default/From<String>/unchecked new; add parse() with
  validation (lowercase alnum+dot, 1-256 chars, no leading/trailing dot)
- Introduce NamespaceToken: sealed authorization proof minted after gate check
- Migrate all runtime CRUD/curation/retrieval signatures from Option<&str> to
  &NamespaceToken
- Pack handlers no longer deserialize namespace: Option<String>; token threaded
  from dispatch after gate authorization
- Add RuntimeError::NamespaceMismatch with opaque messages (no namespace leak)
- Update all integration tests to use token-based API

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ohdearquant and others added 2 commits May 24, 2026 23:25
c20 (embedding_model_registry) landed at V14; c22 (proposals_open) at V15.
ADR-015 §Process mandates ledger update in the same PR that introduces a
migration — both c20 and c22 violated this. Backfill here after integration
codex review caught the gap. V1–V15 are now production schema and frozen.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ntract)

Closes integration codex CRIT-1: both methods previously took LinkSpec with
caller-supplied namespace and minted a NamespaceToken internally via
self.authorize(), bypassing any gate check (same pattern as c22 CRIT-2).

Both methods now require &NamespaceToken as the first param; LinkSpec.namespace
(if set) must match token.namespace() or InvalidInput is returned. Callers
thread the token from dispatch (which mints after gate validation).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ohdearquant and others added 6 commits May 24, 2026 23:40
…d errors

Re-applies the c21 round-1 fix (commit 71d9e6e on c21 branch) that didn't
land on integration because the c21→integration merge happened before the
fix was pushed.

engine migrate/drift-check and vector sweep now return Err with explicit
'not yet implemented' messages citing the deferred follow-up issues.
engine list/status are functional (query _embedding_models registry,
which exists post-V14).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…s and migration

- Deleted the entire proofs directory containing Lean4 theorem files for various retrieval algorithms and scoring properties, as part of the transition to a new structure.
- Removed the README.md file that outlined the proof structure and status.
- Eliminated the check-proof-references.sh script that validated proof correspondence namespace coverage.
- Removed the migrate_notes.py script used for migrating notes from an internal database to the OSS khive-graph database.
- All workspace crate versions: 0.2.0 → 0.2.1 (23 Cargo.toml files)
- All inter-crate `version = "0.2.0"` refs bumped accordingly
- lattice-embed workspace dep: 0.1.2 → 0.2.3
- cli/deno.json + npm/package.json + 6 npm/kernel-*/package.json: 0.2.0 → 0.2.1
- npm umbrella optionalDependencies updated to match

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Workspace Cargo.toml + npm/package.json homepage fields point to Ocean's
GitHub profile. Functional khive.ai references (api endpoint, doc links)
are left as-is.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Doc landing links → https://github.com/ohdearquant/khive
- ADR deep links → GitHub blob URLs (ADR-057 → ADR-035 v1 equivalent)
- CLI help text re-worded to drop khive.ai brand
- api.khive.ai auth endpoint kept as-is (functional default for hosted
  auth; TODO comment added that this needs replacement pre-commercial)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
scripts/check-proof-references.sh was removed in 1cbab62 ("Remove
obsolete proof files and scripts...") but scripts/ci.sh still
referenced it on line 8, causing CI to fail with "No such file or
directory" / exit 127 before any real checks ran.

Drop the orphaned echo + call so CI proceeds to fmt/clippy/test.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant