-
Notifications
You must be signed in to change notification settings - Fork 0
docs: record LARQL phase 1 decisions #174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ttupper92618
wants to merge
8
commits into
main
Choose a base branch
from
feature/larql-phase-1-decisions-publisher
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
7420ce2
docs: record LARQL phase 1 decisions
ttupper92618 9d6f8ac
feat: add LARQL runner isolation groundwork
ttupper92618 4c0b95d
fix: handle LARQL vindex cache paths
ttupper92618 33ff860
fix: monitor LARQL child exits
ttupper92618 e924ce9
fix: preserve read-only model path semantics
ttupper92618 0abdd07
fix: harden larql shutdown and vindex pulls
ttupper92618 6546a49
fix: preserve vindex cache ownership
ttupper92618 6733402
fix: align LARQL CLI range bounds
ttupper92618 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # ADR 0001: LARQL Runner Type | ||
|
|
||
| Status: Accepted for planning | ||
| Date: 2026-05-12 | ||
| Parent roadmap: https://github.com/Foxlight-Foundation/Skulk/issues/173 | ||
| Tracking issue: https://github.com/Foxlight-Foundation/Skulk/issues/153 | ||
| Source plan: https://github.com/Foxlight-Foundation/Skulk/blob/claude/understand-larql-repo-sJqA1/docs/slice-placement-and-vindex-publisher-plan.md | ||
|
|
||
| ## Context | ||
|
|
||
| Skulk's current execution model is centered on worker-managed runner | ||
| subprocesses. A worker observes event-sourced placement state, downloads or | ||
| stages the assigned model artifacts, starts a runner subprocess, and reports | ||
| lifecycle transitions through the same event stream every node applies. | ||
|
|
||
| LARQL introduces a different execution role: a process that serves vindex-backed | ||
| FFN or expert slices over HTTP. That process still needs the same operational | ||
| properties Skulk expects from model runners: deterministic startup, supervised | ||
| shutdown, crash visibility, logging, readiness state, and eventual placement | ||
| metadata. | ||
|
|
||
| ## Decision | ||
|
|
||
| Skulk will model LARQL as a first-class `LarqlRunner` runner type managed by | ||
| the worker. The worker will supervise a child `larql-server` process alongside | ||
| the existing MLX runner subprocesses. | ||
|
|
||
| The `LarqlRunner` will use Skulk's runner lifecycle conventions: | ||
|
|
||
| - worker-owned process supervision | ||
| - stdout/stderr forwarding into Skulk logging | ||
| - readiness and failure reporting through event-sourced state | ||
| - shutdown driven by instance and runner lifecycle events | ||
|
|
||
| The initial implementation will treat `larql-server` as an upstream binary | ||
| dependency, not as in-tree Rust or Python code. | ||
|
ttupper92618 marked this conversation as resolved.
|
||
|
|
||
| ## Consequences | ||
|
|
||
| `LarqlRunner` becomes part of Skulk's runner taxonomy. Future implementation | ||
| work must add explicit runner metadata rather than overloading MLX shard | ||
| metadata or treating LARQL as an external sidecar. | ||
|
|
||
| Operators should be able to reason about LARQL-backed slices through the same | ||
| dashboard, state, diagnostics, and logging surfaces used for MLX runners. | ||
|
|
||
| Skulk remains insulated from LARQL internals. The integration boundary is the | ||
| LARQL server process and its HTTP contract. | ||
|
|
||
| ## Rejected Alternatives | ||
|
|
||
| ### Sidecar | ||
|
|
||
| A sidecar would be quick to prototype, but it moves process lifecycle, | ||
| readiness, logs, and crash recovery outside Skulk. That creates a second | ||
| operator workflow and makes slice placement harder to explain and diagnose. | ||
|
|
||
| ### In-tree Port | ||
|
|
||
| Reimplementing LARQL's slice protocol inside Skulk would be a large, slow fork | ||
| of upstream LARQL. It would also make it harder to pick up future LARQL | ||
| improvements in vindex format, server behavior, and FFN/expert endpoints. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # ADR 0002: MLX Head With LARQL Cold Tier | ||
|
|
||
| Status: Accepted for planning; gated by Phase 3 feasibility | ||
|
ttupper92618 marked this conversation as resolved.
|
||
| Date: 2026-05-12 | ||
| Parent roadmap: https://github.com/Foxlight-Foundation/Skulk/issues/173 | ||
| Tracking issue: https://github.com/Foxlight-Foundation/Skulk/issues/154 | ||
| Gate issues: https://github.com/Foxlight-Foundation/Skulk/issues/161 and https://github.com/Foxlight-Foundation/Skulk/issues/162 | ||
| Source plan: https://github.com/Foxlight-Foundation/Skulk/blob/claude/understand-larql-repo-sJqA1/docs/slice-placement-and-vindex-publisher-plan.md | ||
|
ttupper92618 marked this conversation as resolved.
|
||
|
|
||
| ## Context | ||
|
|
||
| Skulk's strongest runtime path is MLX on Apple Silicon. Slice placement should | ||
| extend that path instead of replacing it. The desired architecture lets a Mac | ||
| head node keep the hot attention/router path local while RAM-rich commodity | ||
| peers serve cold FFN or expert weights from LARQL vindexes. | ||
|
|
||
| This decision is expensive to reverse once placement state, runner assignment, | ||
| and API surfaces begin to encode slice responsibilities. | ||
|
|
||
| ## Decision | ||
|
|
||
| The MLX runner remains the head runtime for slice mode. It owns the standard | ||
| MLX weights needed for the head role: embeddings, attention, norms, router, and | ||
| any locally assigned layers. It delegates selected per-layer FFN or expert work | ||
| to `LarqlRunner` peers over HTTP. | ||
|
|
||
| The MLX head never loads a vindex. Vindexes are cold-tier artifacts consumed by | ||
| LARQL peers. | ||
|
|
||
| The default wire format for delegated tensors is f16. i8 remains an explicit | ||
| future opt-in only where the LARQL contract supports it and Skulk can preserve | ||
| correctness. | ||
|
|
||
| ## Feasibility Gate | ||
|
|
||
| This ADR is accepted for planning, not yet accepted for irreversible runtime | ||
| implementation. Phase 3 must prove that Skulk's MLX path can delegate a | ||
| per-layer FFN or expert step and continue generation with acceptable overhead. | ||
|
|
||
| If MLX does not expose usable hooks and a manual forward-pass split is too | ||
| fragile, too invasive, or too slow, this ADR must be superseded before Phase 4 | ||
| slice-placement work starts. | ||
|
|
||
| ## Consequences | ||
|
|
||
| Existing MLX single-node and MLX pipeline placement remain the default path for | ||
| models that fit on the selected head node. | ||
|
|
||
| Slice placement is additive. It is only considered when the normal MLX path | ||
| cannot fit the model or when the operator explicitly chooses a slice-mode flow | ||
| in future UI/API work. | ||
|
|
||
| The slice plan must identify which LARQL peer serves which preset, layer range, | ||
| expert range, and vindex URI so the MLX head can dispatch remote FFN/expert | ||
| calls deterministically. | ||
|
|
||
| ## Rejected Alternatives | ||
|
|
||
| ### Replace the Head Runtime With LARQL | ||
|
|
||
| Replacing MLX would discard Skulk's current strongest execution path and make | ||
| Apple Silicon performance dependent on a new serving stack. | ||
|
|
||
| ### Load Vindexes on the Head | ||
|
|
||
| Loading vindexes on the head duplicates cold-tier storage and undermines the | ||
| purpose of using commodity RAM-rich peers for dormant weights. | ||
|
|
||
| ### General Remote-Compute Abstraction | ||
|
|
||
| The v1 design targets LARQL's concrete FFN/expert server contract. A generic | ||
| remote execution abstraction would add surface area before Skulk has proven the | ||
| basic slice-mode value proposition. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # ADR 0003: Vindex Provenance | ||
|
|
||
| Status: Accepted for planning | ||
| Date: 2026-05-12 | ||
| Parent roadmap: https://github.com/Foxlight-Foundation/Skulk/issues/173 | ||
| Tracking issue: https://github.com/Foxlight-Foundation/Skulk/issues/155 | ||
| Publisher issue: https://github.com/Foxlight-Foundation/Skulk/issues/156 | ||
| Source plan: https://github.com/Foxlight-Foundation/Skulk/blob/claude/understand-larql-repo-sJqA1/docs/slice-placement-and-vindex-publisher-plan.md | ||
|
ttupper92618 marked this conversation as resolved.
|
||
|
|
||
| ## Context | ||
|
|
||
| LARQL vindexes are directory-shaped artifacts derived from source model | ||
| weights. Extracting them can require substantial scratch disk, time, and | ||
| toolchain setup. Running that extraction on every Skulk user's machine would | ||
| make first use slow and operationally fragile. | ||
|
|
||
| Skulk already has model-store and download concepts for consuming artifacts. | ||
| The clean boundary is to make Skulk a vindex consumer and move extraction into | ||
| a dedicated publisher workflow. | ||
|
|
||
| ## Decision | ||
|
|
||
| Skulk will consume vindexes from HuggingFace URIs such as `hf://...`. Skulk will | ||
| not extract vindexes in-tree. | ||
|
|
||
| Extraction, publication, manifest curation, and catalogue governance live in a | ||
| separate sibling repository: `skulk-vindex-publisher`. | ||
|
|
||
| The publisher repository owns scheduled LARQL extraction and publication jobs. | ||
| Skulk owns runtime consumption, local caching/staging, and placement metadata. | ||
|
|
||
| ## Consequences | ||
|
|
||
| Skulk does not add a Rust toolchain or LARQL extraction dependency to its normal | ||
| runtime setup. | ||
|
|
||
| Future model-store work must support directory-shaped vindex artifacts, but it | ||
| does not need to know how to produce them. | ||
|
|
||
| The published vindex URI convention becomes part of the contract between the | ||
| publisher repo and Skulk's placement/runtime code. | ||
|
|
||
| ## Rejected Alternatives | ||
|
|
||
| ### Extract Inside Skulk | ||
|
|
||
| This would push heavyweight extraction work onto every operator machine, | ||
| including laptops that only need to run inference. It also expands Skulk's | ||
| runtime dependency surface for a build-time artifact-production task. | ||
|
|
||
| ### No Curated Catalogue | ||
|
|
||
| Without a curated catalogue, users would need to find community vindexes or | ||
| produce their own before slice mode is useful. That weakens the operator | ||
| experience and makes supported-model behavior harder to reproduce. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Architecture Decision Records | ||
|
|
||
| Architecture Decision Records capture choices that are expensive to reverse | ||
| once they affect Skulk's event-sourced state, runner taxonomy, placement | ||
| contracts, public APIs, or operator workflow. | ||
|
|
||
| ## Format | ||
|
|
||
| Each ADR uses this structure: | ||
|
|
||
| - **Status:** Proposed, Accepted for planning, Accepted, Superseded, or Rejected. | ||
| - **Context:** The forces that made the decision necessary. | ||
| - **Decision:** The choice Skulk will implement. | ||
| - **Consequences:** Operational and implementation effects of the decision. | ||
| - **Rejected alternatives:** Options considered and why they were not chosen. | ||
|
|
||
| ## Numbering | ||
|
|
||
| Use monotonically increasing four-digit filenames: | ||
|
|
||
| ```text | ||
| 0001-short-title.md | ||
| 0002-short-title.md | ||
| ``` | ||
|
|
||
| Do not renumber existing ADRs. If a decision changes, add a new ADR that | ||
| supersedes the old one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.