docs(stack): canonical stack handbook for agent onboarding - #129
Closed
samo-agent wants to merge 8 commits into
Closed
docs(stack): canonical stack handbook for agent onboarding#129samo-agent wants to merge 8 commits into
samo-agent wants to merge 8 commits into
Conversation
Pins the expected behaviour before any implementation exists: - dblab-module.test.ts: asserts the full cloud-init output installs Docker CE + zfsutils, creates sparse ZFS pool, runs dblab_server 4.1.3 on 127.0.0.1:2345, caps ARC at 256 MB, sets clone cap 2 + 128 MB shared_buffers + 1 GB memory limit, creates 2 GB swapfile, and CRITICALLY never disables fail2ban or drops the UFW SSH rule while keeping the control-plane IP in fail2ban ignoreip. - dblab-env-gate.test.ts: asserts env-create for db=dblab fails LOUD (exit 1) and writes NO env record when the dblab engine is BLOCKED or UNKNOWN, and proceeds normally when READY. Both files fail for the right reasons: dblab.ts does not exist yet (module import error) and the gate logic is absent from runEnvCreate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the minimal verified €0 dblab profile from #127 into the samohost cloud-init provisioning pipeline. ## What it installs (cloud-init runcmd, in order) - Docker CE from the official apt repo + containerd.io; adminUser added to the docker group. - Docker images pre-pulled: dblab-server:4.1.3 + extended-postgres:16-0.6.2. - 2 GB swapfile at /swapfile (/etc/fstab persisted). - ZFS ARC cap: /etc/modprobe.d/zfs.conf → options zfs zfs_arc_max=268435456 (256 MB); also applied live to /sys/module/zfs/parameters/zfs_arc_max and baked into initrd via update-initramfs. - Sparse 10G loopback file at /var/lib/dblab-pool/dblab.img; attached to /dev/loop0; zpool create dblab; dataset dblab/dblab_pool with compression=on atime=off logbias=throughput. - /etc/systemd/system/dblab-loopback.service (Before=zfs-import.target docker.service) re-attaches the loop device on every boot. - /root/.dblab/engine/configs/server.yml: port 2345, enableTelemetry=false, maxCloneCount=2, maxIdleMinutes=20160, cloneAccessAddresses=127.0.0.1, portPool 6000-6099, containerResources.memory=1073741824, shared_buffers 128MB, source=host PG via Docker bridge 172.17.0.1:5432, logical retrieval jobs (logicalDump/logicalRestore/logicalSnapshot), skipStartRefresh=true. - /root/.dblab/engine/startup.sh: busybox-grep workaround (apk add grep). - Verification token generated at runtime (openssl rand -hex 32, stored at /root/.dblab/token 0600, substituted into server.yml). - pg_hba.conf entry for 172.17.0.0/16 (Docker bridge) + listen_addresses patch; postgresql reloaded. - docker run dblab_server: 127.0.0.1:2345:2345, --privileged, rshared, --restart on-failure, startup.sh. - CLI installed via dblab.sh (DBLAB_CLI_VERSION=4.1.3); symlinked into ~/bin/dblab; PATH exported in /root/.bashrc. - CLI inited: dblab init --environment-id solo. - Waits up to 60s for healthz, then verifies it. - Initial refresh triggered (dblab instance snapshots refresh). - systemctl mask ssh.socket (prevent Ubuntu 24.04 socket-activation loop). ## fail2ban preserved (#127 critical lesson) The module NEVER emits any fail2ban disable/stop command or ufw delete. The hardening module (always rendered first) keeps fail2ban enabled and writes jail.local with the control-plane IP in ignoreip via trustedIps. The combined output keeps fail2ban enabled with the whitelisted IP. ## Module registration resolveModules now has a MODULE_REGISTRY map. "dblab" → dblabModule. Unknown module names still return an error (back-compat). ## env-create honesty gate EnvExecDeps gains optional dblabPreflight?: (vm) => Promise<"READY"|"BLOCKED"|"UNKNOWN">. runEnvCreate gates on it when target.dbBackend === "dblab": - BLOCKED/UNKNOWN → exit 1, clear message with VM name and port, NO record written. - preflight throws → exit 1, exception detail in the message, NO record written. - READY → proceeds as before. - dep absent → back-compat, no gate (all existing tests unaffected). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Asserts the root-caused bugs from samograph's dead-engine incident (task wdgv5zs85) against the module before any fix is applied: 1. server.yml MUST have poolManager.preSnapshotSuffix: "_pre" Without it DBLab runs "zfs list | grep -v ''" — BusyBox grep rejects an empty -v pattern (exit 2) → "no available pools" → zero clones. 2. poolManager MUST use selectedPool: (correct yaml struct tag, not pool:) 3. logicalDump source dbname MUST be config-driven via makeDblabModule() factory — preview envs need <app>_template, not hardcoded _prod. 4. defaultEnvExecDeps() MUST export a real dblabPreflight function (currently absent → env-create honesty gate is inert in prod). All 10 tests fail (RED) against the current PR-head. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…onfig-driven sourceDb, preflight wired to prod (#128) Root cause (samograph dead-engine incident, task wdgv5zs85): The generated server.yml was missing poolManager.preSnapshotSuffix. Without it DBLab runs "zfs list ... | grep -v ''" — BusyBox grep (Alpine base image) rejects an empty -v pattern with exit 2 → "no available pools" → the engine cannot see the ZFS pool → zero clones on first boot. Fix 1 (CRITICAL): Add poolManager.preSnapshotSuffix: "_pre" to the YAML template generated by buildServerYml(). Converts "grep -v ''" (exit 2, BusyBox rejects empty pattern) into "grep -v '_pre'" (works on all greps). Fix 2: Change pool: → selectedPool: in poolManager. The old key was silently ignored by DBLab; the correct yaml struct tag is selectedPool. Harmless but correct per the v4.1.3 schema. Fix 3: Source DB is now config-driven via makeDblabModule({ sourceDb }). Preview envs need <app>_template (not <app>_prod) as the retrieval source. The new makeDblabModule() factory bakes the correct DB name into server.yml at provisioning time. dblabModule keeps a ${DBLAB_SOURCE_DB} placeholder for back-compat (operator fills it in manually). Fix 4: Wire dblabPreflight into defaultEnvExecDeps() so the env-create honesty gate fires in production. Previously dblabPreflight was absent from defaultEnvExecDeps() — the gate was silently skipped in prod, allowing fake dblab env records to be written against a non-running engine (samograph's broken path). The production implementation runs the full DBLAB_PROBES audit script over a pinned SSH connection and reuses evaluateDblabPreflight() for the verdict so the gate and `env preflight` cannot diverge. All 10 RED tests now pass GREEN. 89 dblab/preflight/gate tests pass total. Pre-existing smol-toml dependency errors in this test environment are unaffected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add test/dblab-schema-yaml-128.test.ts with js-yaml-powered structural
assertions to catch the 4 bugs found in the live smoke-provision:
BUG-1 cloneAccessAddresses emitted as sequence instead of scalar
BUG-2a skipStartRefresh at retrieval.spec (wrong nesting)
BUG-2b jobs list at retrieval.spec.jobs (wrong level)
BUG-3 losetup hardcodes /dev/loop0
BUG-4 ${DBLAB_SOURCE_USER} never substituted
All 14 new assertions fail RED on current PR head — toContain() string
checks in the old tests let invalid YAML structure through undetected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…128 Root cause: smoke-provision on a real VM produced "[FATAL] failed to parse config" because the generated server.yml had 4 structural bugs vs the known- good samograph config (ssh -p 2223 samo@116.203.249.135). Fixes applied: BUG-1 provision.cloneAccessAddresses: scalar "127.0.0.1" (was YAML sequence) BUG-2a retrieval.refresh.skipStartRefresh (was retrieval.spec.skipStartRefresh) BUG-2b retrieval.jobs: [logicalDump, logicalRestore, logicalSnapshot] flat list under retrieval.jobs (was inline-options objects under retrieval.spec.jobs) job options now correctly at retrieval.spec.<job>.options BUG-3 losetup -f --show picks the next free loop device dynamically; Ubuntu 24.04 + snapd occupy loop0-2 so hardcoding /dev/loop0 fails; zpool create updated to use $LOOP_DEV; dblab-loopback.service uses losetup -j check + losetup -f on re-attach BUG-4 ${DBLAB_SOURCE_USER} substituted with adminUser from ProvisionSpec at build time (passed through buildFragment → buildServerYml); leaving the literal placeholder caused silent refresh failure behind || true All 1449 tests pass; 14 new YAML-structural assertions GREEN. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CloudInitFragment has writeFiles?: and runcmd?: (optional); guard with ?? [] in test/dblab-schema-yaml-128.test.ts to pass bunx tsc --noEmit. No test logic changed; 1449 tests still pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds docs/stack/ — six component docs (postgres, dblab, supabase, zfs, routing, deploy) plus a README with read-order and non-negotiable owner policies. All facts are ground-truth verified from repos and live VM state (audit 2026-07-07); no memory-only claims. Wires the handbook into CLAUDE.md via a SAMO-STACK sync block so every agent session loads the pointer on open. Resolves the PG-version contradiction: 18 is the single documented standard; the samo.team control-plane PG 17 is documented as a known gap. Captures Supabase real state (no full stack deployed anywhere; GoTrue-only on samo.team; plain bcrypt on field-record). Documents the cx23 loopback DBLab minimal profile (verified on samograph 2026-07-07, ~EUR 0 incremental). References: ground-truth audit for #127 (SAMO client package standard). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/stack/— six component docs + README wired into CLAUDE.md as aSAMO-STACKsync block that agents load on session openWhat's in each file
docs/stack/README.mddocs/stack/postgres.mddocs/stack/dblab.mddocs/stack/supabase.mddocs/stack/zfs.mdtank, two shapes (volume-backed vs loopback cx23), PG-on-tank constraint, ARC cap requirementdocs/stack/routing.mddocs/stack/deploy.mdKey contradictions resolved
Wiring into agent context
CLAUDE.mdgets a<!-- SAMO-STACK:START/END -->block (above the existingSAMO-DEV-PRINCIPLESblock) with a pointer todocs/stack/README.mdand a quick-reference of the five most common agent mistakes. Every Claude Code session opening this repo will load the pointer immediately.Not merged — per brief
Ground-truth audit for #127.