From e62f36fc5e16edd6da557141e0b84388d5222c65 Mon Sep 17 00:00:00 2001 From: Fillianore Date: Fri, 17 Jul 2026 17:57:02 +0800 Subject: [PATCH] feat: add lbg-cli skill for Bohrium sandbox management MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New lbg-cli skill: - Full sandbox lifecycle: create, exec, terminal, files, kill, list, describe - references/sandbox/ subdocs: lifecycle, exec modes, templates, terminal, files, network (proxy toggle), pitfalls (30Gi overlay, 60s timeout, etc.) - references/images.md — image commit/build workflow - references/platform-snapshot.md — SKU/machine catalog - references/software-install/ — mirrors and pre-installed packages Dependency bump: - lbg>=4.0.0b56 (was >=1.2.29) for new sdbx subcommands --- pyproject.toml | 2 +- src/matcreator/skills/lbg-cli/SKILL.md | 137 ++++++++ .../skills/lbg-cli/references/images.md | 113 +++++++ .../lbg-cli/references/platform-snapshot.md | 56 ++++ .../skills/lbg-cli/references/sandbox/exec.md | 40 +++ .../references/sandbox/execution-modes.md | 66 ++++ .../lbg-cli/references/sandbox/files.md | 86 +++++ .../lbg-cli/references/sandbox/lifecycle.md | 317 ++++++++++++++++++ .../lbg-cli/references/sandbox/network.md | 172 ++++++++++ .../lbg-cli/references/sandbox/pitfalls.md | 77 +++++ .../lbg-cli/references/sandbox/templates.md | 103 ++++++ .../lbg-cli/references/sandbox/terminal.md | 51 +++ .../references/software-install/README.md | 26 ++ .../references/software-install/mirrors.md | 51 +++ .../software-install/pre-installed.md | 16 + .../skills/lbg-cli/skill-manifest.json | 63 ++++ 16 files changed, 1375 insertions(+), 1 deletion(-) create mode 100644 src/matcreator/skills/lbg-cli/SKILL.md create mode 100644 src/matcreator/skills/lbg-cli/references/images.md create mode 100644 src/matcreator/skills/lbg-cli/references/platform-snapshot.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/exec.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/execution-modes.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/files.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/lifecycle.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/network.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/pitfalls.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/templates.md create mode 100644 src/matcreator/skills/lbg-cli/references/sandbox/terminal.md create mode 100644 src/matcreator/skills/lbg-cli/references/software-install/README.md create mode 100644 src/matcreator/skills/lbg-cli/references/software-install/mirrors.md create mode 100644 src/matcreator/skills/lbg-cli/references/software-install/pre-installed.md create mode 100644 src/matcreator/skills/lbg-cli/skill-manifest.json diff --git a/pyproject.toml b/pyproject.toml index 592318d6..ef6c3e86 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -12,7 +12,7 @@ dependencies = [ "rdkit>=2025.9.1", "google-adk>=1.28.0", "google-cloud-storage", - "lbg>=1.2.29", + "lbg>=4.0.0b56", "litellm>=1.77.4", "periodictable>=2.0.2", "pyyaml>=6.0.2", diff --git a/src/matcreator/skills/lbg-cli/SKILL.md b/src/matcreator/skills/lbg-cli/SKILL.md new file mode 100644 index 00000000..df898aa0 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/SKILL.md @@ -0,0 +1,137 @@ +--- +name: lbg-cli +description: >- + CLI for operating Bohrium cloud sandboxes (sdbx), managing templates, + committing container images, and installing software inside sandboxes. + Use when the user mentions lbg, sdbx, sandbox, or any Bohrium sandbox + operation. +--- + +# lbg CLI — Sandbox (sdbx) + +CLI for managing **cloud sandboxes (sdbx)** on the Bohrium platform. +Covers sandbox lifecycle (`create`, `exec`, `kill`, `list`, `describe`), +interactive PTY sessions, file transfer, template management, image +commit/build, and software installation inside sandboxes. + +> **Proxy toggle.** Sandboxes default to no outbound HTTP proxy. +> For overseas access (PyPI `pypi.org`, GitHub, HuggingFace, etc.), toggle +> the `pai.ga.op.xdptech.com` proxy on with the snippet in +> [`references/sandbox/network.md`](references/sandbox/network.md), +> then toggle it off when done. Domestic access via the image-level Aliyun +> mirror is unaffected. + +> **Default-behavior pitfalls (read first for ML / long-running jobs).** +> Several defaults are fine for quick interactive use but bite training +> workloads: the overlay disk is **fixed at 30Gi** (SKU has no disk field), +> `exec` foreground **`--timeout` is 60s**, `files read` transfers as **text** +> (corrupts binaries), `kill` is **irreversible** and a stopped sandbox's +> files can't be read, the outbound proxy is **off**, and a template image +> with a mutable `:latest` tag is **rejected** (it churns the prewarmed image +> cache — use an immutable tag). See the consolidated list +> in +> [`references/sandbox/pitfalls.md`](references/sandbox/pitfalls.md) before +> driving a sandbox through an agent. + +## Prerequisites + +```bash +lbg login --ak +``` + +Or set `BOHRIUM_ACCESS_KEY` in the environment, or pass `--api-key` per call. + +## Default sandbox template + +`lbg sdbx create` with no template argument creates a sandbox from the +`sdbxagent` template — `pytorch20-scicomp:1.0.1` with the image-level Aliyun +PyPI mirror, so domestic `pip install` is fast out of the box. The outbound +HTTP proxy is **off by default**; for overseas access toggle it on demand via +the `proxy on` / `proxy off` snippets in +[`references/sandbox/network.md`](references/sandbox/network.md). + +For GPU work, pass `--gpu` as a shortcut: bare `--gpu` resolves to +`scicomp-4090` (the default GPU template); `--gpu 5090` and `--gpu l20` pick +the other two GPU templates. Mutually exclusive with the positional +`template` argument. + +## Command surface + +| Command | Purpose | +| --- | --- | +| `lbg login` / `logout` | Persist a Bohrium access key | +| `lbg sdbx` | Operate cloud sandboxes (sdbx) | +| `lbg sdbx image` | Commit sandbox snapshots / build from Dockerfile | +| `lbg skill` | Export / diff / update this agent skill | + +## Skill self-maintenance + +Keep the installed copy of this skill current so agents never operate on stale +sandbox / sdbx docs. Each export writes a `skill-manifest.json` (lbg version, +export time, per-file `sha256`). + +| Subcommand | Description | +| --- | --- | +| `skill export [--output ]` | Write the skill tree to `/lbg-cli/` (fails on conflict) | +| `skill diff --target ` | Report drift vs the bundled version; exits non-zero when out of sync (`--json` for a machine-readable report) | +| `skill update --target --backup` | Snapshot the old copy to `.bak.`, then reinstall (`--force` overwrites in place; `install` is an alias) | + +`--target` points at the skill directory itself (e.g. +`~/.codex/skills/lbg-cli`); `--output` on `export` is the parent directory. + +## Sandbox commands + +| Subcommand | Description | +| --- | --- | +| `create [template]` | Launch a new sandbox (default: `sdbxagent` template) | +| `exec ` | Run a command inside a sandbox (foreground / background); **always pass `--user root`** | +| `kill ` | Terminate a sandbox | +| `list` | List active sandboxes | +| `describe ` | Get sandbox details (incl. IP, status, template) | +| `terminal ` | Open an interactive PTY session | +| `upload ` | Upload files or directories | +| `download [dst]` | Download files or directories | +| `image commit` | Snapshot a running sandbox's filesystem | +| `image build` | Build an image from a Dockerfile | +| `image get ` | Get image build status | +| `image ls` | List images (filterable by sandbox, status, name) | +| `image build-log ` | Fetch or follow build logs | +| `template create` | Create a new sandbox template | +| `template list` | List available templates | +| `template update` | Update a template | +| `template delete` | Delete a template | + +## Discovering commands + +```bash +lbg sdbx --help +lbg sdbx --help +``` + +## Reference docs + +Deep usage docs under `references/`. Load only what's relevant. + +**Sandbox lifecycle & usage:** + +- [`references/sandbox/pitfalls.md`](references/sandbox/pitfalls.md) — default-behavior traps (disk 30Gi, timeouts, binary downloads, kill, proxy, billing) + an ML-training checklist +- [`references/sandbox/lifecycle.md`](references/sandbox/lifecycle.md) — create / list / describe / kill, plus best practices +- [`references/sandbox/execution-modes.md`](references/sandbox/execution-modes.md) — foreground vs background vs PTY, plus the retrieve-before-kill SOP for long-running jobs +- [`references/sandbox/exec.md`](references/sandbox/exec.md) — `lbg sdbx exec` usage, including `--background` +- [`references/sandbox/terminal.md`](references/sandbox/terminal.md) — PTY sessions for REPLs / TUIs / Ctrl-C interaction +- [`references/sandbox/files.md`](references/sandbox/files.md) — upload / download files and directories +- [`references/sandbox/templates.md`](references/sandbox/templates.md) — list, create, update, and delete sandbox templates +- [`references/sandbox/network.md`](references/sandbox/network.md) — on-demand `pai.ga.op.xdptech.com` HTTP proxy toggle + +**Container images (sandbox snapshots & builds):** + +- [`references/images.md`](references/images.md) — `lbg sdbx image commit` / `image build` for producing container images + +**Software install (inside sandboxes):** + +- [`references/software-install/README.md`](references/software-install/README.md) — pre-installed inventory + domestic mirrors for sandbox installs + +## Output formats + +`lbg sdbx` commands accept `--json` for machine-readable output suitable +for scripting and agent use. \ No newline at end of file diff --git a/src/matcreator/skills/lbg-cli/references/images.md b/src/matcreator/skills/lbg-cli/references/images.md new file mode 100644 index 00000000..e87577dd --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/images.md @@ -0,0 +1,113 @@ +# Image commit & dockerfile build + +`lbg sdbx image commit` / `image build`: produce container images. + +Two ways to mint a new container image; both are async, both surface in +`lbg sdbx image ls`, and the `buildType` column tells them apart: + +| Subcommand | When to use | Source | +|---|---|---| +| `image commit` | snapshot a *running* sandbox's filesystem | needs `--sandbox-id` | +| `image build` | reproducible build from source | needs `--dockerfile ` | + +Both jobs return the new image record's `id` *and* its final `imageUrl` +**immediately** on submission — the URL is reserved up-front so it can +be plugged into a later `lbg sdbx template create --image ` +without polling. Polling `get ` is only needed if you actually want +to wait for `success` / `failed` before consuming the image. + +```bash +# A) commit — snapshot a running sandbox +lbg sdbx image commit \ + --sandbox-id \ + --name \ + --desc "training-day-3 snapshot" \ + --project-id \ + --json + +# B) build — from a local Dockerfile (no sandbox needed) +lbg sdbx image build \ + --dockerfile ./Dockerfile \ + --name \ + --desc "pinned cuda 12.1" \ + --project-id \ + --json + +# B') tag the build with the sandbox it originated from (provenance only) +lbg sdbx image build \ + --dockerfile ./Dockerfile \ + --name \ + --sandbox-id \ + --project-id + +# poll either kind (status: 0=creating 1=pending 2=success 3=failed) +lbg sdbx image get --json + +# fetch the kaniko log of a dockerfile build (commit records have no log) +lbg sdbx image build-log # raw text snapshot to stdout, pipe-friendly +lbg sdbx image build-log --json # {"id": ..., "log": "..."} +lbg sdbx image build-log --follow # stream chunks as kaniko writes them, + # auto-EOFs when the build reaches a + # terminal status (success/failed). + # Mutually exclusive with --json. + +# browse your images across all sandboxes / build types +lbg sdbx image ls # default table (incl. statusReason) +lbg sdbx image ls --json # full envelope (items+page+total) +lbg sdbx image ls --sandbox-id # one sandbox's records +lbg sdbx image ls --status 2 --status 3 # successes + failures +lbg sdbx image ls --name train --start-time '2026-04-01 00:00:00' +lbg sdbx image ls -q # ids only, one per line +``` + +## Naming — avoid a `latest` image name + +`--name` is the **user-visible portion** of the image name; the backend may +prepend a user/project prefix and controls the final registry tag, so the CLI +does *not* hard-reject it here. But if the name ends in `latest` +(`--name latest`, `--name img:latest`, `--name img:dev-latest`, …) the CLI +prints a **stderr hint** (stdout / `--json` stay clean) nudging you toward an +immutable/unique name. Reason: a `latest` image is a poor template base — +`lbg sdbx template create/update` **rejects** a `:latest` template image, and +the platform's prewarmed image cache cannot be reused for a mutable tag. Prefer +a version (`:v1`) or a date/build tag so the image you just minted can be +plugged straight into `template create --image` without a rename. + +## Status & failure reason + +`status` is the coarse lifecycle (`0=creating 1=pending 2=success 3=failed`). +For `failed` records, `statusReason` is a **structured machine-readable code** +that explains *why* it failed — pair it with the free-form `errorMsg` for the +full picture. The field is `omitempty`: it never appears on success rows, and +it is also absent on historical failed rows created before this column was +introduced (no backfill). + +| `statusReason` | What happened | +|---|---| +| `build_timeout` | Dockerfile build did not finish within the backend timeout window (default 1800s) — typically still in `ImageBuildPushing[6]` when the deadline hit. Run `lbg sdbx image build-log ` to see how far kaniko got. | +| `kaniko_failed` | mid-lbg-image reported a non-zero kaniko exit (image-not-found, RUN command non-zero, etc.). `lbg sdbx image build-log ` shows the raw kaniko log. | +| `commit_failed` | ACS Commit CRD reported `phase=Failed`. `errorMsg` carries the controller-level reason; there is no kaniko-style stdout log for commit builds. | +| `commit_timeout` | Sandbox-commit did not reach `Succeeded` within the backend timeout window. | +| `register_failed` | Commit succeeded and the image is already in the registry, but the post-commit `RegisterImage` call to mid-lbg-image failed (rare, usually transient — safe to retry). | +| `crd_create_failed` | Commit CRD could not be created (sandbox missing, permissions, or a same-name CRD already exists). | + +Use `--follow` only with **dockerfile** builds (`buildType=2`); the CLI rejects +`build-log --follow` on commit records (`buildType=1`) immediately, because +commit builds run inside Alibaba Cloud ECI and only expose ~5 coarse phase +messages, not per-line kaniko stdout. Inspect commit failures via `image get +` (`statusReason` + `errorMsg`) instead. + +Both `commit` and `build` require `projectId` in the request body. The +CLI sends the same value both in the body and as the `X-PROJECT-ID` +header, so either channel — `--project-id ` or any future global +X-PROJECT-ID source picked up by `SdbxSettings.project_id` — satisfies +the requirement. If neither is set, the CLI refuses the call with +`missing project id` (exit 2) instead of letting the backend return a +generic 400. + +`image build` reads the Dockerfile locally and sends it inline; the +hard cap is **64 KiB** (matches the launching backend's +`BohrSandboxImageBuildReq` limit). The CLI rejects oversized files +client-side with exit 2. The `--json` echo of the request elides the +Dockerfile body — only `dockerfileSize` is shown — so logs stay clean; +the response (`id`, `imageUrl`) is unmodified. diff --git a/src/matcreator/skills/lbg-cli/references/platform-snapshot.md b/src/matcreator/skills/lbg-cli/references/platform-snapshot.md new file mode 100644 index 00000000..d7d8048f --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/platform-snapshot.md @@ -0,0 +1,56 @@ +# Platform snapshot (as of 2026-04-30) + +Pinned CPU / GPU template shortcuts (ephemeral; review by 2026-06-01). + +> Pinned snapshot of platform-side values that may drift over time. +> Everything in this section is ephemeral — review/remove after +> 2026-06-01 or when these stabilise. For a live view, use the +> commands noted below. + +## CPU template shortcuts + +Platform-managed CPU template shortcuts. Pass the `name` directly to +`lbg sdbx create`. Resource sizes are observed `cpuCount` / `memoryMB` +from a create response and may change. + +| name | cpuCount | memoryMB | +| --- | --- | --- | +| `sdbx-cpu-mini` | 1 | 3200 | +| `sdbx-cpu-small` | 2 | 5248 | +| `sdbx-cpu-medium` | 4 | 9344 | +| `sdbx-cpu-large` | 16 | 33920 | + +## GPU template shortcuts + +Platform-managed GPU template shortcuts. Pass the `name` directly to +`lbg sdbx create`, or use the `--gpu` shortcut on the CLI. GPU shortcuts +are platform-managed and may change over time — consult +`lbg sdbx template ls` for the current list. + +| name | image | sku | `--gpu` key | +| --- | --- | --- | --- | +| `scicomp-4090` | `pytorch20-scicomp:1.0.6` | `c16_m64_1 * NVIDIA 4090` | `--gpu` (default) or `--gpu 4090` | +| `scicomp-5090` | `pytorch20-scicomp:1.0.6` | `c16_m64_1 * NVIDIA 5090` | `--gpu 5090` | +| `scicomp-l20` | `pytorch20-scicomp:1.0.6` | `c16_m128_1 * NVIDIA L20` | `--gpu l20` | + +`--gpu` is mutually exclusive with passing a positional template name. +Bare `--gpu` resolves to `scicomp-4090` (the default GPU template). + +## SKU catalog + +SKU availability is dynamic. Use the live list instead of copying values +from documentation: + +```bash +lbg sdbx machine list # list default SKU category +lbg sdbx machine list -c gpu # list GPU SKUs +lbg sdbx machine list --json # machine-readable SKU data +``` + +Pass the returned `sku_name` value to `lbg sdbx template create --sku-name`. +Quote SKU names that contain spaces or shell metacharacters. + +## Example image + +- Example image: `registry.dp.tech/dptech/abacus:LTSv3.10.1` (live list: + `lbg image ls`). diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/exec.md b/src/matcreator/skills/lbg-cli/references/sandbox/exec.md new file mode 100644 index 00000000..0b4a3892 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/exec.md @@ -0,0 +1,40 @@ +# Sandbox exec + +`lbg sdbx exec` usage, including `--background` mechanics and `--user`. + +For the foreground vs background vs PTY mental model and the retrieve-before-kill SOP, see [`execution-modes.md`](./execution-modes.md). + +## CRITICAL — always pass `--user root` + +**Every `lbg sdbx exec` call MUST include `--user root`.** Without it, the +command runs as the sandbox's default non-root user (e.g. uid 1001), which +cannot read files uploaded via `--ti` (owned `root:root`, mode 640), cannot +write to system paths, and cannot install software. There is no scenario +where `--user root` should be omitted. + +```bash +lbg sdbx exec --user root 'cd /workspace && python train.py' +lbg sdbx exec --user root 'cat log.txt | grep ERROR | wc -l' +lbg sdbx exec --user root 'echo hello > /tmp/out.txt' +``` + +## exec command passing + +`exec` uses SSH-style pass-through: positional args are joined with spaces +and sent as a single shell string to `bash -l -c` inside the sandbox. +Shell operators work as written — no extra quoting is needed. + +## Background jobs (`--background`) + +`--background` returns immediately with a `pid` and lets the job keep +running. When `--background` is set, `--timeout` defaults to `0` +(unlimited); do **not** pass a finite `--timeout` unless you actually +want the job killed after that many seconds (the CLI prints a warning +in that case). Re-check state with `lbg sdbx ps ` or by +reading output files via `lbg sdbx files read`. + +```bash +lbg sdbx exec --background --user root 'python train.py > /tmp/out.log 2>&1' +lbg sdbx ps --json +lbg sdbx files read /tmp/out.log +``` diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/execution-modes.md b/src/matcreator/skills/lbg-cli/references/sandbox/execution-modes.md new file mode 100644 index 00000000..8e6163d4 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/execution-modes.md @@ -0,0 +1,66 @@ +# Execution modes + +Foreground vs background vs PTY: which `lbg sdbx` mode survives a local interruption, and how to retrieve outputs without losing them. + +## The three modes + +| Mode | Invocation | Local connection | Where it runs | Survives `--timeout` | Output retrieval | +|------|------------|------------------|---------------|----------------------|------------------| +| Foreground | `lbg sdbx exec --user root ''` | blocks until done | sandbox | dies at `--timeout` (default 60s); local disconnect drops captured output | inline `stdout` / `stderr` from the same call | +| Background | `lbg sdbx exec --background --user root ''` | returns immediately with `pid` | sandbox | unlimited by default (`--timeout 0`); finite `--timeout` kills the remote command at that boundary | read files the command wrote (`lbg sdbx files read`); pid status via `lbg sdbx ps` / `describe --with-processes` | +| Terminal (PTY) | `lbg sdbx terminal create / send / kill` | persistent PTY pid | sandbox | PTY pid lives until `terminal kill ` or sandbox destruction (per-PTY `--timeout` available) | redirect each command (`cmd > /tmp/out 2>&1\n`) and read the file with `lbg sdbx files read` | + +In all three cases the work happens inside the sandbox; what differs is the lifetime of the local connection and how the caller gets the output back. + +## When to use which + +- **Foreground** — quick, deterministic commands that finish well under a minute (`pwd`, `nvidia-smi`, `pip show `, a single test). The caller blocks, the result comes back in one round-trip. +- **Background** — anything that might exceed your local connection's reliable window: model training, long downloads, Dockerfile-equivalent setup, dataset preprocessing, long test suites. The remote process keeps running even if your terminal closes or the agent times out. +- **Terminal (PTY)** — interactive workloads that genuinely need a TTY: REPLs, TUIs (`htop`, `vim`), or sending Ctrl-C to a stuck process. Not a substitute for `exec` when you just want "run this, give me stdout". + +## Long-running jobs: the retrieve-before-kill SOP + +This is the safe sequence for any `--background` job whose outputs you actually need: + +1. **Decide where outputs go.** Pick a persistent path inside the sandbox — convention is `/workspace/out/`. Bake it into the command itself; do not rely on the default cwd. +2. **Launch with `--background`.** The default `--timeout 0` (unlimited) kicks in automatically; do not pass a finite `--timeout` unless you actually want the job killed at that boundary. + ```bash + lbg sdbx exec --background --user root 'mkdir -p /workspace/out && python train.py > /workspace/out/run.log 2>&1' + ``` + The command returns a `pid` (or `{"pid": N, ...}` under `--json`). Record it. +3. **Poll until done.** Check liveness with either of: + ```bash + lbg sdbx ps --json + lbg sdbx describe --with-processes --json + ``` + The `pid` disappears from `running_processes` when the command finishes. You can also tail the log file periodically: + ```bash + lbg sdbx files read /workspace/out/run.log + ``` +4. **Retrieve outputs BEFORE kill.** Pull every file you care about to local disk: + ```bash + lbg sdbx files read /workspace/out/run.log --output ./run.log + lbg sdbx files read /workspace/out/model.bin --format bytes --output ./model.bin + ``` +5. **Verify locally.** Confirm what you needed is on disk (size > 0, expected lines present, checksum matches). +6. **Then kill.** + ```bash + lbg sdbx kill + ``` + +**Why this matters.** The platform does not currently support reading files from a stopped sandbox. A `--background --timeout 0` job will run to completion remotely — but if you `kill` the sandbox before step 4, the contents of `/workspace/` are gone for good. Treat the sandbox as an ephemeral compute node: outputs you want to keep have to be pulled out before teardown. + +## Common mistakes + +- **Foreground for a 30-minute job.** The default 60s `--timeout` kills the captured output, and a flaky network can drop the connection long before that. Use `--background`. +- **`--background` then immediate `kill` with no `files read` in between.** The remote command may have succeeded — you still lost its output. Always retrieve first. +- **`--background --timeout 60` (or any finite value).** Setting a finite `--timeout` under `--background` kills the remote command at that boundary. The CLI prints a warning, but the job dies anyway. Pass `--timeout 0` (or omit `--timeout`) for true long-running work. +- **Treating `terminal send` as a run-and-capture.** `send` returns only `sent_bytes` — the PTY's stdout is not echoed back. For "run a command, get its output" use `exec`. When the workload truly needs a TTY, redirect each command (`cmd > /tmp/out 2>&1\n`) and read the file with `files read`. +- **Background script with multiple instances.** If the same background script is accidentally launched twice (e.g., due to a retry), both instances compete for the same frames and may corrupt output files. Guard against this with a lock file: `mkdir /workspace/.lock 2>/dev/null || { echo "Another instance running"; exit 1; }`. + +## See also + +- [`exec.md`](./exec.md) — `lbg sdbx exec` usage including `--background` +- [`terminal.md`](./terminal.md) — PTY sessions +- [`lifecycle.md`](./lifecycle.md) — `kill` semantics and best practices +- [`files.md`](./files.md) — `files read` / `files write` diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/files.md b/src/matcreator/skills/lbg-cli/references/sandbox/files.md new file mode 100644 index 00000000..3fe90791 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/files.md @@ -0,0 +1,86 @@ +# Sandbox files + +Upload and download files and directories. + +## Uploading files and directories + +`files write --source` accepts either a single file or a directory. When +given a directory, every regular file under it is uploaded in one batch +with relative paths preserved under the remote root. For very large +trees, tar locally then untar inside the sandbox via `exec`. + +```bash +lbg sdbx files write --source ./run.py /workspace/run.py --json +lbg sdbx files write --source ./project /workspace/project --json +``` + +## Fast uploads to /bohr-workspace (tiefblue, `--ti`) + +`/bohr-workspace` is not a sandbox-local disk — it is a **persistent, +session-scoped directory backed by object storage (tiefblue)**. Files written +there survive `kill` and are visible to any sandbox created with the same +session id. Pass `--ti` to upload a file/directory **straight to that object +storage** instead of streaming it through the sandbox filesystem: this is much +faster for large files (automatic multipart) and skips the sandbox hop. + +```bash +# a single file +lbg sdbx files write --ti --session-id --source ./model.pt \ + /bohr-workspace/model.pt --json +# a whole directory +lbg sdbx files write --ti --session-id --source ./out_dir \ + /bohr-workspace/out --json +``` + +Rules and caveats: + +- **`--session-id` is required** and must match the id the sandbox was created + with (`lbg sdbx create --session-id `). `/bohr-workspace` is bound to the + session, so the token is scoped per (user, session). +- Only paths under **`/bohr-workspace`** (or its sub-directories) are supported. + For any other path, upload without `--ti` (via the sandbox filesystem). +- `--content` is not supported with `--ti`; pass a real `--source` file or dir. +- **Directory uploads are recursive and have no dry-run or confirmation** — every + file under `--source` (including hidden files, `.git/`, `node_modules/`, and any + secrets like `.env`) is pushed and overwrites existing remote objects. Double-check + the source path and prune unwanted files first; for big/messy trees prefer + `tar` + `exec` instead. +- The `--json` result reports `object_key` (object-storage key), `sandbox_path` + (where it appears inside the sandbox), `host`, and `bytes` per file. +- Files uploaded via `--ti` appear inside the sandbox owned by `root:root` with + mode `640`, so a non-root sandbox user (e.g. uid 1001) cannot read them + directly — read them with `exec --user root`, or `chown`/`chmod` as root first. +- Other object-storage mounts may live on different tiefblue endpoints and are + not wired to `--ti` yet. + +## Downloading files + +`files read` prints a file to stdout, or writes it to local disk with +`--output`. + +```bash +lbg sdbx files read /workspace/result.csv # print text to stdout +lbg sdbx files read /workspace/result.csv --output ./result.csv # save to disk +lbg sdbx files read /workspace/model.pt --output ./model.pt # binary, auto-detected +``` + +### Text vs bytes (important for binary files) + +When `--format` is omitted it is **auto-detected from the file extension**: +known binary types (`.pt`, `.pth`, `.bin`, `.ckpt`, `.safetensors`, `.npy`, +`.tar`, `.gz`, `.zip`, `.parquet`, `.pdf`, `.so`, images, ...) are read as +**bytes**, everything else as **text**. A `binary file detected ...` warning +is printed to stderr when this happens. + +Reading a binary file as `text` runs it through a charset decode + UTF-8 +re-encode roundtrip that **silently corrupts the bytes** (wrong size, +`torch.load` / unzip fails with "invalid header or archive is corrupted"). +The auto-detection prevents this for known extensions; for an unrecognized +extension on a binary file, pass `--format bytes` explicitly: + +```bash +lbg sdbx files read /workspace/checkpoint.weights --format bytes --output ./checkpoint.weights +``` + +Force `--format text` only when you knowingly want the decoded string of a +binary file (rare). diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/lifecycle.md b/src/matcreator/skills/lbg-cli/references/sandbox/lifecycle.md new file mode 100644 index 00000000..d0185597 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/lifecycle.md @@ -0,0 +1,317 @@ +# Sandbox lifecycle + +Create, list, describe, and kill on-demand sandboxes; plus best practices. + +```bash +lbg sdbx doctor --json # verify configuration and SDK +lbg sdbx create my-template --json # create from explicit template (personal wallet) +lbg sdbx create my-template --project-id --json # bill the sandbox to a project budget +lbg sdbx create my-template --timeout 1800 --json # set sandbox auto-destroy lifetime (seconds) +lbg sdbx create my-template --never-timeout --json # opt out of auto-destroy entirely +lbg sdbx create my-template --mount-user-storage --json # mount caller's personal + share disks +lbg sdbx create my-template --project-id --mount-user-storage --share-subpath sub/dir --json # mount only a sub-dir of the project share disk +lbg sdbx create my-template --env HELLO=WORLD --env DEBUG=1 --json # inject environment variables +lbg sdbx create my-template --image --json # swap the template's container image in place +lbg sdbx create my-template --session-id --json # tag with a session id (shared user-storage subPath) +lbg sdbx list --json # list your sandboxes +lbg sdbx describe --with-processes --json # metadata + running processes +lbg sdbx ps --json # list processes in a sandbox +lbg sdbx exec --user root python script.py --json # foreground command +lbg sdbx files write --source ./local_dir /workspace/dir --json # upload a file or directory +lbg sdbx files read /workspace/result.csv --output ./result.csv +lbg sdbx kill --json # destroy a sandbox +``` + +`lbg sdbx create` POSTs to the sandbox workspace OpenAPI endpoint with +body `{"templateID": ""}`. `--project-id` is **optional** +and only affects billing: without it the sandbox bills against your +personal wallet; with it the value is sent as the `X-PROJECT-ID` +request header and the sandbox bills against that project's budget +(look up project IDs with `lbg project ls`). The response is the raw +backend payload — it carries `sandboxID` (the ID to use with `exec`, +`kill`, etc.), `templateID`, `state`, `cpuCount`, `memoryMB`, and a +`metadata` dict populated by the platform. + +**Sandbox lifetime.** When neither flag is set, the CLI sends +`"timeout": 43200` (12 hours) so multi-hour agent runs aren't cut off +by the much shorter platform default; the create command also prints +a one-line stderr hint reminding you that the default is 12h and +explaining how to extend. Override with one of: + +- `--timeout N` adds `"timeout": N` (seconds, non-negative integer) to + the create body — the sandbox is auto-destroyed N seconds after + creation. `--timeout 0` is the explicit "unlimited" sentinel and is + passed through as-is, never substituted. +- `--never-timeout` adds the metadata flag + `e2b.agents.kruise.io/never-timeout=true`, which tells the platform + to never auto-destroy the sandbox; no `timeout` field is sent. Pair + with explicit `lbg sdbx kill ` when done — long-lived + sandboxes keep billing. + +`--timeout` and `--never-timeout` are mutually exclusive; passing both +fails the request with exit code 2 before any HTTP call. + +**Storage mounts.** Pass `--mount-user-storage` to ask the platform to +mount the caller's personal disk and the share disks the caller has +access to into the sandbox. This adds the metadata flag +`bohr.launching.io/mount-user-storage=true` on the create body and is +independent of every other flag (combine with `--timeout`, +`--never-timeout`, `--reserve-failed-sandbox`, or `--project-id` as +needed). Default is off — sandboxes start with no user storage mounted. + +**Share disk sub-path.** Pass `--share-subpath ` to mount only a +sub-directory *inside* the project share disk instead of its root. This +adds the metadata flag `bohr.launching.io/share-subpath=` on the +create body. It only affects the project **share** disk (personal disk is +unchanged) and only takes effect with `--mount-user-storage` (+ a +`--project-id` that grants a share disk); otherwise the CLI prints a +stderr hint and the flag is a no-op. `` is relative to the project +share root and is **confined to it server-side**: a leading `/` or any +`..` segment is clamped back inside the share (e.g. `../../etc` → +`/etc`), so it can never escape the project's share area. +Leave unset to mount the share root as before. + +**Environment variables.** Pass `--env KEY=VALUE` (repeatable) to inject +environment variables into the sandbox at creation time. The create body +carries the map under **both** `envVars` (the field the e2b orchestrator +actually applies to the sandbox process environment) and `envs` (persisted +by launching into its `bohr_sandbox.envs` record). Splitting is on the +first `=` only, so values may themselves contain `=` (e.g. +`--env URL=a=b=c`). Empty `--env` is omitted from the body so backend +defaults apply. The injected vars are visible to every later +`lbg sdbx exec` / `terminal` shell (any user), e.g. +`lbg sdbx exec --user root 'echo $HELLO'`. + +**In-place image override.** Pass `--image ` to launch the +chosen template but replace its container image with `` for +this one sandbox. This adds the metadata flag +`e2b.agents.kruise.io/image=` on the create body; the +platform keeps the template's SKU / CPU / memory / GPU shape and only +swaps the running container image. Use this to try an image without +creating a dedicated template (for a persistent change, use +`lbg sdbx template update --image ` instead). Find +image references with `lbg sdbx image ls`. Independent of every other +create flag (combine with `--timeout`, `--never-timeout`, +`--mount-user-storage`, `--env`, `--project-id` as needed). + +**Session id.** Pass `--session-id ` to tag the sandbox with a +caller-defined session id. This adds the metadata flag +`bohr.launching.io/session-id=` on the create body; the platform +records it for per-session lookup and uses it as the user-storage mount +subPath (`{prefix}/users/{userId}/{sessionId}/`), so sandboxes created +with the same session id share the same persisted directory (pairs with +`--mount-user-storage`). Independent of every other create flag. + +**`lbg sdbx list` — age column and stale-sandbox warning.** The default +table view appends an `age` column showing how long each sandbox has +been alive (`Ns` / `Nm` / `NhMm` / `NdMh`). Rows older than 30 minutes +are highlighted (yellow on a TTY, ⚠ glyph everywhere — including +captured non-TTY output) and a one-line footer reminds you to kill them +if no longer in use. The footer is suppressed under `-q` and on +machine-readable channels (`--json`, `--csv`, `--yaml`). Structured +outputs add an `age_seconds` integer per entry so consumers can apply +their own highlight policy; the human ⚠ glyph is intentionally absent +from JSON/CSV/YAML. + +**Pass the template `name`, not a SKU or numeric `id`.** Despite the +request field being called `templateID`, the backend accepts the +template's string `name` — the `name` column of `lbg sdbx template ls`, +or a platform shortcut. If you have a SKU value instead, list templates +with `lbg sdbx template ls` and list SKUs with `lbg sdbx machine list`. + +For GPU workloads, use one of the GPU template shortcuts from +[`platform-snapshot.md`](../platform-snapshot.md): + +```bash +lbg sdbx create --json +lbg sdbx exec --user root nvidia-smi +lbg sdbx exec --user root python train.py --json +lbg sdbx kill --json +``` + +## Best practices + +- **Reuse** running sandboxes whenever possible — check `lbg sdbx list` before creating new ones. +- **Check before killing** — run `lbg sdbx ps ` (or `describe --with-processes`) to confirm no job is still running. +- **Save your work** before killing — data is permanently lost after kill. Use `lbg sdbx files` to retrieve important files first (see `lbg sdbx files --help`). For the full retrieve-before-kill SOP for long-running jobs, see [`execution-modes.md`](./execution-modes.md). +- **Kill promptly** when done — release resources with `lbg sdbx kill `. Safety behaviour: + - No running processes → kill proceeds silently. + - Running processes + TTY → interactive confirmation. + - Running processes + non-TTY (agents, CI, piped shells) → the kill is **refused** with a clear error; pass `--force` to acknowledge and proceed. + - Pass `--force` any time you know there is pending work you want to discard. + +## Sandbox creation parameter recommendations + +| Scenario | `--timeout` | `--never-timeout` | Notes | +|----------|-------------|-------------------|-------| +| Interactive debugging | 3600 (1h) | no | Avoid forgotten sandboxes billing | +| Batch computation (DFT, MD, etc.) | 7200–14400 (2–4h) | no | Estimate based on job size | +| Long-term dev environment | — | yes | Requires manual `kill` when done | + +**Default recommendation:** Do not use `--never-timeout` unless you +explicitly need a long-lived sandbox. Always pair it with an eventual +`lbg sdbx kill`. Use `--timeout` as a safety net — even a generous value +(e.g. `--timeout 28800` for 8h) protects against forgotten sandboxes. + +## Auto-cleanup patterns + +Sandboxes bill by time (even when idle). Always destroy them when the +task is done. Since a sandbox cannot kill itself from inside, use one +of these patterns: + +### Pattern 1: Poll-and-kill (single sandbox) + +```bash +SANDBOX_ID=$(lbg sdbx create my-template --timeout 7200 --json | jq -r '.sandbox_id') + +# Launch job in background +lbg sdbx exec --background --user root "$SANDBOX_ID" 'bash /workspace/run.sh' + +# Poll for completion (touch /workspace/DONE at end of run.sh) +while true; do + if lbg sdbx exec --user root "$SANDBOX_ID" 'test -f /workspace/DONE' 2>/dev/null; then + break + fi + sleep 30 +done + +# Retrieve outputs +lbg sdbx files read "$SANDBOX_ID" /workspace/out/results.tar.gz --format bytes --output ./results.tar.gz + +# Destroy +lbg sdbx kill "$SANDBOX_ID" +``` + +### Pattern 2: Multi-sandbox parallel (batch jobs) + +Split a batch across multiple sandboxes for maximum throughput. Each +sandbox runs its assigned frames **serially** (VASP and similar MPI +codes must not run concurrent instances in the same sandbox). + +```bash +SANDBOX_IDS=() +PROJECT_ID=29496 +TEMPLATE=c32_m128_cpu +N_SANDBOXES=4 +FRAMES_PER_SANDBOX=10 + +# Create sandboxes +for i in $(seq 0 $((N_SANDBOXES - 1))); do + id=$(lbg sdbx create "$TEMPLATE" --project-id "$PROJECT_ID" --timeout 14400 --json | jq -r '.sandbox_id') + SANDBOX_IDS+=("$id") +done + +# Upload inputs and launch serial batch per sandbox +for idx in "${!SANDBOX_IDS[@]}"; do + id="${SANDBOX_IDS[$idx]}" + batch_start=$((idx * FRAMES_PER_SANDBOX)) + batch_end=$((batch_start + FRAMES_PER_SANDBOX - 1)) + for i in $(seq "$batch_start" "$batch_end"); do + lbg sdbx files write --source "scf_frame_$i" "$id" /workspace/ + done + # run_batch.sh runs frames one by one: for d in /workspace/frame_*; do (cd "$d" && mpirun -np $NCPU vasp_std); done + lbg sdbx exec --background --user root "$id" "bash /workspace/run_batch.sh" +done + +# Poll all sandboxes for completion +for id in "${SANDBOX_IDS[@]}"; do + while true; do + if lbg sdbx exec --user root "$id" 'test -f /workspace/DONE' 2>/dev/null; then + break + fi + sleep 30 + done +done + +# Retrieve results from all sandboxes +for id in "${SANDBOX_IDS[@]}"; do + lbg sdbx files read "$id" /workspace/out/results.tar.gz --format bytes --output "./results_${id}.tar.gz" +done + +# Destroy all sandboxes +for id in "${SANDBOX_IDS[@]}"; do + lbg sdbx kill "$id" +done +``` + +**Throughput example:** 4 sandboxes × 32 CPUs = 128 cores. Each sandbox +runs its 10 frames serially using all 32 CPUs. With 40 total frames, +wall time drops from ~5h (single sandbox, serial) to ~1.25h (4 sandboxes +in parallel). For even faster turnaround, use more sandboxes (e.g. 8 +sandboxes → ~40 min). + +### Pattern 3: Timeout as safety net + +Always set `--timeout` even if you plan to kill manually — it prevents +billing surprises if the controlling script crashes. + +```bash +# Create with a generous timeout as safety net +lbg sdbx create my-template --timeout 28800 --json # 8h auto-destroy + +# Normal workflow: poll, retrieve, kill +# If the controlling script dies, the sandbox self-destructs after 8h +``` + +## Within-sandbox CPU utilization + +Sandboxes expose all CPUs from their SKU. Detect and use them to avoid +wasting compute: + +```bash +# Detect available CPUs +NCPU=$(nproc --all) +echo "Sandbox has $NCPU CPUs" +``` + +**Strategy A: single large job** — use all CPUs for one MPI run. +This is the default for VASP and other MPI-parallel codes. + +```bash +NP=$NCPU +mpirun -np "$NP" vasp_std +``` + +**Strategy B: serial batch of small jobs** — run multiple frames +**one after another** inside the same sandbox. Each frame uses all +CPUs. This is the correct approach for VASP, which must not run +multiple instances concurrently in the same sandbox (they would +conflict over MPI ports, scratch space, and memory). + +```bash +# Run frames serially — each gets the full CPU count +for i in /workspace/frame_*; do + (cd "$i" && mpirun -np "$NCPU" vasp_std) +done +``` + +> **Do NOT run multiple VASP instances in parallel inside one sandbox.** +> VASP binds to fixed resources; concurrent instances cause crashes, +> hangs, or garbage results. For true parallelism across frames, use +> the multi-sandbox pattern (see § Multi-sandbox parallel above). + +**Guidelines for choosing `NP` (single job) or number of serial frames:** + +| System size (atoms) | Recommended NP | Notes | +|---------------------|----------------|-------| +| < 50 | 2–4 | Small systems don't scale well with more ranks | +| 50–200 | 4–8 | Sweet spot for most DFT codes | +| 200–500 | 8–16 | Benefits from more parallelism | +| > 500 | 16+ or full node | Use all available CPUs | + +For a 64-atom SiC system on a 32-CPU sandbox: run VASP with +`mpirun -np 8` (or up to 32), frames **serially**. For higher +throughput, open multiple sandboxes and run one frame per sandbox +concurrently — see § Multi-sandbox parallel. + +## Checking for leaked sandboxes + +```bash +# List all running sandboxes with age +lbg sdbx list --json | jq '.[] | {id: .sandbox_id, created: .created_at, state: .state, age_seconds}' + +# The default table view highlights sandboxes older than 30 minutes (⚠) +lbg sdbx list +``` + +Run this periodically to catch forgotten sandboxes that are still billing. diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/network.md b/src/matcreator/skills/lbg-cli/references/sandbox/network.md new file mode 100644 index 00000000..9fc8d567 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/network.md @@ -0,0 +1,172 @@ +# Sandbox network — on-demand HTTP proxy toggle + +Sandboxes default to **no outbound HTTP proxy**. The image-level +`/etc/pip.conf` Aliyun mirror keeps domestic PyPI fast out of the box, +so most workflows need no extra setup. For overseas access (PyPI +`pypi.org`, GitHub, HuggingFace, Google Drive), toggle the +`pai.ga.op.xdptech.com:3128` HTTP proxy on with the snippet below; remember to +toggle it off afterwards so domestic mirrors stay fast. + +The snippets below are inline and self-contained — they work on any +sandbox image, not just `sdbxagent`, and do not require fetching any +external script. + +## Proxy on — enable `pai.ga.op.xdptech.com` HTTP proxy + +Single ready-to-paste shell block. Sets the five user-level proxy +configs idempotently. + +```bash +# pip — user-level proxy (image-level /etc/pip.conf Aliyun mirror is left untouched) +mkdir -p ~/.pip && cat > ~/.pip/pip.conf <<'EOF' +[global] +proxy=http://pai.ga.op.xdptech.com:3128 +EOF + +# conda / mamba +cat > ~/.condarc <<'EOF' +proxy_servers: + http: http://pai.ga.op.xdptech.com:3128 + https: http://pai.ga.op.xdptech.com:3128 +ssl_verify: false +EOF + +# wget +cat > ~/.wgetrc <<'EOF' +http_proxy = http://pai.ga.op.xdptech.com:3128 +https_proxy = http://pai.ga.op.xdptech.com:3128 +use_proxy = yes +EOF + +# curl +cat > ~/.curlrc <<'EOF' +proxy = http://pai.ga.op.xdptech.com:3128 +EOF + +# git — global config +git config --global http.proxy http://pai.ga.op.xdptech.com:3128 +git config --global https.proxy http://pai.ga.op.xdptech.com:3128 +``` + +## Proxy off — clear all user-level proxy configs + +Single ready-to-paste shell block. Removes/empties the five user-level +configs and clears env-var fallbacks. Image-level `/etc/pip.conf` +(Aliyun mirror) stays untouched, so domestic `pip install` continues +to work. + +```bash +# remove user-level proxy configs +rm -f ~/.pip/pip.conf ~/.condarc ~/.wgetrc ~/.curlrc + +# unset git's global proxy +git config --global --unset http.proxy 2>/dev/null || true +git config --global --unset https.proxy 2>/dev/null || true + +# clear env-var fallbacks for the current shell +unset HTTP_PROXY HTTPS_PROXY http_proxy https_proxy +``` + +## Side effects of proxy on + +When the proxy is on, **all** HTTP/HTTPS traffic for the configured +tools routes through `pai.ga.op.xdptech.com:3128`, including domestic targets like +`mirrors.aliyun.com`. Domestic access becomes noticeably slower than +the default off state. Turn the proxy on only when you actually need +overseas reach, and flip it off as soon as the overseas step is done. + +The proxy also exhibits intermittent `503 / context deadline / TLS +recv` errors on long-lived TLS sessions (`git clone`, HuggingFace +downloads, large overseas `wget`). Retries usually succeed; for a +single command that keeps failing while the proxy is on, see +*Per-command bypass* below. + +## How to use from an agent / inside `lbg sdbx exec` + +Wrap the snippet in `bash -c` and ship it through `exec`: + +```bash +# turn proxy on +lbg sdbx exec --user root -- bash -c ' +mkdir -p ~/.pip && cat > ~/.pip/pip.conf < ~/.condarc < ~/.wgetrc < ~/.curlrc < -- pip install -i https://pypi.org/simple/ click + +# turn proxy off +lbg sdbx exec --user root -- bash -c ' +rm -f ~/.pip/pip.conf ~/.condarc ~/.wgetrc ~/.curlrc +git config --global --unset http.proxy 2>/dev/null || true +git config --global --unset https.proxy 2>/dev/null || true +' +``` + +The on/off snippets are idempotent — running them twice is safe. + +## Per-command bypass (proxy on, one call needs to go direct) + +When the proxy is on but a single command needs to skip it (e.g. it +keeps timing out), override per invocation without touching the +session-wide config: + +```bash +# wget — explicit flag +wget --no-proxy https://example.com/file + +# curl — explicit flag +curl --noproxy '*' https://example.com/file + +# git — per-invocation override (empty string disables) +git -c http.proxy= -c https.proxy= clone https://github.com/owner/repo + +# pip — explicit empty proxy +pip install --proxy '' some-package + +# any subprocess that respects http_proxy / https_proxy env vars +HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= +``` + +## Reference baselines + +End-to-end validated on a fresh sandbox: + +| State | Domestic (Aliyun mirror, `mirrors.aliyun.com`) | Overseas (PyPI `pypi.org`, GitHub, HuggingFace, Google Drive) | +|---|---|---| +| Proxy off (default) | fast | unreachable | +| Proxy on | slow (all traffic via `pai.ga.op.xdptech.com:3128`) | reachable; HuggingFace and large `git clone` may need retries | + +Verified with proxy on: `pip install -i https://pypi.org/simple/ click`, +`uv pip install --index-url https://pypi.org/simple/ tabulate`, +`git clone https://github.com/octocat/Hello-World`, +Google Drive `wget` (~5 MB/s sustained on a 296 MB file), +`curl -I https://huggingface.co/bert-base-uncased/resolve/main/config.json`. + +`apt` is not available in the user-mode sandbox (no `sudo`/root). +System packages need to be baked at image build time, not added at +runtime. + +## See also + +- [`exec.md`](./exec.md) — running commands inside the sandbox +- [`../software-install/mirrors.md`](../software-install/mirrors.md) — public domestic mirrors (an alternative when running outside the platform) +- [`../platform-snapshot.md`](../platform-snapshot.md) — current default template and SKU shortcuts diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/pitfalls.md b/src/matcreator/skills/lbg-cli/references/sandbox/pitfalls.md new file mode 100644 index 00000000..8b6ab891 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/pitfalls.md @@ -0,0 +1,77 @@ +# Default-behavior pitfalls + +Every item below is a **default** that is safe for quick interactive use but +bites long-running / ML-training workloads. Read this once before driving a +sandbox through an agent; each row links to the deep doc. + +| # | Default | When it bites | Do this instead | +|---|---------|---------------|-----------------| +| 1 | Overlay (writable root) disk is **fixed at 30Gi**; SKU sizes CPU/mem/GPU only, **not disk** | A 7B + 1.5B model plus torch/transformers and a dataset overflow 30Gi → `OSError: No space left on device` | Disk is **set at template-create time**, not on `sdbx create`: build a private template with `--extra-ephemeral-storage-gb N` (quota-gated), or keep big weights off the overlay with `sdbx create --mount-user-storage`. See [`templates.md`](./templates.md). | +| 2 | `files read` transfers as **text** unless told otherwise | Reading a binary (`.pt`, `.tar`, `.gz`, `.bin`, ...) as text decodes+re-encodes the bytes → wrong file size, `torch.load`/unzip fails with "invalid header or archive is corrupted" | Now **auto-detected from the extension** (binary → bytes, with a stderr warning). For an unrecognized binary extension pass `--format bytes`. See [`files.md`](./files.md). | +| 3 | `exec` foreground **`--timeout` is 60s** | A 30-min training/install run is killed at 60s (and a flaky link can drop it sooner) | Use `exec --background` (defaults to unlimited `--timeout 0`) and retrieve outputs before kill. See [`execution-modes.md`](./execution-modes.md). | +| 4 | `--background` with a **finite `--timeout`** kills the remote command at that boundary | `exec --background --timeout 60` silently dies at 60s even though it's "background" | Omit `--timeout` or pass `--timeout 0` for true long-running work. The CLI warns, but the job still dies. | +| 5 | `kill` **permanently destroys** the sandbox; a stopped sandbox's files **cannot be read** | Killing before pulling outputs loses everything under `/workspace/` | Retrieve every file you need with `files read --output ...` **before** `kill`. Non-TTY `kill` with live processes is refused unless you pass `--force`. See [`execution-modes.md`](./execution-modes.md). | +| 6 | Sandbox **auto-destroys after 12h** (CLI default `timeout=43200`); idle sandboxes may also be reclaimed by the platform after a period of no API activity | A multi-day job, or one left idle, gets torn down underneath you | `sdbx create --timeout N` to set the lifetime, or `--never-timeout` to opt out (then `kill` yourself — long-lived sandboxes keep billing). See [`lifecycle.md`](./lifecycle.md). | +| 7 | Outbound **HTTP proxy is off** | `pip install` from `pypi.org`, `git clone` GitHub, HuggingFace downloads hang/fail; domestic Aliyun mirror still works | Toggle the `pai.ga.op.xdptech.com` proxy on for overseas access, then off when done. See [`network.md`](./network.md). | +| 8 | No `--project-id` → bills your **personal wallet** | Create is rejected if the personal wallet balance is ≤ 0; project budget is not used | Pass `--project-id ` to bill a project budget (find IDs with `lbg project ls`). See [`lifecycle.md`](./lifecycle.md). | +| 9 | **No user storage mounted** | Expecting your personal/share disk inside the sandbox; it isn't there | `sdbx create --mount-user-storage`. See [`lifecycle.md`](./lifecycle.md). | +| 10 | `terminal send` has **no implicit newline and no stdout echo** | Treating `terminal send` as run-and-capture returns only `sent_bytes`, not the command output | For "run a command, get output" use `exec`. In a PTY, append `\n` and read the redirected file. See [`terminal.md`](./terminal.md). | +| 11 | A template image with a mutable `:latest` tag is **rejected** by `template create` / `template update --image` | A template's image cache is keyed by the image tag; a `:latest` cannot be reused (old cache deleted, then rebuilt on every create/update), and sharing one `…:latest` across templates churns the cloud image-cache quota → other templates' **new** sandboxes fall back to a slow cold pull (can hit the create timeout) until the cache rebuilds | Point template images at an **immutable / unique tag** (a version, or the unique tag `image build` / `image commit` already produce). `image commit`/`build` also **warn** (non-fatal, `--name` is only the user-visible portion) when you name a new image `…latest`, so you don't mint one you can't use as a template base. See [`templates.md`](./templates.md) and [`images.md`](../images.md). | +| 12 | `exec --background` scripts use `pgrep -f vasp_std` to check if a process is alive | When multiple batch scripts run concurrently, one script's `pgrep` matches **all** VASP processes across scripts → wait logic gets confused or exits prematurely | Use **PID files** for precise tracking: write `echo $! > /tmp/vasp.pid` after launch, check `kill -0 $(cat /tmp/vasp.pid)` for liveness. Do **not** use `pgrep -f` for per-instance process detection. | + +## Quick checklist for an ML training run + +```bash +# 1. Enough disk: build a template with extra overlay storage (one-time) +lbg sdbx template create --name train-tpl --image --sku-name \ + --extra-ephemeral-storage-gb 80 + +# 2. Create with an explicit lifetime + project billing +lbg sdbx create train-tpl --timeout 0 --project-id --json # 0 = no auto-destroy + +# 3. (overseas deps) proxy on, install, proxy off — see network.md + +# 4. Run long jobs in the background, log to a persistent path +lbg sdbx exec --background --user root 'mkdir -p /workspace/out && python train.py > /workspace/out/run.log 2>&1' + +# 5. Retrieve BEFORE kill; binaries download losslessly (auto bytes) +lbg sdbx files read /workspace/out/run.log --output ./run.log +lbg sdbx files read /workspace/out/model.pt --output ./model.pt + +# 6. Done -> kill (frees resources; long-lived sandboxes keep billing) +lbg sdbx kill +``` + +## Quick checklist for batch / multi-frame computation + +```bash +# 1. Detect CPUs +NCPU=$(nproc --all) + +# 2. Create sandbox with timeout as safety net +SANDBOX_ID=$(lbg sdbx create c32_m128_cpu --project-id --timeout 14400 --json | jq -r '.sandbox_id') + +# 3. Upload inputs, run frames SERIALLY (VASP cannot run concurrent instances) +for i in $(seq 0 9); do + lbg sdbx files write --source "frame_$i" "$SANDBOX_ID" /workspace/ +done +lbg sdbx exec --background --user root "$SANDBOX_ID" 'for d in /workspace/frame_*; do (cd "$d" && mpirun -np '"$NCPU"' vasp_std); done && touch /workspace/DONE' + +# 4. Poll for completion, retrieve, then kill +# See lifecycle.md — Auto-cleanup patterns for the full workflow + +# 5. ALWAYS kill when done +lbg sdbx kill "$SANDBOX_ID" +``` + +For true parallelism across frames, run multiple sandboxes — each +sandbox runs one frame with all CPUs. See +[`lifecycle.md`](./lifecycle.md) § Multi-sandbox parallel. + +## See also + +- [`lifecycle.md`](./lifecycle.md) — timeout / billing / mount / kill semantics, auto-cleanup patterns, multi-sandbox parallel, CPU utilization +- [`execution-modes.md`](./execution-modes.md) — foreground vs background vs PTY, retrieve-before-kill SOP +- [`files.md`](./files.md) — text vs bytes download +- [`templates.md`](./templates.md) — disk size via `--extra-ephemeral-storage-gb` +- [`network.md`](./network.md) — outbound proxy toggle diff --git a/src/matcreator/skills/lbg-cli/references/sandbox/templates.md b/src/matcreator/skills/lbg-cli/references/sandbox/templates.md new file mode 100644 index 00000000..d1ca4e02 --- /dev/null +++ b/src/matcreator/skills/lbg-cli/references/sandbox/templates.md @@ -0,0 +1,103 @@ +# Sandbox templates + +List, create, and delete sandbox templates. + +List templates you've already created, or create a new one. + +```bash +lbg sdbx template ls # table of your templates +lbg sdbx template ls --json # machine-readable JSON +lbg sdbx template ls --page 2 --page-size 20 --json +lbg sdbx template ls -q # names only (pipe into `lbg sdbx create`) +``` + +For compatibility, default `--json` output remains a template array. Passing +`--page` or `--page-size` switches listing to the paginated backend endpoint. +When paginating, the default page size is 100, and `--page-size` can be lowered +when an agent wants smaller responses. Paginated `--json` output is an envelope: +`page`, `page_size`, `total`, `total_pages`, and `list`. The template rows are +inside `list`; table output and `-q` render the selected page's `list` only. + +**Compatibility note.** Older automation that uses +`lbg sdbx template ls --json` without pagination can keep reading the top-level +array. Automation that opts into `--page` or `--page-size` must read `list` +instead. + +Creating a template requires an image reference and a SKU name: + +```bash +lbg image ls # find an imagePath (paste into --image) +lbg sdbx machine list # find a SKU name +lbg sdbx template create \ + --name \ + --image \ + --sku-name +``` + +> **`:latest` image tags are rejected.** `template create` and +> `template update --image` refuse any image whose tag ends in `latest` +> (e.g. `:latest`, `:dev-latest`) and ask you to use an immutable tag. +> Why: the backend pre-warms an image cache keyed by the image tag, and a +> mutable `:latest` cannot be reused — it is deleted-and-rebuilt on every +> create/update pointing at it. When several templates share one `…:latest`, +> that churns the cloud image-cache quota and makes every affected +> template's **new** sandboxes cold-pull the image (slow — can hit the +> create timeout) until the cache rebuilds. Use an **immutable / unique +> tag** instead: a version, or the unique tag that `lbg image build` / +> `lbg image commit` already produces. + +## Disk size (extra ephemeral storage) + +A sandbox's overlay (writable root) disk defaults to **30Gi**. The SKU only +sizes CPU/memory/GPU — it does **not** include disk. If models, datasets, or +Python envs fill the default disk you'll hit `OSError: No space left on device` +(e.g. a 7B + 1.5B model plus torch/transformers easily exceeds 30Gi). + +Disk size is fixed per template at creation time (it cannot be changed on +`lbg sdbx create`). Request extra overlay storage with +`--extra-ephemeral-storage-gb` when creating the template: + +```bash +lbg sdbx template create \ + --name \ + --image \ + --sku-name \ + --extra-ephemeral-storage-gb 50 # 30Gi default + 50Gi extra = 80Gi total +``` + +Rules: + +- **Private templates only.** Public templates reject a non-zero value. +- **Quota-gated.** The amount is capped by your per-user quota. If the quota is + unset / `0`, or the value exceeds it, the template is **still created + successfully** but the extra storage is **not applied** (disk falls back to + the default 30Gi) and the response carries a reminder to contact the + administrator to raise the quota. Check the response `message`. +- Alternatively, mount a persistent personal/share disk with + `lbg sdbx create