Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ dependencies = [
"rdkit>=2025.9.1",
"google-adk>=1.28.0",
"google-cloud-storage",
"lbg>=1.2.29",
"lbg>=4.0.0b56",
"litellm>=1.77.4",
"periodictable>=2.0.2",
"pyyaml>=6.0.2",
Expand Down
137 changes: 137 additions & 0 deletions src/matcreator/skills/lbg-cli/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
name: lbg-cli
description: >-
CLI for operating Bohrium cloud sandboxes (sdbx), managing templates,
committing container images, and installing software inside sandboxes.
Use when the user mentions lbg, sdbx, sandbox, or any Bohrium sandbox
operation.
---

# lbg CLI — Sandbox (sdbx)

CLI for managing **cloud sandboxes (sdbx)** on the Bohrium platform.
Covers sandbox lifecycle (`create`, `exec`, `kill`, `list`, `describe`),
interactive PTY sessions, file transfer, template management, image
commit/build, and software installation inside sandboxes.

> **Proxy toggle.** Sandboxes default to no outbound HTTP proxy.
> For overseas access (PyPI `pypi.org`, GitHub, HuggingFace, etc.), toggle
> the `pai.ga.op.xdptech.com` proxy on with the snippet in
> [`references/sandbox/network.md`](references/sandbox/network.md),
> then toggle it off when done. Domestic access via the image-level Aliyun
> mirror is unaffected.

> **Default-behavior pitfalls (read first for ML / long-running jobs).**
> Several defaults are fine for quick interactive use but bite training
> workloads: the overlay disk is **fixed at 30Gi** (SKU has no disk field),
> `exec` foreground **`--timeout` is 60s**, `files read` transfers as **text**
> (corrupts binaries), `kill` is **irreversible** and a stopped sandbox's
> files can't be read, the outbound proxy is **off**, and a template image
> with a mutable `:latest` tag is **rejected** (it churns the prewarmed image
> cache — use an immutable tag). See the consolidated list
> in
> [`references/sandbox/pitfalls.md`](references/sandbox/pitfalls.md) before
> driving a sandbox through an agent.

## Prerequisites

```bash
lbg login --ak <your-access-key>
```

Or set `BOHRIUM_ACCESS_KEY` in the environment, or pass `--api-key` per call.

## Default sandbox template

`lbg sdbx create` with no template argument creates a sandbox from the
`sdbxagent` template — `pytorch20-scicomp:1.0.1` with the image-level Aliyun
PyPI mirror, so domestic `pip install` is fast out of the box. The outbound
HTTP proxy is **off by default**; for overseas access toggle it on demand via
the `proxy on` / `proxy off` snippets in
[`references/sandbox/network.md`](references/sandbox/network.md).

For GPU work, pass `--gpu` as a shortcut: bare `--gpu` resolves to
`scicomp-4090` (the default GPU template); `--gpu 5090` and `--gpu l20` pick
the other two GPU templates. Mutually exclusive with the positional
`template` argument.

## Command surface

| Command | Purpose |
| --- | --- |
| `lbg login` / `logout` | Persist a Bohrium access key |
| `lbg sdbx` | Operate cloud sandboxes (sdbx) |
| `lbg sdbx image` | Commit sandbox snapshots / build from Dockerfile |
| `lbg skill` | Export / diff / update this agent skill |

## Skill self-maintenance

Keep the installed copy of this skill current so agents never operate on stale
sandbox / sdbx docs. Each export writes a `skill-manifest.json` (lbg version,
export time, per-file `sha256`).

| Subcommand | Description |
| --- | --- |
| `skill export [--output <parent>]` | Write the skill tree to `<parent>/lbg-cli/` (fails on conflict) |
| `skill diff --target <skill-dir>` | Report drift vs the bundled version; exits non-zero when out of sync (`--json` for a machine-readable report) |
| `skill update --target <skill-dir> --backup` | Snapshot the old copy to `<name>.bak.<timestamp>`, then reinstall (`--force` overwrites in place; `install` is an alias) |

`--target` points at the skill directory itself (e.g.
`~/.codex/skills/lbg-cli`); `--output` on `export` is the parent directory.

## Sandbox commands

| Subcommand | Description |
| --- | --- |
| `create [template]` | Launch a new sandbox (default: `sdbxagent` template) |
| `exec <id> <cmd>` | Run a command inside a sandbox (foreground / background); **always pass `--user root`** |
| `kill <id>` | Terminate a sandbox |
| `list` | List active sandboxes |
| `describe <id>` | Get sandbox details (incl. IP, status, template) |
| `terminal <id>` | Open an interactive PTY session |
| `upload <id> <src> <dst>` | Upload files or directories |
| `download <id> <src> [dst]` | Download files or directories |
| `image commit` | Snapshot a running sandbox's filesystem |
| `image build` | Build an image from a Dockerfile |
| `image get <id>` | Get image build status |
| `image ls` | List images (filterable by sandbox, status, name) |
| `image build-log <id>` | Fetch or follow build logs |
| `template create` | Create a new sandbox template |
| `template list` | List available templates |
| `template update` | Update a template |
| `template delete` | Delete a template |

## Discovering commands

```bash
lbg sdbx --help
lbg sdbx <subcommand> --help
```

## Reference docs

Deep usage docs under `references/`. Load only what's relevant.

**Sandbox lifecycle & usage:**

- [`references/sandbox/pitfalls.md`](references/sandbox/pitfalls.md) — default-behavior traps (disk 30Gi, timeouts, binary downloads, kill, proxy, billing) + an ML-training checklist
- [`references/sandbox/lifecycle.md`](references/sandbox/lifecycle.md) — create / list / describe / kill, plus best practices
- [`references/sandbox/execution-modes.md`](references/sandbox/execution-modes.md) — foreground vs background vs PTY, plus the retrieve-before-kill SOP for long-running jobs
- [`references/sandbox/exec.md`](references/sandbox/exec.md) — `lbg sdbx exec` usage, including `--background`
- [`references/sandbox/terminal.md`](references/sandbox/terminal.md) — PTY sessions for REPLs / TUIs / Ctrl-C interaction
- [`references/sandbox/files.md`](references/sandbox/files.md) — upload / download files and directories
- [`references/sandbox/templates.md`](references/sandbox/templates.md) — list, create, update, and delete sandbox templates
- [`references/sandbox/network.md`](references/sandbox/network.md) — on-demand `pai.ga.op.xdptech.com` HTTP proxy toggle

**Container images (sandbox snapshots & builds):**

- [`references/images.md`](references/images.md) — `lbg sdbx image commit` / `image build` for producing container images

**Software install (inside sandboxes):**

- [`references/software-install/README.md`](references/software-install/README.md) — pre-installed inventory + domestic mirrors for sandbox installs

## Output formats

`lbg sdbx` commands accept `--json` for machine-readable output suitable
for scripting and agent use.
113 changes: 113 additions & 0 deletions src/matcreator/skills/lbg-cli/references/images.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Image commit & dockerfile build

`lbg sdbx image commit` / `image build`: produce container images.

Two ways to mint a new container image; both are async, both surface in
`lbg sdbx image ls`, and the `buildType` column tells them apart:

| Subcommand | When to use | Source |
|---|---|---|
| `image commit` | snapshot a *running* sandbox's filesystem | needs `--sandbox-id` |
| `image build` | reproducible build from source | needs `--dockerfile <file>` |

Both jobs return the new image record's `id` *and* its final `imageUrl`
**immediately** on submission — the URL is reserved up-front so it can
be plugged into a later `lbg sdbx template create --image <imageUrl>`
without polling. Polling `get <id>` is only needed if you actually want
to wait for `success` / `failed` before consuming the image.

```bash
# A) commit — snapshot a running sandbox
lbg sdbx image commit \
--sandbox-id <sandbox_id> \
--name <image-name> \
--desc "training-day-3 snapshot" \
--project-id <project_id> \
--json

# B) build — from a local Dockerfile (no sandbox needed)
lbg sdbx image build \
--dockerfile ./Dockerfile \
--name <image-name> \
--desc "pinned cuda 12.1" \
--project-id <project_id> \
--json

# B') tag the build with the sandbox it originated from (provenance only)
lbg sdbx image build \
--dockerfile ./Dockerfile \
--name <image-name> \
--sandbox-id <sandbox_id> \
--project-id <project_id>

# poll either kind (status: 0=creating 1=pending 2=success 3=failed)
lbg sdbx image get <id> --json

# fetch the kaniko log of a dockerfile build (commit records have no log)
lbg sdbx image build-log <id> # raw text snapshot to stdout, pipe-friendly
lbg sdbx image build-log <id> --json # {"id": ..., "log": "..."}
lbg sdbx image build-log <id> --follow # stream chunks as kaniko writes them,
# auto-EOFs when the build reaches a
# terminal status (success/failed).
# Mutually exclusive with --json.

# browse your images across all sandboxes / build types
lbg sdbx image ls # default table (incl. statusReason)
lbg sdbx image ls --json # full envelope (items+page+total)
lbg sdbx image ls --sandbox-id <sandbox_id> # one sandbox's records
lbg sdbx image ls --status 2 --status 3 # successes + failures
lbg sdbx image ls --name train --start-time '2026-04-01 00:00:00'
lbg sdbx image ls -q # ids only, one per line
```

## Naming — avoid a `latest` image name

`--name` is the **user-visible portion** of the image name; the backend may
prepend a user/project prefix and controls the final registry tag, so the CLI
does *not* hard-reject it here. But if the name ends in `latest`
(`--name latest`, `--name img:latest`, `--name img:dev-latest`, …) the CLI
prints a **stderr hint** (stdout / `--json` stay clean) nudging you toward an
immutable/unique name. Reason: a `latest` image is a poor template base —
`lbg sdbx template create/update` **rejects** a `:latest` template image, and
the platform's prewarmed image cache cannot be reused for a mutable tag. Prefer
a version (`:v1`) or a date/build tag so the image you just minted can be
plugged straight into `template create --image` without a rename.

## Status & failure reason

`status` is the coarse lifecycle (`0=creating 1=pending 2=success 3=failed`).
For `failed` records, `statusReason` is a **structured machine-readable code**
that explains *why* it failed — pair it with the free-form `errorMsg` for the
full picture. The field is `omitempty`: it never appears on success rows, and
it is also absent on historical failed rows created before this column was
introduced (no backfill).

| `statusReason` | What happened |
|---|---|
| `build_timeout` | Dockerfile build did not finish within the backend timeout window (default 1800s) — typically still in `ImageBuildPushing[6]` when the deadline hit. Run `lbg sdbx image build-log <id>` to see how far kaniko got. |
| `kaniko_failed` | mid-lbg-image reported a non-zero kaniko exit (image-not-found, RUN command non-zero, etc.). `lbg sdbx image build-log <id>` shows the raw kaniko log. |
| `commit_failed` | ACS Commit CRD reported `phase=Failed`. `errorMsg` carries the controller-level reason; there is no kaniko-style stdout log for commit builds. |
| `commit_timeout` | Sandbox-commit did not reach `Succeeded` within the backend timeout window. |
| `register_failed` | Commit succeeded and the image is already in the registry, but the post-commit `RegisterImage` call to mid-lbg-image failed (rare, usually transient — safe to retry). |
| `crd_create_failed` | Commit CRD could not be created (sandbox missing, permissions, or a same-name CRD already exists). |

Use `--follow` only with **dockerfile** builds (`buildType=2`); the CLI rejects
`build-log --follow` on commit records (`buildType=1`) immediately, because
commit builds run inside Alibaba Cloud ECI and only expose ~5 coarse phase
messages, not per-line kaniko stdout. Inspect commit failures via `image get
<id>` (`statusReason` + `errorMsg`) instead.

Both `commit` and `build` require `projectId` in the request body. The
CLI sends the same value both in the body and as the `X-PROJECT-ID`
header, so either channel — `--project-id <id>` or any future global
X-PROJECT-ID source picked up by `SdbxSettings.project_id` — satisfies
the requirement. If neither is set, the CLI refuses the call with
`missing project id` (exit 2) instead of letting the backend return a
generic 400.

`image build` reads the Dockerfile locally and sends it inline; the
hard cap is **64 KiB** (matches the launching backend's
`BohrSandboxImageBuildReq` limit). The CLI rejects oversized files
client-side with exit 2. The `--json` echo of the request elides the
Dockerfile body — only `dockerfileSize` is shown — so logs stay clean;
the response (`id`, `imageUrl`) is unmodified.
56 changes: 56 additions & 0 deletions src/matcreator/skills/lbg-cli/references/platform-snapshot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Platform snapshot (as of 2026-04-30)

Pinned CPU / GPU template shortcuts (ephemeral; review by 2026-06-01).

> Pinned snapshot of platform-side values that may drift over time.
> Everything in this section is ephemeral — review/remove after
> 2026-06-01 or when these stabilise. For a live view, use the
> commands noted below.

## CPU template shortcuts

Platform-managed CPU template shortcuts. Pass the `name` directly to
`lbg sdbx create`. Resource sizes are observed `cpuCount` / `memoryMB`
from a create response and may change.

| name | cpuCount | memoryMB |
| --- | --- | --- |
| `sdbx-cpu-mini` | 1 | 3200 |
| `sdbx-cpu-small` | 2 | 5248 |
| `sdbx-cpu-medium` | 4 | 9344 |
| `sdbx-cpu-large` | 16 | 33920 |

## GPU template shortcuts

Platform-managed GPU template shortcuts. Pass the `name` directly to
`lbg sdbx create`, or use the `--gpu` shortcut on the CLI. GPU shortcuts
are platform-managed and may change over time — consult
`lbg sdbx template ls` for the current list.

| name | image | sku | `--gpu` key |
| --- | --- | --- | --- |
| `scicomp-4090` | `pytorch20-scicomp:1.0.6` | `c16_m64_1 * NVIDIA 4090` | `--gpu` (default) or `--gpu 4090` |
| `scicomp-5090` | `pytorch20-scicomp:1.0.6` | `c16_m64_1 * NVIDIA 5090` | `--gpu 5090` |
| `scicomp-l20` | `pytorch20-scicomp:1.0.6` | `c16_m128_1 * NVIDIA L20` | `--gpu l20` |

`--gpu` is mutually exclusive with passing a positional template name.
Bare `--gpu` resolves to `scicomp-4090` (the default GPU template).

## SKU catalog

SKU availability is dynamic. Use the live list instead of copying values
from documentation:

```bash
lbg sdbx machine list # list default SKU category
lbg sdbx machine list -c gpu # list GPU SKUs
lbg sdbx machine list --json # machine-readable SKU data
```

Pass the returned `sku_name` value to `lbg sdbx template create --sku-name`.
Quote SKU names that contain spaces or shell metacharacters.

## Example image

- Example image: `registry.dp.tech/dptech/abacus:LTSv3.10.1` (live list:
`lbg image ls`).
40 changes: 40 additions & 0 deletions src/matcreator/skills/lbg-cli/references/sandbox/exec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Sandbox exec

`lbg sdbx exec` usage, including `--background` mechanics and `--user`.

For the foreground vs background vs PTY mental model and the retrieve-before-kill SOP, see [`execution-modes.md`](./execution-modes.md).

## CRITICAL — always pass `--user root`

**Every `lbg sdbx exec` call MUST include `--user root`.** Without it, the
command runs as the sandbox's default non-root user (e.g. uid 1001), which
cannot read files uploaded via `--ti` (owned `root:root`, mode 640), cannot
write to system paths, and cannot install software. There is no scenario
where `--user root` should be omitted.

```bash
lbg sdbx exec --user root <sandbox_id> 'cd /workspace && python train.py'
lbg sdbx exec --user root <sandbox_id> 'cat log.txt | grep ERROR | wc -l'
lbg sdbx exec --user root <sandbox_id> 'echo hello > /tmp/out.txt'
```

## exec command passing

`exec` uses SSH-style pass-through: positional args are joined with spaces
and sent as a single shell string to `bash -l -c` inside the sandbox.
Shell operators work as written — no extra quoting is needed.

## Background jobs (`--background`)

`--background` returns immediately with a `pid` and lets the job keep
running. When `--background` is set, `--timeout` defaults to `0`
(unlimited); do **not** pass a finite `--timeout` unless you actually
want the job killed after that many seconds (the CLI prints a warning
in that case). Re-check state with `lbg sdbx ps <sandbox_id>` or by
reading output files via `lbg sdbx files read`.

```bash
lbg sdbx exec --background --user root <sandbox_id> 'python train.py > /tmp/out.log 2>&1'
lbg sdbx ps <sandbox_id> --json
lbg sdbx files read <sandbox_id> /tmp/out.log
```
Loading