diff --git a/.agents/environment.md b/.agents/environment.md index f10b77d..eb9ede9 100644 --- a/.agents/environment.md +++ b/.agents/environment.md @@ -322,3 +322,35 @@ vers=3,proto=tcp,port=…,mountport=…,nolocks,soft,nobrowse 127.0.0.1:/ ./mnt` approval), then `sudo umount -f`, then reboot. A hung `umount` does **not** die on `SIGKILL` — it is parked in the kernel — which is why `run()` in `src/nfs/mount.ts` settles on its own deadline instead of waiting for `close`. + +## Unprivileged interception, no mount (verified 2026-07-29, this Linux host) + +Established while spiking the `proot`-style `exec()` work +(`.agents/proot-plan.md`). Kernel 6.12.96 (Debian 13), glibc 2.43, x86-64, +zig 0.16. + +- **There is no `fusermount3` on this host at all** — not at `/usr/bin`, not + anywhere on `PATH`. So the rootless FUSE path (`src/fuse/fusermount.ts`) + cannot run here, and `pnpm test:rootless`'s FUSE column skips itself for that + reason rather than for a missing prebuilt. +- **Unprivileged user namespaces work**, and are the way around it: + `unshare -Urm` yields uid 0 with `CapEff: 000001ffffffffff`, `/dev/fuse` is + `crw-rw-rw-`, and `mount -t tmpfs` inside succeeds. A FUSE mount made in + there needs **no helper, no root and no native addon** — it is the ordinary + root path in `src/fuse/mount.ts`, which keys off `getuid() === 0`. The mount + is invisible in the host's `/proc/self/mounts` and dies with the namespace. +- **Node can never enter such a namespace itself.** `unshare(CLONE_NEWUSER)` + and `setns(2)` both require a single-threaded caller; Node has the libuv + threadpool up before user code runs. Any design here needs a child process. +- **`ptrace` is unrestricted** (`/proc/sys/kernel/yama/ptrace_scope` is `0`). +- **seccomp user notification works unprivileged.** A filter installed with + `SECCOMP_FILTER_FLAG_NEW_LISTENER` after `PR_SET_NO_NEW_PRIVS` succeeds, and + `SECCOMP_IOCTL_NOTIF_ADDFD` can inject a descriptor into the tracee. This is + true even though the shell already runs under a seccomp filter of its own + (`Seccomp: 2`, `Seccomp_filters: 1` in `/proc/self/status`) — filters stack. +- `zig cc` cross-compiles the three test linkages used by + `test/exec/compare.sh` with no toolchain beyond zig: dynamic glibc, static + musl (`-target x86_64-linux-musl -static`), and no-libc raw syscalls + (`-target x86_64-linux-none -nostdlib -static -ffreestanding -fno-builtin`; + without the last two flags the compiler emits calls to the `strlen` that + binary deliberately does not have). diff --git a/.agents/proot-plan.md b/.agents/proot-plan.md new file mode 100644 index 0000000..dadc174 --- /dev/null +++ b/.agents/proot-plan.md @@ -0,0 +1,368 @@ +# proot-style exec: three spikes, measured + +Question asked (2026-07-29): can mountx give a subprocess access to an +`FsDriver` **without a real kernel mount** — a `proot`-shaped `exec()` that +injects `LD_PRELOAD`, or any other universal syscall-inspection route? + +Answer: yes, two different ways, and they are not close in quality. This file +records what was built, what was measured, and what should happen next. + +**Status (2026-07-29): the recommendation at the bottom was carried out.** +Both surviving mechanisms ship behind one picker at `mountx/exec` — see the +roadmap's "Shipped since v1" entry, `docs/2.transports/6.exec.md` for the +user-facing page, and `AGENTS.md`'s code map for the file-by-file account. +`LD_PRELOAD` was dropped as recommended and is reachable from nothing that +ships. Neither mechanism has a conformance-matrix column, and nothing is +wired into `mountx/auto` — deliberately, since `auto`'s contract is a +mountpoint and this produces none. The measurements below are unchanged and +are what the decision rests on. + +## The structural finding + +**An interceptor does not need a filesystem. It needs a client.** + +Both no-mount spikes translate one intercepted operation into one or two 9P +messages against an unmodified `createP9Server()` — the same server +`mount9p()` points the kernel's v9fs client at. Path resolution, handle +lifetimes, directory paging, error mapping and every conformance question stay +in `src/9p/session.ts`. The native side is a wire adapter with an fd table and +no filesystem logic at all, which is why `src/exec/preload/p9.zig` is shared +verbatim between spike B and spike C. + +This is what makes the approach tractable. A `proot` that had to _be_ a +filesystem would be a second implementation of everything this repository +already tests. + +## What was built + +| | file | what it is | today | +| --- | ------------------------------------------ | ---------------------------------------------------------------------- | ------------------------------ | +| A | `src/exec/userns.ts`, `userns-relay.ts` | FUSE inside an unprivileged user namespace, driver stays in the parent | ships, first choice | +| B | `src/exec/preload.ts`, `preload/shim.zig` | `LD_PRELOAD` libc interposer → 9P | **rejected**, kept as evidence | +| C | `src/exec/seccomp.ts`, `seccomp/trace.zig` | seccomp user-notification supervisor → 9P | ships, second choice | +| — | `src/exec/preload/p9.zig` | the 9P2000.L client both B and C are built on | shared verbatim | +| — | `test/exec/probe.c`, `probe-raw.c` | one workload, three linkages | the spike harness | +| — | `test/exec/compare.sh` | builds everything and runs the matrix | the spike harness | + +## Measured results + +Host: Linux 6.12.96 (Debian 13) x86-64, glibc 2.43, zig 0.16, unprivileged user +namespaces available, **no `fusermount3` installed**, no root used anywhere. + +One workload (`test/exec/probe.c`) built three ways, each reading a 44-byte +file, stat-ing and reading a 3 MiB file whole, and listing a directory. "pass" +means the FNV checksum over the 3 MiB matched byte for byte. + +| | dynamic glibc | static musl | raw syscalls (the Go case) | +| -------------------- | ------------- | ----------- | -------------------------- | +| **A** userns + FUSE | pass | pass | pass | +| **B** `LD_PRELOAD` | pass | **fail** | **fail** | +| **C** seccomp notify | pass | pass | pass | + +Coreutils workload (`ls`, `cat`, `sha256sum`, `wc`, `tail`, `find`, `du`, +`cp -r`) on the dynamic-glibc case all three claim: + +| | sha256 | `wc -l < file` | `find -type f` | `du -s` | +| --- | ------- | -------------- | -------------- | ------- | +| A | correct | 100 | 5 | 3082 | +| B | correct | **0** | 5 | 3082 | +| C | correct | 100 | 5 | 3082 | + +That `0` is spike B's structural defect and is discussed below. + +Artifact sizes, `ReleaseSmall`: shim 153 KB, supervisor 154 KB. Both are +dominated by Zig's formatting machinery pulled in through `std.fmt` and would +shrink a lot under the same discipline `native/src/main.zig` already follows +(no allocation, no strings, no std). + +## Spike A — FUSE in a user namespace + +Not a "no kernel mount" answer, and included anyway because it is the honest +baseline: what the child sees _is_ FUSE, with the kernel's VFS in front of it, +so it inherits the entire conformance column the FUSE transport already +passes. It also ran the static and raw-syscall binaries without a line of +special handling, because nothing about it depends on what the child is linked +against. + +It is a mount, but a mount **nobody outside the process tree can see** — +`/proc/self/mounts` on the host stays empty, and the mount dies with the +namespace. The roadmap ruled a user-namespace mode out when the goal was +"mount a directory for the machine"; for an `exec()`-shaped API the tradeoff +inverts, and invisible is the point. + +Four things it cost, all witnessed — the first three during the spike, the +last one while productionising it: + +- **Node can never enter the namespace.** `unshare(CLONE_NEWUSER)` requires a + single-threaded caller and Node is never single-threaded; `setns(2)` has the + same rule. So the namespace is entered by a child, `/dev/fuse` is opened + there, and the traffic comes back out over a unix socket to a `FuseSession` + in the parent. That relay is not a workaround for the spike — it is the shape + any version of this must take, and it is the "relay mode" the roadmap defers, + arrived at from the other direction. +- **The relay deadlocks if it spawns into its own mount.** Setting the child's + `cwd` to the mountpoint wedged the relay in `D` state at `fuse_get_req` + permanently: `uv_spawn` blocks the calling thread until the child execs, and + the child's first act was a `chdir` that only that thread could answer. Same + hazard `src/fuse/mount.ts` and `src/9p/mount.ts` both document, met from the + inside. Fixed by passing the mountpoint as `$MOUNTX_ROOT` so the `cd` happens + after the exec. +- **A killed parent orphans a wedged mount.** Handled by having the relay exit + when the socket closes. +- **`default_permissions` makes the mount read-only by accident.** Found while + productionising A (2026-07-29), not during the spike, because the spike's + workload never wrote. The option asks the _kernel_ to check the driver's + `uid`/`gid`/`mode` against the caller's credentials, and inside the namespace + those are not the same identity space: `unshare -r` maps exactly one uid, so + a driver reporting the serving process's real uid reports an identity the + namespace renders as `nobody`, and a `nobody`-owned `0755` root directory + refuses every write from the one process meant to have it. Namespace-root's + `CAP_DAC_OVERRIDE` does not rescue it either — that capability does not reach + a file owned by an unmapped uid. It is no longer the default; permission + checking stays with the driver, and the mount carries no `allow_other`, so + the only process that can reach it is the one it was created for. + +Also worth recording, because it changes what `mountx/auto` could do on this +class of host: **this host has no `fusermount3` at all**, so today's rootless +FUSE path cannot run here — yet inside `unshare -Urm` the process is uid 0 with +`CAP_SYS_ADMIN`, `/dev/fuse` is 0666, and the ordinary root mount path works +verbatim with no helper and no native addon. + +## Spike B — `LD_PRELOAD` + +Works, and should not be shipped. **The code has since been removed from the +branch** (`src/exec/preload.ts`, `preload/shim.zig`, `demo-preload.ts`); what +follows is why, and it is kept because the reasoning is what justifies the +shape of the two mechanisms that remain. The 9P client it shared with the +seccomp supervisor stayed and now lives at `src/exec/seccomp/p9.zig`. + +It reached a genuinely useful level of function — `ls -la`, `cat`, `grep`, +`tail`, `sha256sum`, `find`, `du` and a full `cp -r` of the tree all behave — +across 49 exported symbols. Getting there took seven distinct discoveries, and +the pattern they form is the finding: + +1. **`statx` is load-bearing.** glibc 2.33+ routes the public `stat()` to + `statx` _internally_, without a PLT hop, so interposing `stat` and `fstatat` + catches nothing. A shim without `statx` serves `cat` and is invisible to + `ls`. +2. **stdio bypasses `read` entirely.** `sha256sum` imports `fopen`, not `open`. + Bridging it the obvious way — this shim's `open`, then the real `fdopen` — + _appears_ to work and is silently wrong: glibc's `FILE` machinery reads the + descriptor through an internal read the shim never sees, so `sha256sum` on a + 3 MiB file returned **the hash of the empty string, exit status 0**. The fix + is to slurp the file into a `memfd` at `fopen` time, which costs the whole + file in memory and cannot write back. +3. **Fortified variants are separate symbols.** `tail -2` printed nothing and + exited 0 because it imports `__read_chk`. `-D_FORTIFY_SOURCE=2` is the + default on Debian, Fedora and Ubuntu. +4. **`fdopendir` is where `find` dies** — "Not a directory" on a path the shim + had just listed. +5. **`*at()` resolution is needed in three places.** `openat`, `fstatat` _and_ + `statx` each need the relative-against-our-dirfd branch, and missing any one + fails differently. Modern coreutils reach `statx`, so with that one missing, + `find` and `du` reported ENOTDIR for every entry of a directory they had + just enumerated correctly. +6. **`dup` breaks descriptor identity.** `fts` duplicates the directory fd + before walking it, and the duplicate was a real `dup` of the shim's + `/dev/null` placeholder. Handling it means giving the duplicate its own fid, + which **diverges from POSIX**: a real `dup` shares the file offset, this one + does not. +7. **`getxattr` must be answered** or `ls -l` prints every mode string with a + trailing `?`. + +So the surface is not "the POSIX names". It is those names crossed with the +`64` suffix, the `__*_chk` suffix, and the legacy `__xstat` versioned symbols — +with the landing site decided by the glibc a program was _compiled_ against. +That is a maintenance surface that grows with other people's releases. + +**And one hole cannot be closed at all.** A descriptor the shim created does not +survive `exec`: the fd number is inherited but the shim's table lives in +process memory that `exec` discards. So a shell redirection into the virtual +tree silently yields an empty file — `wc -l < /mountx/numbers.txt` returns `0` +where the other two return `100`. There is no symbol to interpose that fixes +this; the state would have to live outside the process, which is what spike C +does by construction. + +Every one of spike B's worst failures is a **silent wrong answer**, not an +error. For a filesystem library that is the wrong failure mode to design in. + +## Spike C — seccomp user notification + +The mechanism that actually answers the question. + +A BPF filter traps eight syscalls and everything else runs natively without +leaving the kernel. Because the boundary is the syscall ABI, the traced program's +linkage is invisible: the static musl binary and the no-libc raw-syscall +binary — neither of which spike B can see at all — pass byte-exact. + +One structural nicety: **no descriptor is passed anywhere.** The usual shape of +this forks, has the child install the filter, and hands the listener fd back +over `SCM_RIGHTS` — the exact `recvmsg` dance that needed a native addon for +`fusermount3`. A seccomp filter is inherited across fork and exec, so the +supervisor installs the filter on _itself_, keeps the listener, and forks. The +price is a rule the file must keep: after installing, the supervisor may never +make a trapped syscall itself, or it suspends waiting for a reply only it can +send. Violating that (an `open("/dev/null")` inside the openat handler) hung +everything the first time anything opened a directory. + +What the spike also measured: + +- **`fstat` is its own syscall number**, and glibc's `opendir` uses it to check + what it just opened. Untrapped, `opendir` saw the placeholder's real type and + answered ENOTDIR. +- **musl uses the legacy `open`(2)/`stat`(4)/`lstat`(6)**, not the `*at` + forms — so even a syscall boundary has more than one door per operation. The + difference from spike B is that this set is finite and fixed by the kernel + ABI rather than growing with each libc release. +- **`dup` is solvable here, unlike in spike B.** Every injected descriptor is a + `memfd` with a unique name, so `/proc//fd/` identifies it as + `/memfd:mx-` for the original and every duplicate alike. No trapping + of `dup` required, and it survives `exec` and `fork` for free. +- **Not trapping `close` costs correctness, not just memory.** Descriptor + numbers get reused immediately, and a stale mapping shadowed a live one: + `fstat` on a freshly opened file returned the _directory_ that previously + held that number, and `cp -r` correctly concluded the file had been replaced + underneath it. Fixed by evicting on reuse; the underlying fid leak stays until + a supervisor exists that does not share a filter with its tracee. +- **`stat` and `statx` must agree on `st_dev`.** `statx` reports a major/minor + pair that glibc recomposes with `makedev()`; a minor over 255 does not + round-trip, so the two forms disagreed and `cp` again reported a replaced + file. Both spikes now use a minor under 256. (Spike B had the same latent + bug and was fixed alongside.) + +Costs as first spiked, and what became of each: + +- ~~A file open copies the whole file into a `memfd`.~~ **Closed.** I/O is + streamed per open-file-description now — `read`/`pread64`/`readv` and the + `write` family and `lseek` are all trapped, with an offset that `dup` shares + — and no copy is made. The one thing the `memfd` bought and streaming cannot + is `mmap` of a file on the tree, which now answers `ENODEV` rather than + silently mapping a stale copy. +- ~~Read-only as spiked.~~ **Closed, and this was the important one.** The + read-only version did not fail cleanly: `memfd`s are always writable and + `write` was untrapped, so a program's writes landed in the in-memory copy and + vanished. Measured at the time: `dd conv=notrunc` reported success and left + the file unchanged; `rm -f` reported success and left the file in place. That + is the same silent-wrong-answer failure class this document rejects spike B + for, and it is why `test/exec/compare.sh` grew a write-back row that asserts + against the driver rather than against an exit status. +- **x86-64 only**, since the filter compares against one syscall table. arm64 is + a second table, not a redesign. +- **Still one notification at a time**, tag always zero, one 9P request in + flight. Multi-threaded and multi-process tracees are correct — the tables are + keyed on the thread group behind `seccomp_notif.pid`, which is a _thread_ id, + and they grow — but they serialize. +- **`execve` of a binary living on the tree is not supported** and was never in + scope; it fails at 127 rather than hanging. + +## Portability: what each one actually needs on a bare system + +Asked directly (2026-07-29): does spike A work on a bare Alpine system with no +kernel modules and no extra shared libraries? Tested in `podman` against +`alpine:latest` and `node:24-alpine`, busybox-only, no `util-linux`, no +`fuse3`, no `fusermount3`. + +**Userland: nothing extra needed, after one fix.** busybox provides both +`unshare` and `mount` applets, and spike A needs no native addon at all. The +one incompatibility found: busybox 1.37's `unshare` has `-r` but **no +`--map-root-user`**, so the long spelling fails outright. `src/exec/userns.ts` +now spawns `-U -r -m --propagation private`, which util-linux and busybox both +accept. + +**Privileges: none.** Verified with `--cap-drop=ALL --user 1000:1000`, no +`--privileged`: the tree mounts and reads back byte-exact. + +**Kernel: yes, and it is not avoidable.** Spike A _is_ FUSE, so it needs the +host kernel's `CONFIG_FUSE_FS` and a working `/dev/fuse` node. + +- `alpine:latest` has **no `/dev/fuse`**, and spike A fails there with an + accurate message rather than anything mysterious. +- A user-namespace root **cannot create it**: `mknod /d/fuse c 10 229` inside + `unshare -Urm` answers `EPERM`. Verified. So the node has to come from + outside — `--device /dev/fuse`, or a host that has it. +- With `--device /dev/fuse` supplied it works completely: full listing, 3 MiB + read, sha256 `bfe74807…` matching every other column. + +**Spike C needs none of that.** The same question, same image, **without** +`--device`, `--cap-drop=ALL`, uid 1000, using a statically linked musl build of +the supervisor: full pass, byte-exact, on the no-libc `probe-raw` binary. It +needs `CONFIG_SECCOMP_FILTER`, which is built into every mainstream kernel and +cannot be a module, plus `no_new_privs` — and no device node, no filesystem +driver, and no shared library of any kind. + +Two portability fixes came out of building the static musl supervisor, both +real bugs rather than build-system friction: + +- **`ioctl` differs between libcs.** glibc's takes `unsigned long`, musl's takes + `int`, and the `SECCOMP_IOCTL_*` request numbers have the high bit set — so + the constants do not survive musl's prototype. The supervisor now issues + `ioctl` as a raw syscall, which takes an unsigned long on every ABI. +- **Reading a path out of another process must respect page boundaries.** + `process_vm_readv` fails the _entire_ request if any part of the range is + unmapped, so reading a fixed 64-byte chunk fails whenever the path sits near + the end of a mapping. This was an intermittent, ASLR-dependent failure: + `probe-raw` would read two files and then fail to open a directory, because + that one path was the environment string at the top of the stack. Reads are + now clamped to the end of the current page. 3/3 on repeat runs after the fix. + +### What this changes about the recommendation + +It sharpens it rather than reversing it. Spike A remains the cheapest correct +thing on a normal Linux host or any container given `--device /dev/fuse`. But +the environments where "no kernel mount" is _wanted_ — a locked-down container, +a CI runner, an unprivileged sandbox — are exactly the ones that withhold +`/dev/fuse`, and there A cannot be made to work from inside by any means. +Spike C is the only one of the two that runs there. + +So: land A for the common case, and treat C as the one that covers the case +that motivated the question in the first place. + +## Recommendation + +1. **Drop spike B.** It is the most familiar approach and the worst one here. + Its coverage excludes Go and static binaries by construction, its symbol + surface tracks other projects' releases, it has a hole (`exec`) that cannot + be closed from inside the process, and its characteristic failure is a + confident wrong answer rather than an error. Keep the file as the written-up + evidence for why, not as a thing to finish. + +2. **Land spike A as the near-term `mountx/exec`.** It is nearly free, it is + _actually FUSE_ so it inherits the existing conformance guarantees whole, it + covers every binary, and on a host like this one it is the only FUSE route + that works at all. Scope it honestly: Linux with unprivileged user + namespaces, and say plainly that it is a namespace-private kernel mount + rather than no mount. + +3. **Pursue spike C as the real no-mount transport.** ~~Next milestone: + streaming rather than slurping, plus a supervisor that can trap `close`.~~ + **Both done.** I/O streams per descriptor, and `close` is trapped, which + required exactly the predicted change — the supervisor no longer shares its + filter with the tracee; the child installs it and passes the listener back + over `SCM_RIGHTS`, reusing `native/src/main.zig`'s cmsg transcription. It + carries the full conformance column (`test/exec/seccomp-conformance.test.ts`, + 65 of 66 cases, the one skip being the root-only `lchown` case every column + skips). What is left is concurrent dispatch, `mmap`, `execve` off the tree, + and arm64. + +4. **Keep `p9.zig` as the shared asset** whichever way this goes. It is the part + that made both spikes small, and it is the reason neither one contains a + filesystem. + +5. **macOS gets nothing from any of this.** No user namespaces, no seccomp, and + SIP blocks `DYLD_INSERT_LIBRARIES` for system binaries. macOS stays NFS-mount + territory. + +## Reproducing + +```sh +sh test/exec/compare.sh # builds all three, runs the matrix, no root +node src/exec/demo-userns.ts # userns + FUSE +MOUNTX_SHIM=… node src/exec/demo-preload.ts +MOUNTX_TRACE=… node src/exec/demo-seccomp.ts +MOUNTX_TRACE_DEBUG=1 # per-syscall tracing for the supervisor +``` + +The runners were `spike-a.ts`/`spike-b.ts`/`spike-c.ts` when the measurements +below were taken; all three are now named after their mechanisms. + +The command sees the driver at `$MOUNTX_ROOT`, which all three set. diff --git a/.agents/roadmap.md b/.agents/roadmap.md index c76c03a..9e9cac3 100644 --- a/.agents/roadmap.md +++ b/.agents/roadmap.md @@ -139,6 +139,38 @@ results. column (no host could mount 9P when the benchmark suite was last run — see "Future / deferred", now that one can). +- **`mountx/exec`** (2026-07-29). A `proot`-shaped `exec()`: run a command + with a driver grafted onto its filesystem view at `$MOUNTX_ROOT`, visible + to that process tree and to nothing else on the machine, resolving with + that command's exit status. Three mechanisms were built and measured + (`.agents/proot-plan.md`); two ship behind one picker. `probeExec()` + publishes what each can do here and why not, `exec()` takes the first + usable one in preference order — the **user namespace** (FUSE inside + `unshare -U -r -m`, driver in the parent, traffic relayed over a unix + socket because `unshare(CLONE_NEWUSER)` refuses a threaded caller) where + the kernel's FUSE is usable, the **seccomp** user-notification supervisor + otherwise, which is the case that motivated the question: a container that + withholds `/dev/fuse` withholds it from a namespace root too (`mknod` + answers `EPERM`, verified on `alpine:latest`). Both arrive through + `await import()`; the result is the mechanism's own object with a + `mechanism` discriminant defined on it. `src/exec/probe.ts` is import-light + in `src/nfs/probe.ts`'s sense and names causes a caller can act on rather + than one errno. + Deliberately **outside `mountx/auto`**, whose contract is a mountpoint this + produces none of — the line `mountx/s3` already sits on. + What was deliberately **not** done: **`LD_PRELOAD` was rejected** rather + than finished (it cannot see a Go or static binary by construction, its + symbol surface tracks other projects' releases, a descriptor it creates + does not survive `exec`, and its characteristic failure is a confident + wrong answer — the code stays as the written-up evidence, reachable from + its own runner and the comparison harness and from nothing that ships); no + conformance-matrix column for either mechanism (the `userns` one would + duplicate FUSE's exactly, the `seccomp` one is not ready for it); no + supervisor binary in the npm package, so the seccomp mechanism needs a Zig + toolchain and `$MOUNTX_TRACE`; and no `default_permissions` on the + namespace mount, because the kernel checking a driver's uid against a + namespace that maps exactly one turns every write into `EACCES`. + ## Finalized decisions (still binding) - **Scope:** FUSE (Linux) + NFSv3 loopback transports. WebDAV deferred. @@ -171,6 +203,29 @@ rather than by accident. is also where `src/9p/`'s deferred `trans=fd` would finally earn its keep: it wants a descriptor the relay already holds, where `mount9p()`'s own `trans=unix` has nothing to relay to. +- **`mountx/exec`'s seccomp mechanism, past the spike.** It ships as the + second choice and covers the case `userns` cannot, but four things are + open and each is named in `src/exec/seccomp.ts` and + `.agents/proot-plan.md`: **streaming instead of slurping** (a file open + copies the whole file into a `memfd`, which is what buys native + `read`/`lseek`/`mmap` afterwards and is wrong for a large file and for + anything that writes — trapping `read`/`write`/`lseek` per descriptor is + the fix, the way `getdents64` already is); **write-back at all**, since it + is read-only as spiked; **arm64**, which is a second syscall table rather + than a redesign; and **trapping `close`**, which means not sharing a filter + with the tracee, i.e. the `SCM_RIGHTS` shape after all — for which + `native/` already has `recvFd`. Also: the supervisor is not in the npm + package, so the mechanism needs a Zig toolchain and `$MOUNTX_TRACE` today. +- **A conformance-matrix column for `mountx/exec`.** Neither mechanism has + one. The `userns` one would duplicate the FUSE column exactly (what the + child sees _is_ FUSE, through the kernel's VFS), which is an argument for + never writing it rather than for writing it later; the `seccomp` one is + the interesting one and wants the streaming work above first, since a + read-only column would be mostly skips. +- **`mountx/exec` on macOS.** Nothing from any of this transfers: no user + namespaces, no seccomp, and SIP blocks `DYLD_INSERT_LIBRARIES` for exactly + the system binaries anyone would want to run. macOS stays NFS-mount + territory, and the honest answer is that this feature is Linux's. - **A 9P bench column.** `bench/` has loopback and NFS columns and a sudo-gated FUSE one; 9P has none yet, and unlike when the transport was designed, a host that can mount it now exists (`.agents/environment.md`). diff --git a/AGENTS.md b/AGENTS.md index 17a9570..d55961e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ Keep important information about project in AGENTS.md. For more detailed info, p # mountx -Mount a JavaScript filesystem: one driver interface (a subset of `node:fs/promises`), multiple transports (FUSE first, then 9P, then NFS — v3, then v4.1) — plus an S3 gateway (`mountx/s3`) that serves the same driver to an S3 client instead of mounting it, deliberately outside `mountx/auto`. User-facing docs live in `docs/` (); `README.md` is the landing page for npm and GitHub and links there. +Mount a JavaScript filesystem: one driver interface (a subset of `node:fs/promises`), multiple transports (FUSE first, then 9P, then NFS — v3, then v4.1) — plus two things that are not mounts and are both deliberately outside `mountx/auto`: an S3 gateway (`mountx/s3`) that serves the same driver to an S3 client, and `mountx/exec`, which grafts it onto one command's filesystem view for as long as that command runs. User-facing docs live in `docs/` (); `README.md` is the landing page for npm and GitHub and links there. Conventions: pure JS/TS, zero runtime deps, pure-JS-first. Single package with subpath exports. Small conventional commits to `main`, tests green (`pnpm test`) before each commit. @@ -79,6 +79,16 @@ S3 (`src/s3/`, exported as `mountx/s3`): the transport that is not a mount — i - `server.ts` — `createS3Server(source, options)`, the only file here that imports `node:http`. With no `credentials`, binds loopback-only and refuses any other `host` with a named `S3BindError`; with credentials, verifies every request and allows any bind. `close()` drains in order: stop accepting, let in-flight replies finish, drop the rest, sweep multipart staging last. - `index.ts` — re-exports the above by name, minus `xml.ts`'s generic XML primitives it is built on (the same treatment `mountx/nfs` gives its sub-struct helpers). +Exec (`src/exec/`, exported as `mountx/exec`): the other transport that is not a mount — it grafts an `FsDriver` onto **one command's** filesystem view, at `$MOUNTX_ROOT`, visible to that process tree and to nothing else on the machine, and hands back the command's exit status. Deliberately outside `mountx/auto` for the same reason `src/s3/` is: `auto`'s contract is a mountpoint, and this produces none. Linux only, and no root on any path. + +- `probe.ts` — `usernsExecProbe()`/`seccompExecProbe()` and the `ExecPlatform` narrowing, import-light in the exact sense `src/nfs/probe.ts` and `src/9p/probe.ts` are — `node:fs` and nothing else, so asking never pulls in a FUSE session or a 9P codec. The `/dev/fuse` check is an `open(2)` rather than a `stat`, and it runs **before** `/proc/filesystems` is read, because `/dev/fuse` is a misc device and opening it is what autoloads the module that puts `fuse` in that table; it also tells the two actionable failures apart, and says of the `EACCES` one that a namespace does not rescue it (`unshare -r` maps exactly one uid, so the check the relay makes inside is the one that already failed outside, and namespace `CAP_DAC_OVERRIDE` does not reach a node owned by an unmapped uid). Three sysctls cover the three distribution idioms for disabling unprivileged user namespaces (`user.max_user_namespaces`, Debian's `unprivileged_userns_clone`, Ubuntu 23.10+'s `apparmor_restrict_unprivileged_userns`), the last two read only when not root because they gate the unprivileged path alone. `NOT_THIS_HOST` is the one verdict a `platform` override forces: every fact below `platform` is a file about _this_ machine, so answering `usable` for Linux from somewhere that is not Linux would rest on checks that never ran. +- `index.ts` — `exec(driver, argv, options)` and `probeExec()`, in `src/auto.ts`'s shape and with all three of its refusals for its reasons: no fallback after a failure (worse here than for a mount — the mechanisms do not have the same semantics, and a command that already ran may have had effects outside the driver), no probe when a mechanism is named, and no loading of what it does not use (each mechanism arrives via `await import()`). The preference is `userns` then `seccomp`, one order on every host because off Linux neither works. The result is the mechanism's own result object with a `mechanism` discriminant defined on it — tagged, not wrapped. Shared options are the ones that mean the same thing in both (`root`, `cwd`, `env`, `useDriverIno`, `debug`); everything else goes in `userns: {…}`/`seccomp: {…}`, because the two `onError`s are genuinely different shapes. +- `userns.ts` — `execUserns()`: FUSE inside an unprivileged user namespace, the driver staying in this process. `unshare(CLONE_NEWUSER)` demands a single-threaded caller and Node is never single-threaded, so the namespace is entered by a child (`userns-relay.ts`) and raw FUSE traffic comes back over a unix socket to a `FuseSession` here — the "relay mode" the roadmap defers, arrived at from the other direction. `cwdRefusal()` is pure and exported: a `cwd` inside the mountpoint is the `uv_spawn` deadlock `src/fuse/mount.ts` and `src/9p/mount.ts` document, met from the inside, and it is refused rather than hung on. `DEFAULT_MOUNT_OPTIONS` is **empty on purpose** — `default_permissions` asks the kernel to check a driver's uid against a namespace that maps exactly one, so a driver reporting the serving process's real uid renders as `nobody` and every write fails; permission checking stays with the driver, and nothing is lost because the mount carries no `allow_other`. `relayPath()` resolves the sibling relay as `.mjs` then `.ts`, so it works from `dist/exec/` and from source alike. +- `userns-relay.ts` — the only thing that runs _inside_ the namespace, and it has no imports from `src/`. Opens `/dev/fuse`, spawns `mount(8)` (in there it is uid 0 with `CAP_SYS_ADMIN`, so this is the ordinary root path — no `fusermount3`, no setuid bit, no native addon), and pumps whole messages both ways; FUSE's own `len` field is the framing, and the one rule the socket does not carry is that a reply must reach the device in a single `write(2)`, which is what the reassembly buffer is for. The command's `cwd` is deliberately _not_ the mountpoint — the mountpoint travels as `$MOUNTX_ROOT` so a `cd` happens after the exec. A closed socket exits the relay, because a parent that went away leaves every later request parked in `fuse_get_req` forever. `fail()` also writes its message to `$MOUNTX_RELAY_STATUS`: the relay's exit code is indistinguishable from the command's, and "the command exited 70" is a thing that happens. +- `seccomp.ts` + `seccomp/trace.zig` — `execSeccomp()`: a seccomp user-notification supervisor over an unchanged `createP9Server()`. Needs no device node, no kernel module and no shared library — the boundary is the syscall ABI, so a static musl binary and a no-libc raw-syscall binary are served identically to a `cat`. Read-only, x86-64 only, and its supervisor is a separately built binary the npm package does not ship, which is why it is second in the preference order rather than first. +- `seccomp/p9.zig` — the 9P2000.L client small enough to live inside a traced process. It is the reason the supervisor contains no filesystem: an interceptor does not need a filesystem, it needs a **client**, and everything that decides what the filesystem does stays in `src/9p/session.ts`. It lived under `preload/` while a second, rejected mechanism shared it — see `.agents/proot-plan.md`, which keeps that mechanism's measurements now that its code is gone. +- `demo-driver.ts`, `demo-userns.ts`, `demo-seccomp.ts` — test benches, not entry points: one demo tree and one runner per mechanism, each calling its mechanism _by name_ so `test/exec/compare.sh`'s matrix compares mechanisms rather than whatever the picker would have chosen. + CLI (`src/cli/`, the `mountx` bin, `pnpm play` from source): - `index.ts` — `node:util`'s `parseArgs`, then a memory driver holding one file — this package's own `README.md`, read through `new URL("../../README.md", import.meta.url)`, which is the package root from `src/cli/` and from `dist/cli/` alike, and npm publishes a README whatever the `files` list says — wrapped in `watch.ts` and mounted through `mountx/auto` at `~/mountx` (`[mountpoint]`/`-m`/`$MOUNTX_MOUNTPOINT`; `-t`, `-q`, `-r`, `--empty`, `--allow-other`). It is a demo and a test bench, not a mount tool — what it serves is always a tree that dies with the process. `process.exit` appears only in paths that run _before_ the mount (with one up it wedges), and the stale-mount cleanup detaches only a `fuse*`/`nfs*`/`9p` mount at that exact path (matched whole for `9p`, so a future `9p2` stays untouched), printing the `sudo` line rather than spawning one when the route needs root. It runs on **both hosts**: Linux reads `/proc/self/mounts` inline, macOS asks `mountEntryAt()` from `src/nfs/mount.ts` (dynamically imported, so a Linux run never loads the NFS codec) and clears with an unprivileged `umount -f`, which works because a BSD lets the mounting user unmount. Linux+NFS and Linux+9P both have no unprivileged route and get the `sudo` line — no worse than it sounds, since both needed root to be mounted in the first place. Bounded by `STALE_TIMEOUT`, because the macOS consent gate turns `umount` into a call that never returns — on expiry it prints `consentAdvice()` and lets `mount()` refuse. @@ -94,6 +104,7 @@ Tests (`test/`): - `fuse/` — protocol/session Tier 0 (`random.ts`, `protocol.test.ts`, `golden.test.ts`, `dirent.test.ts`, `init.test.ts`, `flags.test.ts`, `session.test.ts`, `inodes.test.ts`, `session-fuzz.test.ts`, `synthetic-kernel.ts`, `fuzz.test.ts`), `fusermount.test.ts` (the elevation checks and the device-refusal advice, Tier 0 — pure, so it runs on a host with no `fuse3` at all), `native.test.ts` (the whole addon, Tier 0 — passing a descriptor to yourself needs no helper and no privileges), Tier 2 `mount.test.ts` and `mount-rootless.test.ts` (no sudo), the differential oracle (`differential.ts`+`differential.test.ts`), record/replay (`record-fixtures.ts`+`replay.test.ts`), the FUSE conformance column (`conformance-mount.test.ts`). - `nfs/` — Tier 0 for the shared layer (`xdr.test.ts`, `handles.test.ts`, `mount-options.test.ts` — the platform difference, checked from either host) plus `session.test.ts` for the version router alone: which session a `(prog, vers)` pair reaches, and nothing about what it does once it arrives (that is `v3/session.test.ts`'s and `v4/session.test.ts`'s job). In `v3/` beside the code it covers: Tier 0 for the protocol (`protocol.test.ts`, `golden.test.ts`, `fuzz.test.ts`) plus the Tier-1 JS client (`v3/client.ts`) and its conformance column (`v3/conformance.test.ts`, `v3/session.test.ts`). In `v4/`: `constants.test.ts` (the transcription check — RFC spot-checks at distinct points plus whole-table shape assertions, no gap, no repeat, every value named), `attr.test.ts`, `protocol.test.ts`, `golden.test.ts`, `fuzz.test.ts` for the codec, `state.test.ts` for the state machine alone (synchronous, no socket — the replay cache and the lease clock proved with an injected clock rather than a real `setTimeout`), `session.test.ts` for COMPOUND dispatch driven with encoded bytes, and the Tier-1 JS client (`v4/client.ts`, which does its own path-walking and POSIX-vs-NFSv4 op-collapsing — `unlink` vs `rmdir`, OPEN not being for directories) plus `v4/driver.ts` (the `FsDriver` over it) and `v4/conformance.test.ts`, so NFSv4.1 is a conformance-matrix column of its own alongside loopback, FUSE, 9P, NFSv3 and S3. Tier 2 `mount.test.ts` is v3-only so far (gated on `nfsClientProbe()`; sudo on Linux, `pnpm test:rootless` on macOS, where it needs none) — a real-mount NFSv4.1 column is not written yet; the dev host has no `mount.nfs` to write it against either way. - `s3/` — Tier 0 (`sigv4.test.ts` against the official `aws-sig-v4-test-suite` goldens, `xml.test.ts`, `chunked.test.ts`, `protocol.test.ts`, `constants.test.ts` — the errno↔S3-error table's totality), `server.test.ts` (real sockets, driven with `fetch`), the Tier-1 signing JS client (`client.ts`, the `test/nfs/v3/client.ts` pattern) and its conformance column (`conformance.test.ts`, `session.test.ts`, in-process against the memory driver, no sockets), and `oracle.test.ts` — a real `rclone`/`curl` against the gateway, gated on `command -v rclone`/`curl` (the `nfsClientProbe` pattern) and needing no root, so it runs as part of `pnpm test` and skips clean when either binary is absent. +- `exec/` — `strategy.test.ts` is Tier 0 for `mountx/exec`'s choice, the way `auto.test.ts` is for the transports': the preference order, the reason each mechanism is ruled out, the named-mechanism paths that must not consult the probe, and `cwdRefusal()`, whose only other way of being checked costs a hung process. Answered for darwin, win32 and arm64 from any host through the `platform`/`arch`/`supervisor` overrides — including the refusal to answer for Linux from a host that cannot read Linux's files. `userns.test.ts` is Tier 2 and the one mount suite in the repo that needs neither root nor a `fusermount3`; it is gated on `usernsExecProbe().usable` plus the same raised-threadpool rule the FUSE rootless file uses, and it runs under `pnpm test:rootless`. It does not re-test the filesystem — what the command sees is FUSE, which has its own conformance column — only what is different: the driver at `$MOUNTX_ROOT`, a write landing in the driver, the mount staying out of the _host's_ `/proc/self/mounts`, the command's own status surviving, and a relay failure arriving as an error. `compare.sh`+`probe.c`/`probe-raw.c` are the spike harness the comparison in `.agents/proot-plan.md` is measured with: one workload in three linkages (dynamic glibc, static musl, no libc at all), run through every mechanism, needing a Zig toolchain and no root. - `pjdfstest/` — `run.sh`+`run.ts` drive the pinned pjdfstest clone (gitignored) against a real mount and write the committed analysis. - `matrix.ts` — generates `.agents/conformance-matrix.md`. Its `unmetIn()` counts a capability as met when **at least one** target in the column passed a case naming it, not every one — the drivers sharing a column need not have the same capabilities now that `unstorage` runs beside `memory`. `root.sh` — runs any Tier-2 vitest file under sudo with the environment fixed up (raised `UV_THREADPOOL_SIZE`, redirected `TMPDIR`, forwarded `MOUNTX_*`); every Tier-2 file skips itself when not root. `rootless.sh` is the same idea minus the `sudo` and minus everything `sudo` made necessary — `UV_THREADPOOL_SIZE` is all that is left, and `mount-rootless.test.ts` skips itself unless it has been raised. @@ -108,7 +119,7 @@ Native (`native/`), the only non-JS in the repository: `bench/` — `harness.ts` (warmup, adaptive loop, percentiles), `scenarios.ts` (written once against the driver interface), `index.ts` (loopback + NFS columns), `fuse.ts`+`fuse-client.ts` (the FUSE column, client in a child process); generates `.agents/benchmarks.md`. -Docs (`docs/`) — the [undocs](https://undocs.dev) site at . A **standalone pnpm project**, deliberately outside the root workspace (its own `package.json`, lockfile and `pnpm-workspace.yaml`), so the site's dependency tree never reaches the package's: `pnpm install && pnpm dev` inside `docs/`. Three sections, numbered-prefix routing (`1.guide/` → `/guide`): `1.guide/` (introduction, quick start, the CLI, `3.drivers/` — a directory: the interface, then the built-in three and writing your own — mounting, tuning, troubleshooting), `2.transports/` (the overview of what `auto` is choosing between, then `mountx/auto`, FUSE, 9P, NFS — one page carrying both v3 and v4.1, since they are one server behind one `mountNfs()` — and the S3 gateway, guide-level prose first and the full export surface below it, because a transport's API and the protocol it speaks are one subject and two pages of it drift), `3.reference/` (what is left once each entry point is documented beside what it does: the root `mountx` export, split by subject rather than carried as one long page — the driver interface, capabilities, errors, the loopback harness, paths and locking — plus the index that maps every subpath to its page and lists those five — the `mountx/drivers/*` subpaths live in the guide beside the interface they implement, `mountx/auto`, `mountx/fuse`, `mountx/9p`, `mountx/nfs` and `mountx/s3` in `2.transports/`, and the CLI in the guide, since it is a demo and a test bench rather than an entry point). `.config/docs.yaml` is the landing page and the site config; `.docs/public/` holds the icons. **This is where user-facing prose goes now.** `README.md` was cut back to the intro, its snippet, install and a link list when the site landed — a second full copy is one that goes stale, and the CLI mounts the README, so it stays short on purpose. Anything longer than a paragraph belongs on a page here. +Docs (`docs/`) — the [undocs](https://undocs.dev) site at . A **standalone pnpm project**, deliberately outside the root workspace (its own `package.json`, lockfile and `pnpm-workspace.yaml`), so the site's dependency tree never reaches the package's: `pnpm install && pnpm dev` inside `docs/`. Three sections, numbered-prefix routing (`1.guide/` → `/guide`): `1.guide/` (introduction, quick start, the CLI, `3.drivers/` — a directory: the interface, then the built-in three and writing your own — mounting, tuning, troubleshooting), `2.transports/` (the overview of what `auto` is choosing between, then `mountx/auto`, FUSE, 9P, NFS — one page carrying both v3 and v4.1, since they are one server behind one `mountNfs()` — the S3 gateway, and `mountx/exec` — which is here rather than in the guide for the reason S3 is: a transport is everything between the driver and whatever is going to read it, and "whatever is going to read it" being one command rather than the machine does not change what the page has to explain. Guide-level prose first and the full export surface below it, because a transport's API and the protocol it speaks are one subject and two pages of it drift), `3.reference/` (what is left once each entry point is documented beside what it does: the root `mountx` export, split by subject rather than carried as one long page — the driver interface, capabilities, errors, the loopback harness, paths and locking — plus the index that maps every subpath to its page and lists those five — the `mountx/drivers/*` subpaths live in the guide beside the interface they implement, `mountx/auto`, `mountx/fuse`, `mountx/9p`, `mountx/nfs`, `mountx/s3` and `mountx/exec` in `2.transports/`, and the CLI in the guide, since it is a demo and a test bench rather than an entry point). `.config/docs.yaml` is the landing page and the site config; `.docs/public/` holds the icons. **This is where user-facing prose goes now.** `README.md` was cut back to the intro, its snippet, install and a link list when the site landed — a second full copy is one that goes stale, and the CLI mounts the README, so it stays short on purpose. Anything longer than a paragraph belongs on a page here. ## Invariants (do not break) @@ -142,7 +153,7 @@ Docs (`docs/`) — the [undocs](https://undocs.dev) site at over TCP"] s3["mountx/s3
S3 REST + SigV4"] ks3["any S3 client
over HTTP"] + ex["mountx/exec
userns + FUSE, or seccomp + 9P"] + kex["one command
and its children"] driver --> auto driver --> s3 + driver --> ex auto --> fuse auto --> p9 auto --> nfs @@ -30,6 +33,7 @@ graph TB p9 --> kv9fs nfs --> knfs s3 --> ks3 + ex --> kex ``` ## What it is choosing between @@ -103,9 +107,13 @@ It is optional, lazy, and never on the root path: mounting as root opens `/dev/f It also ships as compressed base64 inside a JavaScript module rather than as a `.node` file — a binary is loaded by path, and a path is the one thing a bundle does not have. The loader extracts it to a private temporary directory, `dlopen`s it, and deletes it again. So it bundles: nothing to configure, nothing to mark external, no sibling file to copy into your output. -## The fourth transport is not a mount +## The two that are not mounts -[`mountx/s3`](/transports/s3) serves the same `FsDriver` to an S3 client instead — `rclone`, the AWS CLI, an SDK, a presigned URL — over plain HTTP. Nothing about it produces a mountpoint, so it sits outside everything above: `probeTransports()` never mentions it, `mountx/auto` never picks it, and pinning it is the only way to reach it. Reach for it when what is in front of the driver is an S3 client rather than `ls` and `cat`. +Neither produces a mountpoint, so both sit outside everything above: `probeTransports()` mentions neither, `mountx/auto` picks neither, and pinning is the only way to reach either. + +[`mountx/s3`](/transports/s3) serves the same `FsDriver` to an S3 client — `rclone`, the AWS CLI, an SDK, a presigned URL — over plain HTTP. Reach for it when what is in front of the driver is an S3 client rather than `ls` and `cat`. + +[`mountx/exec`](/transports/exec) runs one command with the driver grafted onto its filesystem view, at `$MOUNTX_ROOT`, visible to that process tree and to nothing else on the machine. It picks between a FUSE mount inside an unprivileged user namespace and a seccomp user-notification supervisor that needs no device node at all — its own probe, its own preference order, the same no-fallback rule. Reach for it when the consumer is a program you are about to run rather than the machine. ## Next @@ -114,3 +122,4 @@ It also ships as compressed base64 inside a JavaScript module rather than as a ` - [9P2000.L](/transports/9p) — stateful and root-only, for a Linux host or a VM guest. - [NFS](/transports/nfs) — both versions, including serving without mounting at all. - [S3](/transports/s3) — the gateway transport, and why it stays out of `auto`. +- [Exec](/transports/exec) — a filesystem for one command, with no mount the host can see. diff --git a/docs/2.transports/6.exec.md b/docs/2.transports/6.exec.md new file mode 100644 index 0000000..2c124ba --- /dev/null +++ b/docs/2.transports/6.exec.md @@ -0,0 +1,233 @@ +--- +icon: lucide:square-terminal +title: Exec +--- + +# Exec + +**Give one command a filesystem, instead of giving the machine a folder.** + +`mountx/exec` runs a command with an `FsDriver` grafted onto its filesystem view. The command and everything it spawns see the driver at `$MOUNTX_ROOT`; nothing else on the host does, and nothing is left behind when it exits. It is the `proot`-shaped question — "give this subprocess a filesystem" — and on Linux it has two answers, neither of which is a mount anybody outside the process tree can see. + +```ts +import { exec } from "mountx/exec"; +import { createMemoryDriver } from "mountx/drivers/memory"; + +const ran = await exec(createMemoryDriver(), ["sh", "-c", 'ls -la "$MOUNTX_ROOT"']); +ran.mechanism; // "userns" | "seccomp" +ran.code; // the command's exit status +``` + +Reach for it when the consumer is _a program you are about to run_ rather than a person at a shell: a build step that should read generated files that were never written to disk, a test fixture, an agent's sandbox. Reach for [`mountx/auto`](/transports/auto) when the consumer is the machine. + +## Not a mount transport + +`mountx/auto`'s whole contract is "hand back a mounted directory". This hands back a child process's exit status and has no mountpoint to give anyone, so it sits outside: `probeTransports()` never mentions it, `mount()` never picks it, and importing `mountx/auto` loads none of it. Same line [S3](/transports/s3) sits on, arrived at from the other side. + +Linux only, both mechanisms. A user namespace is a Linux object and seccomp user notification is a Linux facility; macOS has neither, and `DYLD_INSERT_LIBRARIES` — the one thing it does have — is blocked by SIP for exactly the system binaries anyone would want to run. macOS stays [NFS](/transports/nfs) territory. + +**Neither needs root.** An unprivileged user namespace is unprivileged by construction, and an unprivileged seccomp filter needs only `no_new_privs`, which the supervisor sets on itself. + +## What it is choosing between + +| | `userns` | `seccomp` | +| --------------------- | ----------------------------- | -------------------------------------- | +| what the child sees | FUSE, behind the kernel's VFS | eight trapped syscalls | +| needs `/dev/fuse` | **yes** | no | +| needs a kernel module | `fuse` | nothing loadable — seccomp is built in | +| what the child links | anything, including nothing | anything, including nothing | +| writes | yes | **no** — read-only as it stands | +| architectures | any | x86-64 | +| extra build step | none | a supervisor binary, built with Zig | + +**`userns` is preferred wherever the kernel's FUSE is usable.** What the child sees genuinely _is_ FUSE, with the kernel's own VFS in front of it, so it inherits the whole conformance column [the FUSE transport](/transports/fuse) already passes — every syscall, full read/write, every errno. It is also blind to what the child is linked against, because nothing about it depends on that. + +What it gives up is that it _is_ a real kernel mount. It is simply a mount nobody outside the namespace can see: `/proc/self/mounts` on the host stays empty, and the mount dies with the namespace. For an `exec()`-shaped API that is the point rather than the cost. + +**`seccomp` covers the case that motivated the question.** The environments where "no kernel mount" is _wanted_ — a locked-down container, a CI runner, an unprivileged sandbox — are exactly the ones that withhold `/dev/fuse`, and there `userns` cannot be made to work from the inside by any means: a user-namespace root cannot even create the device node, since `mknod /dev/fuse c 10 229` in there answers `EPERM`. A seccomp filter needs no device node, no filesystem driver and no shared library of any kind. Because the boundary is the syscall ABI, a static musl binary and a no-libc raw-syscall binary are served identically to a `cat`. + +It is the newer and narrower of the two — read-only, x86-64 only, one request in flight, and its supervisor is a separately built binary that the npm package does not ship — which is why it is second rather than first. + +## How it decides + +```ts +import { probeExec } from "mountx/exec"; + +const probe = probeExec(); +probe.chosen; // "userns" | "seccomp" | undefined +probe.userns.reason; // why not, when it cannot +``` + +The probe reads `/dev/fuse`, `/proc` and `$PATH` and nothing else — it loads neither a FUSE session nor a 9P codec, so asking costs nothing. When a mechanism is unusable it says which piece is missing, in terms someone can act on: + +| missing | what it says | +| -------------------------------------------- | --------------------------------------------------------------------------------------- | +| no `/dev/fuse` | the device has to come from outside (`--device /dev/fuse`); a namespace cannot make one | +| `/dev/fuse` exists but will not open | the namespace maps only your uid, so it will not rescue an `EACCES` out here | +| `user.max_user_namespaces` is 0 | the sysctl to raise | +| `kernel.unprivileged_userns_clone` is 0 | Debian's knob, and the sysctl to set | +| `apparmor_restrict_unprivileged_userns` is 1 | Ubuntu 23.10+'s default, and the three ways past it | +| no `unshare` on `$PATH` | util-linux or busybox both provide one | +| not x86-64 | a second syscall table for the seccomp filter, which is not written yet | +| no supervisor binary | build it from `src/exec/seccomp/` and point `$MOUNTX_TRACE` at it | + +::note +The `/dev/fuse` check is an `open(2)`, not a `stat`. A device that exists but refuses to open is a different sentence from one that is not there, and only opening it tells the two apart — which matters because entering the namespace does not change the answer: `unshare -r` maps exactly one uid, so the permission check the relay makes inside is the one that already failed outside. +:: + +### Three things it deliberately does not do + +Lifted from [`mountx/auto`](/transports/auto#three-things-auto-deliberately-does-not-do), because the arguments are the same ones: + +- **No fallback after a failure.** The probe decides once, from host facts; a mechanism that then fails reports its own error. Quietly re-running the command under the _other_ mechanism would be worse here than it is for a mount — the two do not have the same semantics, and a command that has already run once may have had effects outside the driver. +- **No probing when you name a mechanism.** `mechanism: "seccomp"` calls the supervisor, whose own errors are more specific than anything the picker could say. +- **No wrapping.** The result is the mechanism's own result object with a `mechanism` discriminant defined on it, so narrowing on it reaches everything that mechanism reports. + +## `$MOUNTX_ROOT`, and the one rule about `cwd` + +Both mechanisms set `MOUNTX_ROOT` in the command's environment, and reading it is the portable spelling: `userns` defaults to a private temporary directory, `seccomp` to `/mountx`, and [`root`](#execoptions) overrides either. + +**Do not point `cwd` inside it.** Under `userns` that is a deadlock rather than a slow path, and `execUserns()` refuses it up front instead of hanging: + +```ts +// ✗ deadlocks — refused +await exec(driver, ["ls"], { root: "/mnt/x", cwd: "/mnt/x" }); + +// ✓ the cd happens after the exec +await exec(driver, ["sh", "-c", 'cd "$MOUNTX_ROOT" && ls']); +``` + +`uv_spawn` blocks the thread answering FUSE requests until the child execs, and a child whose first act is a `chdir` into the mount waits for a reply only that thread can send. It is the same hazard [`mount9p()`](/transports/9p) and the FUSE transport both document, met from the inside — and the same rule covers a command binary that _lives_ on the driver. + +## `exec(driver, argv, options?)` + +```ts +function exec( + driver: FsDriver, + argv: readonly string[], + options?: ExecOptions, +): Promise; +``` + +Resolves when the command exits, with that command's status. **A command that fails is not an error here**, exactly as it is not for `node:child_process` — which is what makes a thrown error mean "the mechanism could not be set up" and nothing else. `stdio` is inherited throughout, so the command's output is this process's output. + +### `ExecOptions` + +| option | default | | +| -------------- | --------- | ------------------------------------------------------------------------ | +| `mechanism` | `"auto"` | `"userns"` \| `"seccomp"` \| `"auto"`; naming one skips the probe | +| `root` | see above | where the driver appears, and the value of `$MOUNTX_ROOT` | +| `cwd` | this one | working directory for the command — never inside `root` | +| `env` | this one | environment for the command; `MOUNTX_ROOT` is added to it | +| `useDriverIno` | `false` | report the driver's own `ino` values instead of synthesising them | +| `debug` | `false` | log protocol traffic to stderr | +| `userns` | none | `ExecUsernsOptions`, applied after the shared ones and therefore winning | +| `seccomp` | none | `ExecSeccompOptions`, likewise | + +The shared options are the ones that mean the same thing in both mechanisms. Everything else goes in the two escape hatches, for the reason `AutoMountOptions` has three: same-named options with genuinely different shapes (`onError` hands the FUSE side a request and the 9P side a message header) merge into something that either lies or is unusable. + +### `ExecResult` + +```ts +type ExecResult = + | (ExecUsernsResult & { readonly mechanism: "userns" }) + | (ExecSeccompResult & { readonly mechanism: "seccomp" }); +``` + +A discriminated union over the mechanism's own result. Both carry `code` (`number | null`) and `signal` (`NodeJS.Signals | null`); `userns` adds `mountpoint`, `seccomp` adds `root` and `requests`. + +### `probeExec(platform?, arch?, supervisor?)` + +```ts +interface ExecProbe { + platform: NodeJS.Platform; + chosen: ExecMechanism | undefined; + preference: readonly ExecMechanism[]; // always ["userns", "seccomp"] + userns: UsernsExecProbe; + seccomp: SeccompExecProbe; + reason: string | undefined; // names both, when neither can run +} +``` + +Synchronous, unlike [`probeTransports()`](/transports/auto#probetransportsplatform) — every fact here is a file read, where FUSE's rootless probe has to load the native addon before it can answer. The three parameters exist to be overridden in tests; leave them alone otherwise. + +There is one preference order on every host, because off Linux neither mechanism can work and there is nothing for a second order to say. + +### `usernsExecProbe(platform?)` / `seccompExecProbe(platform?, arch?, supervisor?)` + +```ts +interface UsernsExecProbe { + usable: boolean; + platform: "linux" | undefined; + kernel: boolean; // `fuse` in /proc/filesystems + device: boolean; // /dev/fuse exists *and* opens + userns: boolean; // this process may create one + unshare: string | undefined; // the resolved binary + reason: string | undefined; +} + +interface SeccompExecProbe { + usable: boolean; + platform: "linux" | undefined; + arch: boolean; // an architecture the filter covers + userNotif: boolean; // SECCOMP_RET_USER_NOTIF in actions_avail + supervisor: string | undefined; // the resolved binary + reason: string | undefined; +} +``` + +The per-mechanism probes `probeExec()` composes, exported because a caller — or a test suite gating itself — often wants exactly one of them. Cheap enough to call unconditionally. + +## `execUserns(driver, argv, options?)` + +The user-namespace mechanism on its own, without the probe: + +```ts +import { execUserns } from "mountx/exec"; + +const ran = await execUserns(driver, ["sh", "-c", 'cat "$MOUNTX_ROOT/hello.txt"'], { + mountpoint: "/mnt/x", // created if missing +}); +ran.mountpoint; +``` + +`ExecUsernsOptions` extends `FuseSessionOptions`, so every caching and attribute knob the [FUSE transport](/transports/fuse#fusesessionoptions) has applies here too, plus `mountpoint`, `cwd`, `env` and `mountOptions`. + +**`default_permissions` is not among the default `-o` options,** which is worth knowing before setting it. It asks the _kernel_ to check the driver's `uid`/`gid`/`mode` against the caller's credentials, and inside a namespace those are not the same identity space: `unshare -r` maps exactly one uid, so a driver reporting the serving process's real uid is reporting an identity the namespace renders as `nobody` — and a `nobody`-owned `0755` root directory refuses every write from the one process meant to have it. Permission checking stays with the driver instead. Nothing is lost by it: the mount carries no `allow_other`, so the only process that can reach it is the one the call created it for. + +### How it is built + +`unshare(CLONE_NEWUSER)` demands a single-threaded caller and Node is never single-threaded, so a mountx process cannot enter the namespace it needs — not with `unshare(2)`, and not with `setns(2)`, which has the same rule. The namespace is entered by a **child**, `/dev/fuse` is opened in there, and raw FUSE traffic comes back out over a unix socket to a `FuseSession` in the parent, which is where your driver stays. Inside the namespace that child is uid 0 with `CAP_SYS_ADMIN`, so it takes the ordinary root mount path: no `fusermount3`, no setuid bit, and **no native addon**. On a host with no `fuse3` package installed at all, this is the only FUSE route that works. + +::note +A relay failure — `mount(8)` not coming up, a command that is not on `$PATH` — reaches you as a thrown error rather than as an exit status. It has to: the relay's own exit code is indistinguishable from the command's, and "the command exited 70" is a thing that happens. +:: + +## `execSeccomp(driver, argv, options?)` + +The seccomp supervisor on its own. It needs a built binary — pass `trace`, or set `$MOUNTX_TRACE`: + +```sh +zig build-exe -lc -O ReleaseSmall -femit-bin=mountx-trace \ + --dep p9 -Mroot=src/exec/seccomp/trace.zig -Mp9=src/exec/seccomp/p9.zig +``` + +A BPF filter traps eight syscalls and everything else runs natively without leaving the kernel. The supervisor installs the filter on **itself** and forks, rather than forking and passing a listener descriptor back over `SCM_RIGHTS` — a seccomp filter is inherited across `fork` and `exec`, so no descriptor is passed anywhere and none of the `recvmsg` machinery unprivileged FUSE mounting needs is involved. + +The parent side is a `createP9Server()` on a private unix socket, unchanged: **an interceptor does not need a filesystem, it needs a client.** Path resolution, handle lifetimes, directory paging and error mapping all stay in the [9P session](/transports/9p), so the supervisor is a wire adapter with an fd table and no filesystem logic at all. + +Read the [9P transport page](/transports/9p) for what that server does; read `src/exec/seccomp.ts` for what the supervisor currently covers, which is less than the 9P server does. + +## Not available + +- **Writes through `seccomp`.** Read-only as it stands. `userns` has no such limit. +- **`seccomp` on anything but x86-64.** The filter compares against one syscall table; arm64 is a second table rather than a redesign. +- **A shipped supervisor binary.** The npm package carries no build of it, unlike the ~7 KB FUSE helper addon, which [is embedded](/transports#no-native-code-with-one-exception). +- **macOS and Windows.** Neither has user namespaces or seccomp. +- **`LD_PRELOAD`.** Built, measured and rejected — it cannot see a Go or static binary at all, its symbol surface tracks other projects' releases, a descriptor it creates does not survive `exec`, and its characteristic failure is a confident wrong answer rather than an error. The code stays in the tree as the evidence and is reachable from nothing that ships. + +## Next + +- [FUSE](/transports/fuse) — what the `userns` child is actually talking to. +- [9P2000.L](/transports/9p) — what the `seccomp` supervisor is actually talking to. +- [`mountx/auto`](/transports/auto) — when the consumer is the machine rather than one command. diff --git a/docs/3.reference/0.index.md b/docs/3.reference/0.index.md index 48b86aa..d1d0f9e 100644 --- a/docs/3.reference/0.index.md +++ b/docs/3.reference/0.index.md @@ -5,7 +5,7 @@ title: Entry Points # Reference -**Nine subpaths. Each loads only what it needs.** +**Ten subpaths. Each loads only what it needs.** | import from | what you get | | --------------------------------------------------------------- | -------------------------------------------------------------------- | @@ -18,8 +18,9 @@ title: Entry Points | [`mountx/9p`](/transports/9p) | the 9P2000.L transport, and `createP9Server()` | | [`mountx/nfs`](/transports/nfs) | the NFS transport (v3 default, v4.1 opt-in), and `createNfsServer()` | | [`mountx/s3`](/transports/s3) | the S3 gateway, and `createS3Server()` — not a mount transport | +| [`mountx/exec`](/transports/exec) | `exec()` — a filesystem for one command, not a mount transport | -The three `mountx/drivers/*` subpaths are documented in the guide, beside the interface they implement: [built-in drivers](/guide/drivers/built-in). `mountx/auto` and the four transport subpaths are documented in [Transports](/transports), beside the protocols they speak. +The three `mountx/drivers/*` subpaths are documented in the guide, beside the interface they implement: [built-in drivers](/guide/drivers/built-in). `mountx/auto` and the five transport subpaths are documented in [Transports](/transports), beside the protocols they speak. Plus the [`mountx` CLI](/guide/cli), which the package installs as a binary — a demo and a test bench, documented in the guide. @@ -70,5 +71,6 @@ Two rules the type system holds, and both are checked in CI: | `mount9p` | ✅ root | — | — | | `mountNfs` | ✅ root | ✅ no root | — | | `mount` (auto) | ✅ FUSE, then 9P | ✅ NFS | — | +| `exec` | ✅ no root | — | — | `createP9Server` runs anywhere Node does — only `mount9p()` is Linux- and root-only, the same split `mountx/nfs` has between `createNfsServer` and `mountNfs`. Everything in `mountx/fuse` except `mount.ts`, everything in `mountx/9p` except `server.ts`/`mount.ts`, and everything in `mountx/nfs` except `server.ts`/`mount.ts`, is pure data transformation and runs anywhere. diff --git a/package.json b/package.json index 33b49df..5d5c776 100644 --- a/package.json +++ b/package.json @@ -44,6 +44,10 @@ "types": "./dist/s3/index.d.mts", "default": "./dist/s3/index.mjs" }, + "./exec": { + "types": "./dist/exec/index.d.mts", + "default": "./dist/exec/index.mjs" + }, "./drivers/memory": { "types": "./dist/drivers/memory.d.mts", "default": "./dist/drivers/memory.mjs" @@ -77,7 +81,7 @@ "test:nfs:mount": "sh test/root.sh test/nfs/mount.test.ts", "test:9p:mount": "sh test/root.sh test/9p/mount.test.ts", "test:pjdfstest": "sh test/pjdfstest/run.sh", - "test:rootless": "sh test/rootless.sh test/fuse/mount-rootless.test.ts test/auto-mount.test.ts test/nfs/mount.test.ts", + "test:rootless": "sh test/rootless.sh test/fuse/mount-rootless.test.ts test/auto-mount.test.ts test/nfs/mount.test.ts test/exec/userns.test.ts", "test:root": "sh test/root.sh test/fuse/mount.test.ts test/fuse/differential.test.ts test/fuse/conformance-mount.test.ts test/nfs/mount.test.ts test/9p/mount.test.ts", "typecheck": "tsc --noEmit --skipLibCheck" }, diff --git a/src/exec/demo-driver.ts b/src/exec/demo-driver.ts new file mode 100644 index 0000000..8fb3d6a --- /dev/null +++ b/src/exec/demo-driver.ts @@ -0,0 +1,27 @@ +/** The tree both mechanisms are pointed at, so their results compare. */ + +import { createMemoryDriver } from "../drivers/memory.ts"; +import { createLoopback } from "../harness.ts"; +import type { FsDriver } from "../types.ts"; + +/** A memory driver holding a handful of files a shell can obviously exercise. */ +export async function createDemoDriver(): Promise { + const driver = createMemoryDriver(); + const fs = createLoopback(driver); + await fs.mkdir("/docs", { recursive: true }); + await fs.writeFile("/hello.txt", "hello from a driver that is not on any disk\n"); + await fs.writeFile("/docs/a.txt", "alpha\n"); + await fs.writeFile("/docs/b.txt", "bravo\n"); + await fs.writeFile( + "/numbers.txt", + `${Array.from({ length: 100 }, (_, i) => i + 1).join("\n")}\n`, + ); + // 3 MiB of deterministic bytes, for a byte-exactness check that is bigger + // than any single message on any of the three transports. + const big = Buffer.allocUnsafe(3 * 1024 * 1024); + for (let i = 0; i < big.length; i++) { + big[i] = (i * 31 + 7) & 0xff; + } + await fs.writeFile("/big.bin", big); + return driver; +} diff --git a/src/exec/demo-seccomp.ts b/src/exec/demo-seccomp.ts new file mode 100644 index 0000000..6273d61 --- /dev/null +++ b/src/exec/demo-seccomp.ts @@ -0,0 +1,29 @@ +/** + * The seccomp mechanism, run against the shared demo tree. + * + * ```sh + * MOUNTX_TRACE=/path/to/mountx-trace node src/exec/demo-seccomp.ts [command...] + * ``` + * + * A test bench rather than an entry point — `mountx/exec` is the entry point, + * and this is what `test/exec/compare.sh` drives to fill one column of the + * comparison in `.agents/proot-plan.md`. It calls `execSeccomp()` by name on + * purpose: the value of that comparison is that each column is one *named* + * mechanism rather than whatever the picker would have chosen. + */ + +import { createDemoDriver } from "./demo-driver.ts"; +import { execSeccomp } from "./seccomp.ts"; + +const root = process.env.MOUNTX_TEST_ROOT ?? "/mountx"; +const command = + process.argv.length > 2 + ? process.argv.slice(2) + : ["sh", "-c", `ls -la ${root} && cat ${root}/hello.txt`]; + +const driver = await createDemoDriver(); +const result = await execSeccomp(driver, command, { root }); +process.stderr.write( + `\n[seccomp] root=${result.root} code=${result.code} signal=${result.signal} 9p-requests=${result.requests}\n`, +); +process.exitCode = result.code ?? 1; diff --git a/src/exec/demo-userns.ts b/src/exec/demo-userns.ts new file mode 100644 index 0000000..b53e6aa --- /dev/null +++ b/src/exec/demo-userns.ts @@ -0,0 +1,34 @@ +/** + * The user-namespace mechanism, run against the shared demo tree. + * + * ```sh + * node src/exec/demo-userns.ts [command...] + * ``` + * + * A test bench rather than an entry point — `mountx/exec` is the entry point, + * and this is what `test/exec/compare.sh` drives to fill one column of the + * comparison in `.agents/proot-plan.md`. It calls `execUserns()` by name on + * purpose: the value of that comparison is that each column is one *named* + * mechanism rather than whatever the picker would have chosen. + */ + +import { createDemoDriver } from "./demo-driver.ts"; +import { execUserns } from "./userns.ts"; + +// `$MOUNTX_ROOT` rather than a hardcoded path: the relay sets it, and a `cd` +// that happens after the exec is the only kind that does not deadlock. +const command = + process.argv.length > 2 + ? process.argv.slice(2) + : [ + "sh", + "-c", + 'cd "$MOUNTX_ROOT" && ls -la && cat hello.txt && wc -c big.bin && sha256sum big.bin', + ]; + +const driver = await createDemoDriver(); +const result = await execUserns(driver, command, { debug: process.env.MOUNTX_DEBUG === "1" }); +process.stderr.write( + `\n[userns] mountpoint=${result.mountpoint} code=${result.code} signal=${result.signal}\n`, +); +process.exitCode = result.code ?? 1; diff --git a/src/exec/index.ts b/src/exec/index.ts new file mode 100644 index 0000000..192ed12 --- /dev/null +++ b/src/exec/index.ts @@ -0,0 +1,278 @@ +/** + * `mountx/exec` — run a command with a driver grafted onto its filesystem view. + * + * ```ts + * import { exec } from "mountx/exec"; + * const ran = await exec(createMemoryDriver(), ["sh", "-c", "ls -la $MOUNTX_ROOT"]); + * ran.mechanism; // "userns" | "seccomp" + * ran.code; // the command's exit status + * ``` + * + * Where `mountx/auto` attaches a driver to a *directory on the machine*, this + * attaches one to a *process tree*: the command and everything it spawns see + * the driver at `$MOUNTX_ROOT`, and nothing else on the host does. That is the + * `proot`-shaped question — "give this subprocess a filesystem" — and it has + * two answers on Linux, neither of which is a mount anybody else can see. + * + * | mechanism | what the child sees | what it needs | + * | ---------- | ---------------------------- | --------------------------------------- | + * | `userns` | FUSE, behind the kernel VFS | `/dev/fuse`, user namespaces, `unshare` | + * | `seccomp` | trapped syscalls, no mount | `SECCOMP_RET_USER_NOTIF`, a supervisor | + * + * **The strategy is `userns` where the kernel's FUSE is usable, `seccomp` + * otherwise.** `userns` is the cheapest correct thing by a wide margin: what + * the child sees genuinely *is* FUSE, with the kernel's own VFS in front of it, + * so it inherits the whole conformance column `src/fuse/` already passes, and + * it is blind to what the child is linked against. It is simply a mount nobody + * outside the namespace can see — `/proc/self/mounts` on the host stays empty + * and the mount dies with the namespace. + * + * `seccomp` covers the case that motivated the question in the first place. The + * environments where "no kernel mount" is *wanted* — a locked-down container, a + * CI runner, an unprivileged sandbox — are exactly the ones that withhold + * `/dev/fuse`, and there `userns` cannot be made to work from the inside by any + * means: a user-namespace root cannot even create the device node + * (`mknod /dev/fuse c 10 229` answers `EPERM`, verified on `alpine:latest`). + * A seccomp filter needs no device node, no filesystem driver and no shared + * library of any kind. It is also the newer and narrower of the two — read-only + * as it stands, x86-64 only, and its supervisor is a separately built binary — + * which is why it is second and not first. + * + * **Deliberately outside `mountx/auto`.** `auto`'s whole contract is "hand back + * a mounted directory"; this hands back a child process's exit status and has + * no mountpoint to give anyone. Same reasoning that keeps `mountx/s3` out of + * it, arrived at from the other side: `probeTransports()` never mentions this, + * `mount()` never picks it, and importing `mountx/auto` loads none of it. + * + * **Three things it deliberately does not do**, all three lifted from + * `src/auto.ts` because the arguments are the same ones: + * + * - **No fallback after a failure.** {@link probeExec} decides once, from host + * facts; a mechanism that then fails reports its own error. Quietly re-running + * the command under the *other* mechanism would be worse here than it is for a + * mount — the two do not have the same semantics (one is a real filesystem, + * the other is eight trapped syscalls and read-only), and a command that has + * already run once may have had effects outside the driver. + * - **No probing when you name a mechanism.** `mechanism: "seccomp"` calls the + * supervisor, whose own errors are more specific than anything this file + * could say about it. + * - **No loading of what it does not use.** Each mechanism arrives through + * `await import()`, so choosing `userns` never loads the 9P codec behind the + * seccomp supervisor and choosing `seccomp` never loads the FUSE session. + * The probe reaches only `src/exec/probe.ts`, which imports `node:fs` and + * nothing else. + * + * The result is the mechanism's own result object with a `mechanism` + * discriminant defined on it — tagged, not wrapped, the way `mountx/auto` tags + * a mount — so narrowing on it reaches everything that mechanism reports. + */ + +import type { FsDriver } from "../types.ts"; +import { + type SeccompExecProbe, + seccompExecProbe, + type UsernsExecProbe, + usernsExecProbe, +} from "./probe.ts"; +import type { ExecSeccompOptions, ExecSeccompResult } from "./seccomp.ts"; +import type { ExecUsernsOptions, ExecUsernsResult } from "./userns.ts"; + +export type { ExecPlatform, SeccompExecProbe, UsernsExecProbe } from "./probe.ts"; +export { execPlatform, seccompExecProbe, usernsExecProbe } from "./probe.ts"; +export type { ExecSeccompOptions, ExecSeccompResult } from "./seccomp.ts"; +export type { ExecUsernsOptions, ExecUsernsResult } from "./userns.ts"; + +/** The mechanisms {@link exec} can choose between. */ +export type ExecMechanism = "userns" | "seccomp"; + +/** What this host can run a command with, and what {@link exec} would pick. */ +export interface ExecProbe { + /** The platform the probe was answered for. */ + platform: NodeJS.Platform; + /** What {@link exec} would use, or `undefined` if neither works here. */ + chosen: ExecMechanism | undefined; + /** Preference order — the list `chosen` was picked from. */ + preference: readonly ExecMechanism[]; + userns: UsernsExecProbe; + seccomp: SeccompExecProbe; + /** Why nothing can run, naming both mechanisms. `undefined` when {@link chosen}. */ + reason: string | undefined; +} + +/** + * Options common to both mechanisms, plus an escape hatch for each. + * + * Deliberately *not* the union of the two option types, for the reason + * `AutoMountOptions` is not the union of three: they have same-named options + * with genuinely different shapes (`onError` hands the FUSE side a request and + * the 9P side a message header), and a merged type would either lie about that + * or collapse to something unusable. So the shared options are the ones that + * mean the same thing in both, and anything mechanism-specific goes in + * {@link ExecOptions.userns} or {@link ExecOptions.seccomp} — which are applied + * *after* the shared ones and therefore win. + */ +export interface ExecOptions { + /** Which mechanism to use. Default `"auto"` — see {@link probeExec}. */ + mechanism?: ExecMechanism | "auto"; + /** + * Where the driver appears to the command, and the value of `$MOUNTX_ROOT`. + * + * The two mechanisms mean subtly different things by it and both honour it: + * for `userns` it is a **real directory** that gets a namespace-private FUSE + * mount on it (created if missing, and defaulting to a private temporary + * directory), for `seccomp` it is a **path prefix the supervisor claims** and + * nothing is mounted there at all (default `/mountx`). Portable code should + * read `$MOUNTX_ROOT` from inside the command rather than assume either. + */ + root?: string; + /** Working directory for the command. Never default to inside `root` — see below. */ + cwd?: string; + /** Environment for the command. Defaults to this process's. */ + env?: NodeJS.ProcessEnv; + /** Report the driver's own `ino` values instead of synthesising them. */ + useDriverIno?: boolean; + /** Log protocol traffic to stderr. */ + debug?: boolean; + /** Options for the user-namespace mechanism only. Applied after the shared ones. */ + userns?: ExecUsernsOptions; + /** Options for the seccomp mechanism only. Applied after the shared ones. */ + seccomp?: ExecSeccompOptions; +} + +/** + * How a command ended, tagged with the mechanism that ran it. + * + * The tag is a discriminant, so narrowing on it gives the mechanism's full + * result — `ran.mechanism === "userns"` reaches `mountpoint`, `"seccomp"` + * reaches `root` and `requests`. What both share is `code` and `signal`. + */ +export type ExecResult = + | (ExecUsernsResult & { readonly mechanism: "userns" }) + | (ExecSeccompResult & { readonly mechanism: "seccomp" }); + +/** + * What this host can run a command with, and what {@link exec} would choose. + * + * Cheap enough to call before deciding whether to offer this at all, and + * specific enough to print: when neither mechanism works, `reason` names what + * each one is missing rather than reporting the last failure. + * + * Synchronous, unlike `probeTransports()` — every fact here is a `node:fs` read, + * where FUSE's rootless probe has to load the native addon before it can answer. + * + * `platform`, `arch` and `supervisor` exist to be overridden in tests; leave + * them alone otherwise. + */ +export function probeExec( + platform: NodeJS.Platform = process.platform, + arch: string = process.arch, + supervisor: string | undefined = process.env.MOUNTX_TRACE, +): ExecProbe { + const userns = usernsExecProbe(platform); + const seccomp = seccompExecProbe(platform, arch, supervisor); + // One order on every host, because off Linux neither can work and there is + // nothing for a second order to say. `userns` leads because it is a real + // filesystem rather than eight trapped syscalls: full read/write, every + // syscall, and the conformance column `src/fuse/` already passes. `seccomp` + // follows because what it buys — no device node, no kernel module, no shared + // library — only matters on a host where `userns` cannot run at all. + const preference: readonly ExecMechanism[] = ["userns", "seccomp"]; + const probes = { userns, seccomp }; + const chosen = preference.find((mechanism) => probes[mechanism].usable); + return { + platform, + chosen, + preference, + userns, + seccomp, + reason: + chosen === undefined + ? `no mechanism can run a command with a driver on this host — user namespace: ` + + `${userns.reason}; seccomp: ${seccomp.reason}` + : undefined, + }; +} + +/** The shared options, in the shape each mechanism wants them. */ +function shared(options: ExecOptions): Pick { + return { + cwd: options.cwd, + env: options.env, + useDriverIno: options.useDriverIno, + debug: options.debug, + }; +} + +/** + * Stamp the mechanism onto the result the mechanism just returned. + * + * The result object itself is tagged rather than wrapped, so every + * mechanism-specific member keeps working on the thing the caller holds. Same + * shape as `src/auto.ts`'s `tag()`, and idempotent for the same reason. + */ +function tag( + result: R, + mechanism: M, +): R & { readonly mechanism: M } { + if (!("mechanism" in result)) { + Object.defineProperty(result, "mechanism", { value: mechanism, enumerable: true }); + } + return result as R & { readonly mechanism: M }; +} + +/** + * Run `argv` with `driver` grafted onto its filesystem view. + * + * Resolves when the command exits, with that command's status — a command that + * fails is not an error here, exactly as it is not for `child_process`. An + * error is thrown only when the *mechanism* could not be set up, and it names + * the missing piece. + * + * The command finds the driver at `$MOUNTX_ROOT`, which both mechanisms set in + * its environment. Read it rather than hardcoding {@link ExecOptions.root}: it + * is the one spelling that is right under either mechanism and with the default + * (a private temporary directory) in play. + * + * **Do not point `cwd` inside the root.** Under `userns` that is a deadlock, + * not a slow path: `uv_spawn` blocks the thread that answers FUSE requests + * until the child execs, and a child whose first act is a `chdir` into the + * mount waits for a reply only that thread can send. `sh -c 'cd "$MOUNTX_ROOT" + * && …'` does the same thing safely, because the `cd` happens after the exec. + * The same rule covers a command binary that *lives* on the driver. + */ +export async function exec( + driver: FsDriver, + argv: readonly string[], + options: ExecOptions = {}, +): Promise { + if (argv.length === 0) { + throw new Error("mountx: exec needs a command to run"); + } + const requested = options.mechanism ?? "auto"; + let mechanism: ExecMechanism; + if (requested === "auto") { + const probe = probeExec(process.platform, process.arch, options.seccomp?.trace); + if (probe.chosen === undefined) { + throw new Error(`mountx: ${probe.reason}`); + } + mechanism = probe.chosen; + } else { + mechanism = requested; + } + if (mechanism === "userns") { + const { execUserns } = await import("./userns.ts"); + return tag( + await execUserns(driver, argv, { + ...shared(options), + mountpoint: options.root, + ...options.userns, + }), + "userns", + ); + } + const { execSeccomp } = await import("./seccomp.ts"); + return tag( + await execSeccomp(driver, argv, { ...shared(options), root: options.root, ...options.seccomp }), + "seccomp", + ); +} diff --git a/src/exec/probe.ts b/src/exec/probe.ts new file mode 100644 index 0000000..b115e54 --- /dev/null +++ b/src/exec/probe.ts @@ -0,0 +1,359 @@ +/** + * Can this host graft a driver onto a subprocess's filesystem view, and if not, + * which piece is missing? + * + * Split out of the two mechanisms for the reason `src/nfs/probe.ts` and + * `src/9p/probe.ts` are split out of their transports: asking should cost + * nothing. `userns.ts` reaches the whole FUSE session and `seccomp.ts` reaches + * the whole 9P2000.L codec — a lot of module graph to answer a question that + * reads four files under `/proc` and one directory of `$PATH`. So this file + * imports `node:fs` and nothing else, and `src/exec/index.ts` decides from it + * before loading either mechanism. + * + * **Linux only, both of them.** A user namespace is a Linux object and seccomp + * user notification is a Linux facility; macOS has neither, and `DYLD_INSERT_ + * LIBRARIES` — the one thing it does have — is blocked by SIP for exactly the + * system binaries anyone would want to run. macOS stays NFS-mount territory. + * + * **Neither needs root.** That is the whole point of both: an unprivileged user + * namespace is unprivileged by construction, and an unprivileged seccomp filter + * needs only `no_new_privs`, which the supervisor sets on itself. + */ + +import * as fs from "node:fs"; + +/** + * The verdict for `probe("linux")` asked from somewhere that is not Linux. + * + * Every fact below `platform` is a file this process reads about *itself*, so a + * `platform` override — which exists for the tests — takes them all away at + * once. Answering `usable` on the strength of a check that never ran would be + * the one lie a probe must not tell, so the missing evidence is the refusal. + */ +const NOT_THIS_HOST = + "this probe was asked about Linux from a host that is not Linux, so nothing it reads — " + + "/dev/fuse, /proc, $PATH — describes the machine in question"; + +/** The one host either mechanism runs on. */ +export type ExecPlatform = "linux"; + +/** `process.platform`, narrowed to the host that can run a command, or `undefined`. */ +export function execPlatform(platform: NodeJS.Platform): ExecPlatform | undefined { + return platform === "linux" ? "linux" : undefined; +} + +/** What this host can and cannot do about the user-namespace mechanism. */ +export interface UsernsExecProbe { + /** Can a command be run with a namespace-private FUSE mount here? */ + usable: boolean; + /** `"linux"`, or `undefined` on a host with no user namespaces. */ + platform: ExecPlatform | undefined; + /** Is `fuse` listed in `/proc/filesystems`? */ + kernel: boolean; + /** Does `/dev/fuse` exist *and* open for reading and writing? */ + device: boolean; + /** Can this process create a user namespace? */ + userns: boolean; + /** The `unshare(1)` on `$PATH`, or `undefined`. */ + unshare: string | undefined; + /** Everything that is missing, in a sentence. `undefined` when {@link usable}. */ + reason: string | undefined; +} + +/** What this host can and cannot do about the seccomp mechanism. */ +export interface SeccompExecProbe { + /** Can a command be run under a seccomp user-notification supervisor here? */ + usable: boolean; + /** `"linux"`, or `undefined` on a host with no seccomp. */ + platform: ExecPlatform | undefined; + /** Is this an architecture the supervisor's syscall table covers? */ + arch: boolean; + /** Does the kernel offer `SECCOMP_RET_USER_NOTIF`? */ + userNotif: boolean; + /** The built supervisor binary, or `undefined` when there is not one. */ + supervisor: string | undefined; + /** Everything that is missing, in a sentence. `undefined` when {@link usable}. */ + reason: string | undefined; +} + +/** The contents of `path`, trimmed, or `undefined` if it could not be read. */ +function readQuietly(path: string): string | undefined { + try { + return fs.readFileSync(path, "utf8").trim(); + } catch { + return undefined; + } +} + +/** Is `name` one of the filesystems the kernel knows, per `/proc/filesystems`? */ +function hasFilesystem(name: string): boolean { + const table = readQuietly("/proc/filesystems"); + if (table === undefined) { + return false; + } + // Each line is `nodev\tfuse` or `\text4`: the name is the last + // whitespace-separated field, and matching it whole is what keeps `fuse` from + // being found inside `fuseblk`. + return table.split("\n").some((line) => line.trim().split(/\s+/).pop() === name); +} + +/** + * Why `/dev/fuse` cannot be used from here, or `undefined` if it can. + * + * Opened rather than stat'd, because the two failures a caller can act on are + * different sentences and only an `open(2)` tells them apart. Both refusals say + * the same last thing for the same reason: this is the case + * `src/exec/seccomp.ts` exists for. + */ +function deviceRefusal(): string | undefined { + let device: number; + try { + device = fs.openSync("/dev/fuse", fs.constants.O_RDWR); + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + if (code === "ENOENT" || code === "ENXIO" || code === "ENODEV") { + return ( + "there is no usable /dev/fuse on this host — the fuse module is not loaded, or this " + + "container was never given the device (docker/podman: --device /dev/fuse). A user " + + "namespace cannot conjure one either: `mknod /dev/fuse c 10 229` as namespace-root " + + "answers EPERM (verified on alpine:latest), so the node has to come from outside" + ); + } + if (code === "EACCES" || code === "EPERM") { + return ( + "/dev/fuse exists but this process cannot open it — and entering a user namespace does " + + "not help, because the namespace maps only your own uid: the permission check the relay " + + "makes inside it is the one that just failed out here, and the namespace's " + + "CAP_DAC_OVERRIDE does not reach a device node owned by a uid it does not map" + ); + } + return `/dev/fuse could not be opened (${code ?? "unknown error"})`; + } + fs.closeSync(device); + return undefined; +} + +/** + * Why this process cannot create a user namespace, or `undefined` if it can. + * + * Three sysctls, each of which turns unprivileged namespace creation off in a + * different distribution's idiom, plus the kernel-level "compiled without + * namespaces at all". Root is exempt from the last two, which gate the + * *unprivileged* path only. + */ +function usernsRefusal(root: boolean): string | undefined { + if (!fs.existsSync("/proc/self/ns/user")) { + return ( + "this kernel has no user namespaces at all (no /proc/self/ns/user), so there is no " + + "namespace to mount FUSE inside" + ); + } + // Present on every namespace-capable kernel; zero is an administrator turning + // the whole facility off, root included. + const max = readQuietly("/proc/sys/user/max_user_namespaces"); + if (max === "0") { + return ( + "user namespaces are disabled on this host (/proc/sys/user/max_user_namespaces is 0) — " + + "raise it with `sysctl -w user.max_user_namespaces=`, or use the seccomp mechanism, " + + "which needs no namespace" + ); + } + if (root) { + return undefined; + } + // Debian's long-standing knob, and Ubuntu's newer AppArmor one. Both gate the + // unprivileged path only, which is why they are read after the root check. + if (readQuietly("/proc/sys/kernel/unprivileged_userns_clone") === "0") { + return ( + "unprivileged user namespaces are disabled " + + "(/proc/sys/kernel/unprivileged_userns_clone is 0) — enable them with `sysctl -w " + + "kernel.unprivileged_userns_clone=1`, run as root, or use the seccomp mechanism" + ); + } + if (readQuietly("/proc/sys/kernel/apparmor_restrict_unprivileged_userns") === "1") { + return ( + "AppArmor restricts unprivileged user namespaces on this host " + + "(/proc/sys/kernel/apparmor_restrict_unprivileged_userns is 1, the Ubuntu 23.10+ " + + "default) — an unconfined program gets EPERM from unshare(CLONE_NEWUSER) with no other " + + "clue. Turn it off with `sysctl -w kernel.apparmor_restrict_unprivileged_userns=0`, " + + "install a profile for this program, or use the seccomp mechanism" + ); + } + return undefined; +} + +/** + * The named executable on `$PATH`, or `undefined`. + * + * `$PATH` is split on `:` and joined with `/` rather than through `node:path`, + * which would be the only other import in this file: both mechanisms are Linux + * only, and on Linux those two characters are the whole of the question. + */ +function onPath(name: string): string | undefined { + for (const directory of (process.env.PATH ?? "").split(":")) { + if (directory === "") { + continue; + } + const candidate = `${directory}/${name}`; + try { + fs.accessSync(candidate, fs.constants.X_OK); + return candidate; + } catch { + // Not here, or here and not executable — either way, keep looking. + } + } + return undefined; +} + +/** + * Can this host run a command inside an unprivileged user namespace with the + * driver mounted over FUSE? + * + * Synchronous and cheap, so a test can gate itself on it and `exec()` can refuse + * with the missing piece named rather than with the raw `ENOENT` from + * `open("/dev/fuse")` that a relay three processes away would otherwise report. + * + * **What makes it usable:** Linux, a `/dev/fuse` this process can open, `fuse` + * in `/proc/filesystems`, user namespaces this process may create, and + * `unshare(1)` on `$PATH` (busybox's applet counts — see `src/exec/userns.ts` + * on why the flags are spelled short). + * + * The device is opened *before* `/proc/filesystems` is read, deliberately: + * `/dev/fuse` is a misc device, so opening it is what triggers the module + * autoload that puts `fuse` in that table in the first place. Reading the table + * first would refuse a host that was one `open(2)` away from working. + * + * `platform` exists to be overridden in tests; leave it alone otherwise. + */ +export function usernsExecProbe(platform: NodeJS.Platform = process.platform): UsernsExecProbe { + const host = execPlatform(platform); + // Only ask this host about itself: with a `platform` override in play the + // files below describe the machine the test runs on rather than the one it is + // asking about, and reporting those would be a lie in both directions. + const linux = host !== undefined && process.platform === "linux"; + const missing: string[] = []; + if (host === undefined) { + missing.push( + `this is ${platform}; a namespace-private FUSE mount needs Linux — no other kernel ` + + `has user namespaces, and macFUSE speaks a dialect mountx does not implement`, + ); + } + const device = linux ? deviceRefusal() : NOT_THIS_HOST; + const kernel = linux && hasFilesystem("fuse"); + const namespaces = linux ? usernsRefusal((process.getuid?.() ?? -1) === 0) : NOT_THIS_HOST; + const unshare = linux ? onPath("unshare") : undefined; + if (host !== undefined && !linux) { + missing.push(NOT_THIS_HOST); + } + if (linux) { + if (device !== undefined) { + missing.push(device); + } + if (device === undefined && !kernel) { + missing.push( + "the kernel has no `fuse` in /proc/filesystems even after /dev/fuse opened, which is " + + "a kernel built without CONFIG_FUSE_FS", + ); + } + if (namespaces !== undefined) { + missing.push(namespaces); + } + if (unshare === undefined) { + missing.push( + "there is no `unshare` on $PATH — util-linux and busybox both provide one, and the " + + "namespace can only be entered by a child process (Node is never single-threaded, " + + "and unshare(CLONE_NEWUSER) refuses a threaded caller)", + ); + } + } + return { + usable: missing.length === 0, + platform: host, + kernel, + device: device === undefined, + userns: namespaces === undefined, + unshare, + reason: missing.length === 0 ? undefined : missing.join("; "), + }; +} + +/** + * Can this host run a command under a seccomp user-notification supervisor? + * + * **What makes it usable:** Linux, an architecture the supervisor's syscall + * table covers (x86-64 as built — a second table, not a redesign, is what arm64 + * would take), a kernel offering `SECCOMP_RET_USER_NOTIF`, and the built + * supervisor binary itself. `CONFIG_SECCOMP_FILTER` is in every mainstream + * kernel and cannot be a module, so the kernel half is nearly always yes; the + * binary is the half that usually is not, because it is not shipped in the npm + * package (see `src/exec/seccomp.ts`). + * + * `platform`, `arch` and `supervisor` exist to be overridden in tests; leave + * them alone otherwise. + */ +export function seccompExecProbe( + platform: NodeJS.Platform = process.platform, + arch: string = process.arch, + supervisor: string | undefined = process.env.MOUNTX_TRACE, +): SeccompExecProbe { + const host = execPlatform(platform); + const linux = host !== undefined && process.platform === "linux"; + const okArch = arch === "x64"; + // `/proc/sys/kernel/seccomp/actions_avail` is a space-separated list of the + // return actions this kernel implements. Absent on a kernel too old to have + // the file, which is also too old to have user notification. + const actions = linux ? (readQuietly("/proc/sys/kernel/seccomp/actions_avail") ?? "") : ""; + const userNotif = actions.split(/\s+/).includes("user_notif"); + const found = supervisor !== undefined && isExecutable(supervisor) ? supervisor : undefined; + + const missing: string[] = []; + if (host === undefined) { + missing.push( + `this is ${platform}; seccomp user notification is a Linux facility and has no ` + + `counterpart on any other kernel`, + ); + } + if (host !== undefined && !okArch) { + missing.push( + `the supervisor's syscall filter is written against x86-64 and this is ${arch} — a ` + + `second syscall table, not a redesign, but it is not written yet`, + ); + } + if (host !== undefined && !linux) { + missing.push(NOT_THIS_HOST); + } + if (linux && !userNotif) { + missing.push( + "this kernel does not offer SECCOMP_RET_USER_NOTIF (no `user_notif` in " + + "/proc/sys/kernel/seccomp/actions_avail), which needs Linux 5.0 or newer built with " + + "CONFIG_SECCOMP_FILTER", + ); + } + if (host !== undefined && found === undefined) { + missing.push( + supervisor === undefined + ? "no supervisor binary — it is built from `src/exec/seccomp/` with a Zig toolchain " + + "and pointed at with $MOUNTX_TRACE or the `trace` option; it is not shipped in the " + + "npm package" + : `the supervisor at ${supervisor} is not an executable file`, + ); + } + return { + usable: missing.length === 0, + platform: host, + arch: okArch, + userNotif, + supervisor: found, + reason: missing.length === 0 ? undefined : missing.join("; "), + }; +} + +/** Is `path` a file this process may execute? */ +function isExecutable(path: string): boolean { + try { + fs.accessSync(path, fs.constants.X_OK); + return fs.statSync(path).isFile(); + } catch { + return false; + } +} diff --git a/src/exec/seccomp.ts b/src/exec/seccomp.ts new file mode 100644 index 0000000..9f047a2 --- /dev/null +++ b/src/exec/seccomp.ts @@ -0,0 +1,109 @@ +/** + * `execSeccomp()` — run a command whose filesystem syscalls are answered by an + * `FsDriver`, with no kernel mount and no libc involvement. + * + * The parent side is `createP9Server()` on a private unix socket, which is the + * point worth noticing: the supervisor is a *client* of the same unchanged + * server `mount9p()` points the kernel's v9fs client at. Everything that + * decides what the filesystem does still lives in `src/9p/session.ts`, and this + * side is a socket and a process. + * + * What makes the mechanism worth having is where the boundary sits. An + * `LD_PRELOAD` interposer sees glibc's exported symbols and therefore serves + * only what dynamically links glibc; a seccomp filter sees the syscall ABI, so + * a static musl binary, a Go binary and `cat` are indistinguishable to it. + * `src/exec/seccomp/trace.zig` is the supervisor, and its header is where the + * mechanism, and every gap it still has, is written down. + * + * Needs no privileges — `no_new_privs` is all an unprivileged filter requires — + * and no namespace, no device node and no filesystem driver. Linux only, and + * x86-64 only, since the filter compares against one syscall table and refuses + * to interpret any other. + */ + +import { type ChildProcess, spawn, type StdioOptions } from "node:child_process"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { createP9Server, type P9ServerOptions } from "../9p/server.ts"; +import type { FsDriver } from "../types.ts"; + +export interface ExecSeccompOptions extends P9ServerOptions { + /** The path prefix the supervisor claims. Nothing is mounted there. */ + root?: string; + /** The built supervisor binary. Defaults to `$MOUNTX_TRACE`. */ + trace?: string; + cwd?: string; + env?: NodeJS.ProcessEnv; + /** + * How the command's standard streams are wired, as `child_process` spells it. + * Defaults to `"inherit"`, which is what a command run for its output wants. + */ + stdio?: StdioOptions; + /** + * The child, the moment it exists. + * + * `execSeccomp` resolves when the command has *exited*, which is the wrong + * shape for a command that is being driven — a helper answering requests on + * its standard input, say. This is the handle for that: whatever it is given + * has already been spawned, and the promise is still outstanding. + */ + onSpawn?: (child: ChildProcess) => void; +} + +export interface ExecSeccompResult { + code: number | null; + signal: NodeJS.Signals | null; + root: string; + requests: number; +} + +export async function execSeccomp( + driver: FsDriver, + argv: readonly string[], + options: ExecSeccompOptions = {}, +): Promise { + if (argv.length === 0) { + throw new Error("mountx: execSeccomp needs a command to run"); + } + const trace = options.trace ?? process.env.MOUNTX_TRACE; + if (trace === undefined) { + throw new Error( + "mountx: execSeccomp needs the built supervisor — pass `trace` or set $MOUNTX_TRACE", + ); + } + const root = options.root ?? "/mountx"; + const scratch = await mkdtemp(join(tmpdir(), "mountx-seccomp-")); + const socketPath = join(scratch, "9p.sock"); + const server = createP9Server(driver, { ...options, path: socketPath }); + await server.listen(); + + const seen = new Set<(typeof server.clients)[number]["session"]>(); + const sampler = setInterval(() => { + for (const connection of server.clients) seen.add(connection.session); + }, 20); + sampler.unref(); + + try { + const child = spawn(trace, [socketPath, root, "--", ...argv], { + stdio: options.stdio ?? "inherit", + cwd: options.cwd ?? process.cwd(), + env: { ...(options.env ?? process.env), MOUNTX_ROOT: root }, + }); + options.onSpawn?.(child); + const result = await new Promise<{ code: number | null; signal: NodeJS.Signals | null }>( + (resolveExit, rejectExit) => { + child.on("error", rejectExit); + child.on("exit", (code, signal) => resolveExit({ code, signal })); + }, + ); + for (const connection of server.clients) seen.add(connection.session); + let requests = 0; + for (const session of seen) requests += session.stats.requests; + return { ...result, root, requests }; + } finally { + clearInterval(sampler); + await server.close(); + await rm(scratch, { recursive: true, force: true }).catch(() => {}); + } +} diff --git a/src/exec/seccomp/linux.zig b/src/exec/seccomp/linux.zig new file mode 100644 index 0000000..cd59ed0 --- /dev/null +++ b/src/exec/seccomp/linux.zig @@ -0,0 +1,315 @@ +//! The kernel ABI the supervisor speaks, transcribed rather than imported. +//! +//! Every number and every struct in this file crosses a process boundary: the +//! syscall numbers select what the BPF filter traps, and the structs are +//! written into *another process's* memory. So the layouts have to be the +//! kernel's, not whatever this binary's libc believes — which is the same rule +//! `src/9p/constants.ts` and `src/fuse/constants.ts` are held to, met from the +//! other side. Sources are named per group and are all at tag **v6.12**, the +//! tag the rest of the repository transcribes from. +//! +//! The one thing deliberately *not* here is the errno table: it is transcribed +//! once in `src/errors.ts`, `Rlerror` carries those numbers verbatim, and the +//! handlers pass them straight back to the tracee. The few values below are the +//! ones the supervisor *originates* rather than forwards. + +// --------------------------------------------------------------------------- +// Syscall numbers — arch/x86/entry/syscalls/syscall_64.tbl +// +// x86-64 only, and the filter refuses any other syscall table outright rather +// than reading these numbers against it (see `notify.zig`). arm64 would be a +// second table here, not a redesign. +// --------------------------------------------------------------------------- + +pub const SYS = struct { + pub const read = 0; + pub const write = 1; + pub const open = 2; + pub const close = 3; + pub const stat = 4; + pub const fstat = 5; + pub const lstat = 6; + pub const lseek = 8; + pub const mmap = 9; + pub const dup2 = 33; + pub const pread64 = 17; + pub const pwrite64 = 18; + pub const readv = 19; + pub const writev = 20; + pub const access = 21; + pub const sendfile = 40; + pub const fsync = 74; + pub const fdatasync = 75; + pub const truncate = 76; + pub const ftruncate = 77; + pub const getcwd = 79; + pub const chdir = 80; + pub const fchdir = 81; + pub const rename = 82; + pub const mkdir = 83; + pub const rmdir = 84; + pub const creat = 85; + pub const link = 86; + pub const unlink = 87; + pub const symlink = 88; + pub const readlink = 89; + pub const chmod = 90; + pub const fchmod = 91; + pub const chown = 92; + pub const fchown = 93; + pub const lchown = 94; + pub const utime = 132; + pub const mknod = 133; + pub const statfs = 137; + pub const fstatfs = 138; + pub const setxattr = 188; + pub const lsetxattr = 189; + pub const fsetxattr = 190; + pub const getxattr = 191; + pub const lgetxattr = 192; + pub const fgetxattr = 193; + pub const listxattr = 194; + pub const llistxattr = 195; + pub const flistxattr = 196; + pub const removexattr = 197; + pub const lremovexattr = 198; + pub const fremovexattr = 199; + pub const getdents64 = 217; + pub const exit_group = 231; + pub const utimes = 235; + pub const openat = 257; + pub const mkdirat = 258; + pub const mknodat = 259; + pub const fchownat = 260; + pub const futimesat = 261; + pub const newfstatat = 262; + pub const unlinkat = 263; + pub const renameat = 264; + pub const linkat = 265; + pub const symlinkat = 266; + pub const readlinkat = 267; + pub const fchmodat = 268; + pub const faccessat = 269; + pub const splice = 275; + pub const utimensat = 280; + pub const fallocate = 285; + pub const preadv = 295; + pub const pwritev = 296; + pub const renameat2 = 316; + pub const copy_file_range = 326; + pub const preadv2 = 327; + pub const pwritev2 = 328; + pub const dup3 = 292; + pub const statx = 332; + pub const close_range = 436; + pub const openat2 = 437; + pub const faccessat2 = 439; +}; + +// --------------------------------------------------------------------------- +// open(2) flags — include/uapi/asm-generic/fcntl.h (x86-64 uses the generic set) +// +// This is the *kernel's* namespace, and it is also the one 9P2000.L's +// `Tlopen.flags` carries: the wire and the host are the same kernel here, so +// the flags cross to the driver untranslated, exactly as +// `src/9p/session.ts` documents for `driverOpenFlags()`. +// --------------------------------------------------------------------------- + +pub const O_ACCMODE: u32 = 0o3; +pub const O_RDONLY: u32 = 0o0; +pub const O_WRONLY: u32 = 0o1; +pub const O_RDWR: u32 = 0o2; +pub const O_CREAT: u32 = 0o100; +pub const O_EXCL: u32 = 0o200; +pub const O_NOCTTY: u32 = 0o400; +pub const O_TRUNC: u32 = 0o1000; +pub const O_APPEND: u32 = 0o2000; +pub const O_NONBLOCK: u32 = 0o4000; +pub const O_DIRECTORY: u32 = 0o200000; +pub const O_NOFOLLOW: u32 = 0o400000; +pub const O_CLOEXEC: u32 = 0o2000000; +pub const O_PATH: u32 = 0o10000000; +pub const O_TMPFILE: u32 = 0o20000000; + +// --------------------------------------------------------------------------- +// *at() flags — include/uapi/linux/fcntl.h +// --------------------------------------------------------------------------- + +pub const AT_FDCWD: i32 = -100; +pub const AT_SYMLINK_NOFOLLOW: u32 = 0x100; +pub const AT_REMOVEDIR: u32 = 0x200; +pub const AT_SYMLINK_FOLLOW: u32 = 0x400; +pub const AT_EMPTY_PATH: u32 = 0x1000; +/// `faccessat2`'s only interesting flag; `AT_EACCESS` is the other and this +/// supervisor makes no distinction between real and effective ids. +pub const AT_EACCESS: u32 = 0x200; + +// --------------------------------------------------------------------------- +// File modes — include/uapi/linux/stat.h +// --------------------------------------------------------------------------- + +pub const S_IFMT: u32 = 0o170000; +pub const S_IFSOCK: u32 = 0o140000; +pub const S_IFLNK: u32 = 0o120000; +pub const S_IFREG: u32 = 0o100000; +pub const S_IFBLK: u32 = 0o060000; +pub const S_IFDIR: u32 = 0o040000; +pub const S_IFCHR: u32 = 0o020000; +pub const S_IFIFO: u32 = 0o010000; + +// --------------------------------------------------------------------------- +// Errno — the values `src/errors.ts` transcribes, for the ones this supervisor +// raises itself rather than forwarding from an `Rlerror`. +// --------------------------------------------------------------------------- + +pub const EPERM: i32 = 1; +pub const ENOENT: i32 = 2; +pub const EIO: i32 = 5; +pub const EBADF: i32 = 9; +pub const ENOMEM: i32 = 12; +pub const EACCES: i32 = 13; +pub const EFAULT: i32 = 14; +pub const EBUSY: i32 = 16; +pub const EEXIST: i32 = 17; +pub const EXDEV: i32 = 18; +pub const ENODEV: i32 = 19; +pub const ENOTDIR: i32 = 20; +pub const EISDIR: i32 = 21; +pub const EINVAL: i32 = 22; +pub const EMFILE: i32 = 24; +pub const ESPIPE: i32 = 29; +pub const ERANGE: i32 = 34; +pub const ENAMETOOLONG: i32 = 36; +pub const ENOSYS: i32 = 38; +pub const ELOOP: i32 = 40; +pub const EOVERFLOW: i32 = 75; +pub const ENOTSUP: i32 = 95; + +// --------------------------------------------------------------------------- +// access(2) — include/uapi/linux/fcntl.h / unistd.h +// --------------------------------------------------------------------------- + +pub const F_OK: u32 = 0; +pub const X_OK: u32 = 1; +pub const W_OK: u32 = 2; +pub const R_OK: u32 = 4; + +// --------------------------------------------------------------------------- +// lseek(2) — include/uapi/linux/fs.h +// --------------------------------------------------------------------------- + +pub const SEEK_SET: u32 = 0; +pub const SEEK_CUR: u32 = 1; +pub const SEEK_END: u32 = 2; + +/// `mmap(2)`'s `MAP_ANONYMOUS` — include/uapi/asm-generic/mman-common.h. +pub const MAP_ANONYMOUS: u64 = 0x20; + +/// `utimensat(2)`'s two sentinels — include/uapi/linux/stat.h. +pub const UTIME_NOW: i64 = (1 << 30) - 1; +pub const UTIME_OMIT: i64 = (1 << 30) - 2; + +/// `d_type` values — include/uapi/linux/fs.h (`DT_*`), the same byte +/// `src/9p/protocol.ts` packs into a dirent. +pub const DT_UNKNOWN: u8 = 0; +pub const DT_FIFO: u8 = 1; +pub const DT_CHR: u8 = 2; +pub const DT_DIR: u8 = 4; +pub const DT_BLK: u8 = 6; +pub const DT_REG: u8 = 8; +pub const DT_LNK: u8 = 10; +pub const DT_SOCK: u8 = 12; + +// --------------------------------------------------------------------------- +// Structs written into tracee memory +// --------------------------------------------------------------------------- + +/// `struct stat` as x86-64 Linux lays it out — arch/x86/include/uapi/asm/stat.h. +pub const Stat = extern struct { + st_dev: u64, + st_ino: u64, + st_nlink: u64, + st_mode: u32, + st_uid: u32, + st_gid: u32, + __pad0: u32, + st_rdev: u64, + st_size: i64, + st_blksize: i64, + st_blocks: i64, + st_atime: u64, + st_atime_nsec: u64, + st_mtime: u64, + st_mtime_nsec: u64, + st_ctime: u64, + st_ctime_nsec: u64, + __unused: [3]i64, +}; + +/// `struct statx` — include/uapi/linux/stat.h. Only the fields this supervisor +/// fills are named; the tail is zeroed. +pub const Statx = extern struct { + stx_mask: u32, + stx_blksize: u32, + stx_attributes: u64, + stx_nlink: u32, + stx_uid: u32, + stx_gid: u32, + stx_mode: u16, + __spare0: u16, + stx_ino: u64, + stx_size: u64, + stx_blocks: u64, + stx_attributes_mask: u64, + stx_atime: Timestamp, + stx_btime: Timestamp, + stx_ctime: Timestamp, + stx_mtime: Timestamp, + stx_rdev_major: u32, + stx_rdev_minor: u32, + stx_dev_major: u32, + stx_dev_minor: u32, + stx_mnt_id: u64, + __spare2: u64, + __spare3: [12]u64, + + pub const Timestamp = extern struct { sec: i64, nsec: u32, __pad: i32 }; +}; + +/// `STATX_BASIC_STATS` — the fields a `struct stat` has. +pub const STATX_BASIC_STATS: u32 = 0x0000_07ff; + +/// `struct statfs` on x86-64 — include/uapi/asm-generic/statfs.h with +/// `__statfs_word` as `__kernel_long_t`, so every field is 64 bits and the +/// whole struct is 120 bytes. +pub const Statfs = extern struct { + f_type: i64, + f_bsize: i64, + f_blocks: u64, + f_bfree: u64, + f_bavail: u64, + f_files: u64, + f_ffree: u64, + f_fsid: [2]i32, + f_namelen: i64, + f_frsize: i64, + f_flags: i64, + f_spare: [4]i64, +}; + +/// `struct iovec` — include/uapi/linux/uio.h. +pub const Iovec = extern struct { base: u64, len: u64 }; + +/// `struct timespec` — include/uapi/linux/time.h. +pub const Timespec = extern struct { sec: i64, nsec: i64 }; + +/// `struct timeval`, for the legacy `utimes`/`futimesat`. +pub const Timeval = extern struct { sec: i64, usec: i64 }; + +/// `struct utimbuf`, for the legacy `utime`. +pub const Utimbuf = extern struct { actime: i64, modtime: i64 }; + +/// The header of a `struct linux_dirent64` — include/uapi/linux/fs.h. Variable +/// length, so `getdents64` builds it by hand: `d_ino[8] d_off[8] d_reclen[2] +/// d_type[1]` then the name and its NUL, padded to 8. +pub const DIRENT64_HEADER = 19; diff --git a/src/exec/seccomp/notify.zig b/src/exec/seccomp/notify.zig new file mode 100644 index 0000000..965cdbb --- /dev/null +++ b/src/exec/seccomp/notify.zig @@ -0,0 +1,457 @@ +//! The seccomp user-notification mechanism itself: the filter, the listener, +//! and the two ways the supervisor touches a tracee (its memory and its +//! `/proc` entry). +//! +//! ### Why the listener now crosses a socket +//! +//! The spike installed the filter on the supervisor *itself* and forked, +//! because a seccomp filter is inherited across `fork` and `exec` and that +//! avoided passing a descriptor anywhere. It cost one rule the file had to +//! keep: **the supervisor may never make a trapped syscall**, since it would +//! suspend itself waiting for a reply only it could send. That rule is what +//! kept `close` out of the trapped set — and leaving `close` untrapped costs +//! correctness, not just memory: descriptor numbers are reused immediately, so +//! a stale mapping shadows a live one. +//! +//! So the shape is inverted here, into the one `native/src/main.zig` already +//! implements for `fusermount3`: **fork, have the *child* install the filter, +//! and hand the listener back over `SCM_RIGHTS`**. The supervisor then carries +//! no filter at all and may call anything, which is what makes trapping +//! `close`, `read`, `write`, `mmap` and the rest possible. The `cmsg` +//! arithmetic below is transcribed from `include/linux/socket.h`, exactly as it +//! is there — the kernel's own definitions rather than glibc's wrappers, since +//! this binary must build against musl too. + +const std = @import("std"); +const linux = std.os.linux; + +const c = @cImport({ + @cDefine("_GNU_SOURCE", "1"); + @cInclude("sys/ioctl.h"); + @cInclude("sys/prctl.h"); + @cInclude("linux/seccomp.h"); + @cInclude("linux/filter.h"); + @cInclude("errno.h"); +}); + +pub const Notif = c.struct_seccomp_notif; +pub const NotifResp = c.struct_seccomp_notif_resp; +pub const NotifAddfd = c.struct_seccomp_notif_addfd; + +/// `EM_X86_64 | __AUDIT_ARCH_64BIT | __AUDIT_ARCH_LE` — 62 | 0x80000000 | +/// 0x40000000, from `linux/audit.h`. Transcribed rather than `@cImport`ed +/// because it overflows a C `int` and does not survive translation. +pub const AUDIT_ARCH_X86_64: u32 = 0xc000_003e; + +/// BPF instruction classes, from `linux/bpf_common.h`. +const BPF_LD_W_ABS: u16 = 0x20; +const BPF_JMP_JEQ_K: u16 = 0x15; +const BPF_RET_K: u16 = 0x06; +/// `offsetof(struct seccomp_data, ...)`. +const OFF_NR: u32 = 0; +const OFF_ARCH: u32 = 4; + +/// `ioctl` by raw syscall rather than through libc. +/// +/// The request numbers here have the high bit set (`SECCOMP_IOCTL_NOTIF_RECV` +/// is 0xc0502100) and the two libcs disagree about the parameter's type: +/// glibc's `ioctl` takes `unsigned long`, musl's takes `int`. Passing the +/// constant through libc therefore fails to compile against musl and would +/// sign-extend if forced. The syscall takes an unsigned long on every ABI. +const SYS_ioctl: usize = 16; + +fn syscall3(n: usize, a1: usize, a2: usize, a3: usize) isize { + return asm volatile ("syscall" + : [ret] "={rax}" (-> isize), + : [number] "{rax}" (n), + [arg1] "{rdi}" (a1), + [arg2] "{rsi}" (a2), + [arg3] "{rdx}" (a3), + : .{ .rcx = true, .r11 = true, .memory = true }); +} + +fn ioctl(fd: i32, request: u64, arg: usize) isize { + return syscall3(SYS_ioctl, @intCast(fd), @intCast(request), arg); +} + +/// The listener, once the handshake below has delivered it. +var listener: i32 = -1; + +pub fn setListener(fd: i32) void { + listener = fd; +} + +// --------------------------------------------------------------------------- +// The filter +// --------------------------------------------------------------------------- + +/// Build and install a filter trapping exactly `trapped`, and return the +/// listener. Called **in the child**, which then hands the listener back. +/// +/// The arch check is first and refuses to interpret a syscall table that is not +/// the one these numbers belong to: a 32-bit call arriving on an x86-64 kernel +/// has entirely different numbers, and answering it as if it did not would be +/// worse than letting it through. +pub fn installFilter(trapped: []const u32) !i32 { + var prog: [6 + 128]c.struct_sock_filter = undefined; + if (trapped.len > 128) return error.TooManySyscalls; + var n: usize = 0; + prog[n] = .{ .code = BPF_LD_W_ABS, .jt = 0, .jf = 0, .k = OFF_ARCH }; + n += 1; + prog[n] = .{ .code = BPF_JMP_JEQ_K, .jt = 1, .jf = 0, .k = AUDIT_ARCH_X86_64 }; + n += 1; + prog[n] = .{ .code = BPF_RET_K, .jt = 0, .jf = 0, .k = c.SECCOMP_RET_ALLOW }; + n += 1; + prog[n] = .{ .code = BPF_LD_W_ABS, .jt = 0, .jf = 0, .k = OFF_NR }; + n += 1; + // Each comparison jumps forward to the single USER_NOTIF at the end; the + // distance shrinks by one for each comparison already passed. + const count: usize = trapped.len; + for (trapped, 0..) |nr, i| { + prog[n] = .{ .code = BPF_JMP_JEQ_K, .jt = @intCast(count - i), .jf = 0, .k = nr }; + n += 1; + } + prog[n] = .{ .code = BPF_RET_K, .jt = 0, .jf = 0, .k = c.SECCOMP_RET_ALLOW }; + n += 1; + prog[n] = .{ .code = BPF_RET_K, .jt = 0, .jf = 0, .k = c.SECCOMP_RET_USER_NOTIF }; + n += 1; + + const fprog = c.struct_sock_fprog{ .len = @intCast(n), .filter = &prog }; + // Without `no_new_privs` an unprivileged process may not install a filter + // at all — the kernel's guard against using seccomp to confuse a setuid + // binary it then execs. Setting it is irreversible and inherited across + // `exec`, which is exactly the intent: the traced command runs under it. + if (c.prctl(c.PR_SET_NO_NEW_PRIVS, @as(c_ulong, 1), @as(c_ulong, 0), @as(c_ulong, 0), @as(c_ulong, 0)) != 0) { + return error.NoNewPrivs; + } + const rc = linux.syscall3( + .seccomp, + c.SECCOMP_SET_MODE_FILTER, + c.SECCOMP_FILTER_FLAG_NEW_LISTENER, + @intFromPtr(&fprog), + ); + const signed: isize = @bitCast(rc); + if (signed < 0) return error.SeccompFailed; + return @intCast(signed); +} + +// --------------------------------------------------------------------------- +// cmsg(3) arithmetic — include/linux/socket.h, the same transcription +// `native/src/main.zig` carries for the `fusermount3` handshake. +// --------------------------------------------------------------------------- + +const CMSG_ALIGNMENT = @sizeOf(usize); + +fn cmsgAlign(length: usize) usize { + return (length + CMSG_ALIGNMENT - 1) & ~@as(usize, CMSG_ALIGNMENT - 1); +} + +const CMSG_DATA_OFFSET = cmsgAlign(@sizeOf(linux.cmsghdr)); + +fn cmsgLen(payload: usize) usize { + return CMSG_DATA_OFFSET + payload; +} + +fn cmsgSpace(payload: usize) usize { + return CMSG_DATA_OFFSET + cmsgAlign(payload); +} + +const ONE_FD_SPACE = cmsgSpace(@sizeOf(i32)); + +const ControlBuffer = extern struct { + bytes: [ONE_FD_SPACE]u8 align(@alignOf(linux.cmsghdr)), + + fn header(self: *ControlBuffer) *linux.cmsghdr { + return @ptrCast(&self.bytes); + } + + fn payload(self: *ControlBuffer) *align(CMSG_ALIGNMENT) i32 { + return @ptrCast(@alignCast(&self.bytes[CMSG_DATA_OFFSET])); + } +}; + +/// `socketpair(AF_UNIX, SOCK_STREAM, 0)` — no `SOCK_CLOEXEC`, because the whole +/// point of this pair is that the forked child inherits its end. +pub fn socketpair() ![2]i32 { + var fds: [2]i32 = undefined; + const rc = linux.socketpair(linux.AF.UNIX, linux.SOCK.STREAM, 0, &fds); + if (linux.errno(rc) != .SUCCESS) return error.SocketPair; + return fds; +} + +/// One descriptor, one byte of payload. The byte is mandatory: `unix(7)` will +/// not carry ancillary data on an empty message. +pub fn sendFd(socket: i32, fd: i32) !void { + var control: ControlBuffer = undefined; + control.header().* = .{ + .len = cmsgLen(@sizeOf(i32)), + .level = linux.SOL.SOCKET, + .type = linux.SCM.RIGHTS, + }; + control.payload().* = fd; + + var byte: u8 = 0; + const iov = [1]std.posix.iovec_const{.{ .base = @ptrCast(&byte), .len = 1 }}; + const message = linux.msghdr_const{ + .name = null, + .namelen = 0, + .iov = &iov, + .iovlen = 1, + .control = &control, + .controllen = cmsgLen(@sizeOf(i32)), + .flags = 0, + }; + while (true) { + const rc = linux.sendmsg(socket, &message, 0); + switch (linux.errno(rc)) { + .SUCCESS => return, + .INTR => continue, + else => return error.SendFd, + } + } +} + +/// The other half. `MSG_CMSG_CLOEXEC` keeps the listener out of anything this +/// process spawns later. +pub fn recvFd(socket: i32) !i32 { + var control: ControlBuffer = undefined; + var byte: u8 = 0; + var iov = [1]std.posix.iovec{.{ .base = @ptrCast(&byte), .len = 1 }}; + var message = linux.msghdr{ + .name = null, + .namelen = 0, + .iov = &iov, + .iovlen = 1, + .control = &control, + .controllen = ONE_FD_SPACE, + .flags = 0, + }; + const received = while (true) { + const rc = linux.recvmsg(socket, &message, linux.MSG.CMSG_CLOEXEC); + switch (linux.errno(rc)) { + .SUCCESS => break rc, + .INTR => continue, + else => return error.RecvFd, + } + }; + if (received == 0) return error.PeerClosed; + if (message.flags & linux.MSG.CTRUNC != 0) return error.Truncated; + if (message.controllen < cmsgLen(@sizeOf(i32))) return error.NoDescriptor; + const header = control.header(); + if (header.level != linux.SOL.SOCKET or header.type != linux.SCM.RIGHTS) { + return error.NotScmRights; + } + return control.payload().*; +} + +// --------------------------------------------------------------------------- +// Talking to the listener +// --------------------------------------------------------------------------- + +/// Block until a notification arrives, or until `timeout_ms` passes with none. +/// +/// The spike blocked in `SECCOMP_IOCTL_NOTIF_RECV` and checked `waitpid` before +/// re-entering it, which cannot notice a tracee that dies while the supervisor +/// is parked inside the ioctl. Polling first makes the loop's exit condition +/// checkable on a timer without ever leaving a notification unanswered. +pub fn wait(timeout_ms: i32) enum { ready, idle, failed } { + var fds = [1]linux.pollfd{.{ .fd = listener, .events = linux.POLL.IN, .revents = 0 }}; + const rc = linux.poll(&fds, 1, timeout_ms); + return switch (linux.errno(rc)) { + .SUCCESS => if (rc == 0) .idle else .ready, + .INTR => .idle, + else => .failed, + }; +} + +/// Take the next notification. False means the listener is done: every tracee +/// that could have sent one is gone. +pub fn receive(notif: *Notif) bool { + @memset(@as([*]u8, @ptrCast(notif))[0..@sizeOf(Notif)], 0); + return ioctl(listener, c.SECCOMP_IOCTL_NOTIF_RECV, @intFromPtr(notif)) == 0; +} + +/// Answer a notification with a return value or an errno. Exactly one of these +/// (or one `passthrough`) happens per notification, which is the errno +/// discipline `AGENTS.md` states, met at the syscall boundary. +pub fn respond(id: u64, val: i64, err: i32) void { + var resp = NotifResp{ .id = id, .val = val, .@"error" = err, .flags = 0 }; + _ = ioctl(listener, c.SECCOMP_IOCTL_NOTIF_SEND, @intFromPtr(&resp)); +} + +/// Let the kernel run the syscall as it stands. +pub fn passthrough(id: u64) void { + var resp = NotifResp{ + .id = id, + .val = 0, + .@"error" = 0, + .flags = c.SECCOMP_USER_NOTIF_FLAG_CONTINUE, + }; + _ = ioctl(listener, c.SECCOMP_IOCTL_NOTIF_SEND, @intFromPtr(&resp)); +} + +/// Install `fd` into the tracee and return the number it landed on there. +pub fn addFd(id: u64, fd: i32) i32 { + var req = NotifAddfd{ + .id = id, + .flags = 0, + .srcfd = @intCast(fd), + .newfd = 0, + .newfd_flags = 0, + }; + const got = ioctl(listener, c.SECCOMP_IOCTL_NOTIF_ADDFD, @intFromPtr(&req)); + if (got < 0) return -1; + return @intCast(got); +} + +/// Still the same syscall we were notified about? +/// +/// Between the notification arriving and the supervisor acting on it, the +/// traced thread can be killed and its pid reused — at which point every +/// address read out of "its" memory belongs to somebody else. This is the check +/// that makes reading tracee memory sound rather than probably-fine, and it has +/// to happen *after* the read, not before. +pub fn stillValid(id: u64) bool { + var copy = id; + return ioctl(listener, c.SECCOMP_IOCTL_NOTIF_ID_VALID, @intFromPtr(©)) == 0; +} + +// --------------------------------------------------------------------------- +// Tracee memory +// --------------------------------------------------------------------------- + +pub fn readTracee(pid: i32, remote: u64, into: []u8) bool { + if (into.len == 0) return true; + const liov = [1]std.posix.iovec{.{ .base = into.ptr, .len = into.len }}; + const riov = [1]std.posix.iovec_const{.{ .base = @ptrFromInt(remote), .len = into.len }}; + const rc = linux.process_vm_readv(pid, &liov, &riov, 0); + if (linux.errno(rc) != .SUCCESS) return false; + return rc == into.len; +} + +pub fn writeTracee(pid: i32, remote: u64, from: []const u8) bool { + if (from.len == 0) return true; + const liov = [1]std.posix.iovec_const{.{ .base = from.ptr, .len = from.len }}; + const riov = [1]std.posix.iovec_const{.{ .base = @ptrFromInt(remote), .len = from.len }}; + const rc = linux.process_vm_writev(pid, &liov, &riov, 0); + if (linux.errno(rc) != .SUCCESS) return false; + return rc == from.len; +} + +/// A NUL-terminated string out of the tracee, one page-safe chunk at a time. +/// +/// Reading a path from another process is the one genuinely delicate part of +/// this mechanism. **Every read is clamped to the end of the current page**: a +/// path can sit anywhere, including the last few bytes of a mapping — an argv +/// or envp string lives at the very top of the stack — and `process_vm_readv` +/// fails the *whole* request if any part of it is unmapped. Reading a fixed 64 +/// bytes therefore fails intermittently depending on where ASLR put the string, +/// which is exactly how this showed up in the spike: a raw-syscall probe opened +/// two files fine and then could not open a directory. +pub fn readTraceePath(pid: i32, remote: u64, into: []u8) ?[]const u8 { + if (remote == 0) return null; + var got: usize = 0; + while (got < into.len) { + const addr = remote + got; + const to_page_end = 4096 - (addr & 0xfff); + const chunk = @min(@min(@as(u64, 64), to_page_end), into.len - got); + if (!readTracee(pid, addr, into[got .. got + chunk])) { + if (got == 0) return null; + break; + } + for (into[got .. got + chunk], got..) |ch, i| { + if (ch == 0) return into[0..i]; + } + got += chunk; + } + return null; +} + +// --------------------------------------------------------------------------- +// /proc, for the three things a notification does not carry +// --------------------------------------------------------------------------- + +/// What `/proc//status` answers about a thread. +pub const ProcInfo = struct { + /// The thread group — i.e. the *process*, which is what owns descriptors. + tgid: i32, + ppid: i32, + umask: u32, +}; + +fn readSmallFile(path: [:0]const u8, into: []u8) ?[]const u8 { + const fd = linux.open(path, .{ .ACCMODE = .RDONLY, .CLOEXEC = true }, 0); + if (linux.errno(fd) != .SUCCESS) return null; + const handle: i32 = @intCast(fd); + defer _ = linux.close(handle); + var at: usize = 0; + while (at < into.len) { + const rc = linux.read(handle, into.ptr + at, into.len - at); + if (linux.errno(rc) != .SUCCESS) return null; + if (rc == 0) break; + at += rc; + } + return into[0..at]; +} + +fn fieldValue(text: []const u8, name: []const u8) ?[]const u8 { + var at: usize = 0; + while (at < text.len) { + const end = std.mem.indexOfScalarPos(u8, text, at, '\n') orelse text.len; + const line = text[at..end]; + if (std.mem.startsWith(u8, line, name)) { + var value = line[name.len..]; + while (value.len > 0 and (value[0] == ' ' or value[0] == '\t')) value = value[1..]; + return value; + } + at = end + 1; + } + return null; +} + +fn parseInt(comptime T: type, text: []const u8, base: u8) ?T { + var value: T = 0; + var any = false; + for (text) |ch| { + const digit: u8 = switch (ch) { + '0'...'9' => ch - '0', + 'a'...'f' => ch - 'a' + 10, + 'A'...'F' => ch - 'A' + 10, + else => break, + }; + if (digit >= base) break; + value = value * base + digit; + any = true; + } + return if (any) value else null; +} + +/// `Tgid`, `PPid` and `Umask` in one read. +/// +/// **`seccomp_notif.pid` is a thread id, not a process id** (the kernel fills it +/// with `task_pid_vnr`), and descriptors, the working directory and the umask +/// all belong to the *thread group*. A supervisor that keyed its tables on the +/// notification's `pid` would lose every descriptor the moment a second thread +/// touched it — which is precisely the multi-threaded tracee this has to serve. +pub fn procInfo(tid: i32) ?ProcInfo { + var name: [64]u8 = undefined; + const path = std.fmt.bufPrintZ(&name, "/proc/{d}/status", .{tid}) catch return null; + var buf: [4096]u8 = undefined; + const text = readSmallFile(path, &buf) orelse return null; + const tgid = parseInt(i32, fieldValue(text, "Tgid:") orelse return null, 10) orelse return null; + const ppid = parseInt(i32, fieldValue(text, "PPid:") orelse "0", 10) orelse 0; + const umask = if (fieldValue(text, "Umask:")) |value| + parseInt(u32, value, 8) orelse 0 + else + 0; + return .{ .tgid = tgid, .ppid = ppid, .umask = umask }; +} + +/// What `/proc//fd/` points at, for the descriptor-identity walk. +pub fn fdLink(pid: i32, fd: i32, into: []u8) ?[]const u8 { + var name: [64]u8 = undefined; + const path = std.fmt.bufPrintZ(&name, "/proc/{d}/fd/{d}", .{ pid, fd }) catch return null; + const rc = linux.readlink(path, into.ptr, into.len); + if (linux.errno(rc) != .SUCCESS) return null; + return into[0..rc]; +} diff --git a/src/exec/seccomp/p9.zig b/src/exec/seccomp/p9.zig new file mode 100644 index 0000000..f6eff18 --- /dev/null +++ b/src/exec/seccomp/p9.zig @@ -0,0 +1,815 @@ +//! SPIKE — a 9P2000.L client small enough to live inside an intercepted +//! process. Shared by spike B (the `LD_PRELOAD` interposer) and spike C (the +//! seccomp supervisor). +//! +//! **This file is the whole argument for the approach.** A syscall interceptor +//! that had to *implement* a filesystem would be reimplementing path +//! resolution, handle lifetimes, directory paging and error mapping — all of +//! which `src/9p/session.ts` already does, and already passes a conformance +//! column on. Speaking 9P instead makes the interceptor a *client*: it +//! translates one libc call into one or two 9P messages and translates the +//! answer back. Every filesystem question is settled on the far side of the +//! socket, in TypeScript, by code that is already tested. +//! +//! Constants and message layouts are transcribed from `src/9p/constants.ts` +//! and `src/9p/protocol.ts` in this repository, which are themselves +//! transcribed from the kernel's `include/net/9p/9p.h` at tag v6.12. Nothing +//! here is guessed and nothing is read out of a host header — the same rule +//! the TypeScript side is held to. +//! +//! Deliberately narrow for a spike: one request in flight at a time under a +//! spinlock, tag always zero, and read-mostly operations. That is enough to +//! answer the question the spikes exist to answer (can an intercepted process +//! be served at all, and by what) and nowhere near enough to ship. + +// --------------------------------------------------------------------------- +// Raw syscalls. +// +// The client does its own socket I/O with `syscall` instructions rather than +// through libc, for one reason that is specific to spike B: the interposer +// *is* libc's `read`/`write` for the duration of this process, so a client +// calling `read()` on its own socket would re-enter the shim. Going straight +// to the kernel makes that structurally impossible rather than merely +// avoided. +// --------------------------------------------------------------------------- + +pub const SYS_read = 0; +pub const SYS_write = 1; +pub const SYS_close = 3; +pub const SYS_lseek = 8; +pub const SYS_socket = 41; +pub const SYS_connect = 42; +pub const SYS_getpid = 39; +pub const SYS_memfd_create = 319; + +pub fn syscall3(n: usize, a1: usize, a2: usize, a3: usize) isize { + return asm volatile ("syscall" + : [ret] "={rax}" (-> isize), + : [number] "{rax}" (n), + [arg1] "{rdi}" (a1), + [arg2] "{rsi}" (a2), + [arg3] "{rdx}" (a3), + : .{ .rcx = true, .r11 = true, .memory = true }); +} + +pub fn syscall0(n: usize) isize { + return asm volatile ("syscall" + : [ret] "={rax}" (-> isize), + : [number] "{rax}" (n), + : .{ .rcx = true, .r11 = true, .memory = true }); +} + +const AF_UNIX = 1; +const SOCK_STREAM = 1; + +// --------------------------------------------------------------------------- +// Wire constants — transcribed from src/9p/constants.ts. +// --------------------------------------------------------------------------- + +pub const P9_RLERROR = 7; +pub const P9_TSTATFS = 8; +pub const P9_TLOPEN = 12; +pub const P9_TLCREATE = 14; +pub const P9_TSYMLINK = 16; +pub const P9_TMKNOD = 18; +pub const P9_TRENAME = 20; +pub const P9_TREADLINK = 22; +pub const P9_TGETATTR = 24; +pub const P9_TSETATTR = 26; +pub const P9_TREADDIR = 40; +pub const P9_TFSYNC = 50; +pub const P9_TLINK = 70; +pub const P9_TMKDIR = 72; +pub const P9_TRENAMEAT = 74; +pub const P9_TUNLINKAT = 76; +pub const P9_TVERSION = 100; +pub const P9_TATTACH = 104; +pub const P9_TWALK = 110; +pub const P9_TREAD = 116; +pub const P9_TWRITE = 118; +pub const P9_TCLUNK = 120; + +pub const P9_NOFID: u32 = 0xffff_ffff; +pub const P9_MAXWELEM = 16; +pub const P9_IOHDRSZ = 24; +/// `P9_GETATTR_BASIC` — everything a `struct stat` needs and nothing reserved. +pub const P9_GETATTR_BASIC: u64 = 0x0000_07ff; + +/// `Tsetattr.valid` bits — `P9_ATTR_*` in the kernel's `fs/9p/vfs_inode_dotl.c`, +/// spelled after the field each one fills the way `src/9p/constants.ts` spells +/// them. +pub const P9_SETATTR_MODE: u32 = 1 << 0; +pub const P9_SETATTR_UID: u32 = 1 << 1; +pub const P9_SETATTR_GID: u32 = 1 << 2; +pub const P9_SETATTR_SIZE: u32 = 1 << 3; +pub const P9_SETATTR_ATIME: u32 = 1 << 4; +pub const P9_SETATTR_MTIME: u32 = 1 << 5; +pub const P9_SETATTR_CTIME: u32 = 1 << 6; +pub const P9_SETATTR_ATIME_SET: u32 = 1 << 7; +pub const P9_SETATTR_MTIME_SET: u32 = 1 << 8; + +/// `Tunlinkat.flags` — the `AT_REMOVEDIR` of `unlinkat(2)`, and the only flag +/// the message carries. +pub const P9_DOTL_AT_REMOVEDIR: u32 = 0x200; + +/// Qid type bits, the file-type half of a 9P identity. +pub const P9_QTDIR: u8 = 0x80; +pub const P9_QTSYMLINK: u8 = 0x02; + +/// The kernel's own `DEFAULT_MSIZE` payload, matching what `mount9p()` picks. +pub const MSIZE: u32 = 128 * 1024 + P9_IOHDRSZ; + +/// Header: `size[4] type[1] tag[2]`. +const HDR = 7; + +// --------------------------------------------------------------------------- + +pub const Error = error{ + Disconnected, + Protocol, + /// The server answered `Rlerror`; the errno is in `Client.last_errno`. + Remote, +}; + +/// Little-endian, unaligned, the same shape as `src/9p/wire.ts`'s writer. +const Writer = struct { + buf: []u8, + at: usize = HDR, + + fn u8v(self: *Writer, v: u8) void { + self.buf[self.at] = v; + self.at += 1; + } + fn u16v(self: *Writer, v: u16) void { + self.buf[self.at] = @truncate(v); + self.buf[self.at + 1] = @truncate(v >> 8); + self.at += 2; + } + fn u32v(self: *Writer, v: u32) void { + var i: usize = 0; + while (i < 4) : (i += 1) self.buf[self.at + i] = @truncate(v >> @intCast(i * 8)); + self.at += 4; + } + fn u64v(self: *Writer, v: u64) void { + var i: usize = 0; + while (i < 8) : (i += 1) self.buf[self.at + i] = @truncate(v >> @intCast(i * 8)); + self.at += 8; + } + fn str(self: *Writer, s: []const u8) void { + self.u16v(@intCast(s.len)); + @memcpy(self.buf[self.at .. self.at + s.len], s); + self.at += s.len; + } +}; + +/// Bounds-checked in the sense that matters here: every read is against the +/// declared message length, and a short message answers `Protocol` rather than +/// reading into whatever the buffer held last time. +pub const Reader = struct { + buf: []const u8, + at: usize = 0, + + pub fn u8v(self: *Reader) Error!u8 { + if (self.at + 1 > self.buf.len) return Error.Protocol; + defer self.at += 1; + return self.buf[self.at]; + } + pub fn u16v(self: *Reader) Error!u16 { + if (self.at + 2 > self.buf.len) return Error.Protocol; + defer self.at += 2; + return @as(u16, self.buf[self.at]) | (@as(u16, self.buf[self.at + 1]) << 8); + } + pub fn u32v(self: *Reader) Error!u32 { + if (self.at + 4 > self.buf.len) return Error.Protocol; + var v: u32 = 0; + var i: usize = 0; + while (i < 4) : (i += 1) v |= @as(u32, self.buf[self.at + i]) << @intCast(i * 8); + self.at += 4; + return v; + } + pub fn u64v(self: *Reader) Error!u64 { + if (self.at + 8 > self.buf.len) return Error.Protocol; + var v: u64 = 0; + var i: usize = 0; + while (i < 8) : (i += 1) v |= @as(u64, self.buf[self.at + i]) << @intCast(i * 8); + self.at += 8; + return v; + } + pub fn skip(self: *Reader, n: usize) Error!void { + if (self.at + n > self.buf.len) return Error.Protocol; + self.at += n; + } + pub fn str(self: *Reader) Error![]const u8 { + const n = try self.u16v(); + if (self.at + n > self.buf.len) return Error.Protocol; + defer self.at += n; + return self.buf[self.at .. self.at + n]; + } +}; + +/// The 13 bytes of a qid: `type[1] version[4] path[8]`. +pub const Qid = struct { + qtype: u8, + version: u32, + path: u64, + + pub fn read(r: *Reader) Error!Qid { + return .{ .qtype = try r.u8v(), .version = try r.u32v(), .path = try r.u64v() }; + } +}; + +/// The subset of `Rgetattr` a `struct stat` is built from. Field order is +/// `src/9p/protocol.ts`'s `writeRgetattr`, which is where the widths that do +/// *not* match the C struct come from: `mode`/`uid`/`gid` are 32-bit while +/// `nlink`/`rdev`/`blksize`/`blocks` are 64. +pub const Attr = struct { + qid: Qid, + mode: u32, + uid: u32, + gid: u32, + nlink: u64, + rdev: u64, + size: u64, + blksize: u64, + blocks: u64, + atime_sec: u64, + mtime_sec: u64, + ctime_sec: u64, + atime_nsec: u64, + mtime_nsec: u64, + ctime_nsec: u64, +}; + +pub const Client = struct { + fd: i32 = -1, + msize: u32 = MSIZE, + root_fid: u32 = 0, + next_fid: u32 = 1, + /// The errno from the most recent `Rlerror`, positive and Linux's, which + /// is exactly what the shim needs to put in its own `errno`. 9P is the one + /// transport in this repo with no status-mapping layer at all. + last_errno: i32 = 0, + /// Owning pid. `fork()` hands the child a copy of the socket, and two + /// processes taking turns on one connection with the tag always zero is a + /// corruption bug that only shows up under load. Checked per request. + owner_pid: i32 = 0, + lock: Lock = .{}, + /// Recycled fid numbers. + /// + /// A counter that only ever goes up is fine for a spike and wrong for a + /// supervisor that outlives a `cp -r`: every walk takes a fid and every + /// clunk gives one back, so without recycling a long run climbs towards + /// `P9_NOFID` while the server holds nothing. The free list is fixed-size + /// on purpose — this struct is embedded in a process that must not depend + /// on an allocator — and overflowing it costs a fid number, not + /// correctness. + free: [256]u32 = undefined, + free_len: usize = 0, + buf: [MSIZE]u8 = undefined, + + pub fn connect(self: *Client, path: []const u8) Error!void { + const fd = syscall3(SYS_socket, AF_UNIX, SOCK_STREAM, 0); + if (fd < 0) return Error.Disconnected; + // `struct sockaddr_un`: family[2] path[108]. + var addr: [110]u8 = @splat(0); + addr[0] = AF_UNIX; + addr[1] = 0; + if (path.len >= 107) return Error.Disconnected; + @memcpy(addr[2 .. 2 + path.len], path); + if (syscall3(SYS_connect, @intCast(fd), @intFromPtr(&addr), 2 + path.len + 1) < 0) { + _ = syscall3(SYS_close, @intCast(fd), 0, 0); + return Error.Disconnected; + } + self.fd = @intCast(fd); + self.owner_pid = @intCast(syscall0(SYS_getpid)); + self.next_fid = 1; + self.root_fid = 0; + try self.version(); + try self.attach(); + } + + fn writeAll(self: *Client, bytes: []const u8) Error!void { + var off: usize = 0; + while (off < bytes.len) { + const n = syscall3(SYS_write, @intCast(self.fd), @intFromPtr(bytes.ptr) + off, bytes.len - off); + if (n <= 0) return Error.Disconnected; + off += @intCast(n); + } + } + + fn readAll(self: *Client, into: []u8) Error!void { + var off: usize = 0; + while (off < into.len) { + const n = syscall3(SYS_read, @intCast(self.fd), @intFromPtr(into.ptr) + off, into.len - off); + if (n <= 0) return Error.Disconnected; + off += @intCast(n); + } + } + + /// Send what the writer built, then read one whole reply into `buf`. + /// Returns a reader positioned just past the header, and the reply type. + fn roundTrip(self: *Client, w: *Writer, ttype: u8) Error!struct { Reader, u8 } { + const size: u32 = @intCast(w.at); + self.buf[0] = @truncate(size); + self.buf[1] = @truncate(size >> 8); + self.buf[2] = @truncate(size >> 16); + self.buf[3] = @truncate(size >> 24); + self.buf[4] = ttype; + self.buf[5] = 0; + self.buf[6] = 0; + try self.writeAll(self.buf[0..w.at]); + + try self.readAll(self.buf[0..4]); + var len: u32 = 0; + var i: usize = 0; + while (i < 4) : (i += 1) len |= @as(u32, self.buf[i]) << @intCast(i * 8); + if (len < HDR or len > self.buf.len) return Error.Protocol; + try self.readAll(self.buf[4..len]); + const rtype = self.buf[4]; + var r = Reader{ .buf = self.buf[0..len], .at = HDR }; + if (rtype == P9_RLERROR) { + self.last_errno = @intCast(try r.u32v()); + return Error.Remote; + } + // Every reply to `T` is `T + 1`; anything else is a desynced stream. + if (rtype != ttype + 1) return Error.Protocol; + return .{ r, rtype }; + } + + fn version(self: *Client) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(MSIZE); + w.str("9P2000.L"); + var got = try self.roundTrip(&w, P9_TVERSION); + const negotiated = try got[0].u32v(); + const ver = try got[0].str(); + if (ver.len != 8 or ver[2] != '2') return Error.Protocol; + self.msize = if (negotiated < MSIZE) negotiated else MSIZE; + } + + fn attach(self: *Client) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(self.root_fid); + w.u32v(P9_NOFID); + w.str("mountx"); + w.str(""); + w.u32v(0xffff_ffff); + _ = try self.roundTrip(&w, P9_TATTACH); + } + + pub fn allocFid(self: *Client) u32 { + if (self.free_len > 0) { + self.free_len -= 1; + return self.free[self.free_len]; + } + self.next_fid += 1; + return self.next_fid; + } + + /// Hand a fid number back, once the server has forgotten it. + /// + /// Only ever called after a `Tclunk`, because a number recycled while the + /// server still holds it comes back as `EINVAL: fid already in use`. + pub fn freeFid(self: *Client, fid: u32) void { + if (self.free_len < self.free.len) { + self.free[self.free_len] = fid; + self.free_len += 1; + } + } + + /// One `Twalk` of at most `P9_MAXWELEM` names, `from` → `newfid`. + /// + /// Reports a *partial* walk as `ENOENT` rather than as a short reply: this + /// client asked for a path, not a prefix, and `src/9p/session.ts` answers + /// `Rwalk` with fewer qids rather than an error because that is the + /// protocol's rule. `walked` is how many names actually resolved, which is + /// what makes a caller able to tell "the first name failed" from "the + /// fourth did" without a second round trip. + pub fn walkOnce( + self: *Client, + from: u32, + newfid: u32, + path: []const u8, + out_qid: ?*Qid, + out_walked: ?*u16, + ) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(from); + w.u32v(newfid); + const count_at = w.at; + w.u16v(0); + var n: u16 = 0; + var it = Split{ .s = path }; + while (it.next()) |part| { + if (n >= P9_MAXWELEM) return Error.Protocol; + w.str(part); + n += 1; + } + self.buf[count_at] = @truncate(n); + self.buf[count_at + 1] = @truncate(n >> 8); + var got = try self.roundTrip(&w, P9_TWALK); + const nwqid = try got[0].u16v(); + if (out_walked) |slot| slot.* = nwqid; + var last: Qid = .{ .qtype = P9_QTDIR, .version = 0, .path = 0 }; + var i: u16 = 0; + while (i < nwqid) : (i += 1) last = try Qid.read(&got[0]); + if (nwqid != n) { + self.last_errno = 2; // ENOENT + return Error.Remote; + } + if (out_qid) |slot| slot.* = last; + } + + /// Walk `path` (slash-separated, relative to the attach root) onto a fresh + /// fid, in `P9_MAXWELEM`-sized steps. A zero-element walk clones the root + /// fid, which is how the root itself is reached. + /// + /// The walk continues *in place* (`newfid == fid`) after the first step, + /// which the protocol allows and the session treats as a clone onto + /// itself — so a path of any depth costs one fid rather than one per 16 + /// components. + pub fn walk(self: *Client, path: []const u8, out_qid: ?*Qid) Error!u32 { + const newfid = self.allocFid(); + errdefer self.freeFid(newfid); + var from = self.root_fid; + var it = Split{ .s = path }; + var chunk: [P9_MAXWELEM]([]const u8) = undefined; + var n: usize = 0; + var any = false; + var qid: Qid = .{ .qtype = P9_QTDIR, .version = 0, .path = 0 }; + while (true) { + const part = it.next(); + if (part) |p| { + chunk[n] = p; + n += 1; + } + if (n == 0) break; + if (part != null and n < P9_MAXWELEM) continue; + // `chunk` holds slices of `path`, and `walkOnce` writes them into + // the client buffer as it goes, so the join has to happen here + // rather than by handing `walkOnce` a second path string. + var joined: [4096]u8 = undefined; + var at: usize = 0; + for (chunk[0..n]) |p| { + if (at != 0) { + joined[at] = '/'; + at += 1; + } + if (at + p.len > joined.len) return Error.Protocol; + @memcpy(joined[at .. at + p.len], p); + at += p.len; + } + try self.walkOnce(from, newfid, joined[0..at], &qid, null); + any = true; + from = newfid; + n = 0; + if (part == null) break; + } + if (!any) { + // The root itself: a zero-element walk, which clones the fid and + // answers no qids at all — so the qid, if anybody wants one, has to + // come from `Tgetattr` instead. + try self.walkOnce(self.root_fid, newfid, "", null, null); + if (out_qid != null) qid = try self.getattrQid(newfid); + } + if (out_qid) |slot| slot.* = qid; + return newfid; + } + + /// The qid of what a fid names, for the one caller that has no walk step to + /// take it from. + fn getattrQid(self: *Client, fid: u32) Error!Qid { + const a = try self.getattr(fid); + return a.qid; + } + + pub fn clunk(self: *Client, fid: u32) void { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + // Recycled only on success: a fid the server still holds comes back as + // `EINVAL: fid already in use` on the next walk that draws it. + if (self.roundTrip(&w, P9_TCLUNK)) |_| self.freeFid(fid) else |_| {} + } + + pub fn getattr(self: *Client, fid: u32) Error!Attr { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u64v(P9_GETATTR_BASIC); + const got = try self.roundTrip(&w, P9_TGETATTR); + var r = got[0]; + _ = try r.u64v(); // valid + const qid = try Qid.read(&r); + const mode = try r.u32v(); + const uid = try r.u32v(); + const gid = try r.u32v(); + const nlink = try r.u64v(); + const rdev = try r.u64v(); + const size = try r.u64v(); + const blksize = try r.u64v(); + const blocks = try r.u64v(); + const atime_sec = try r.u64v(); + const atime_nsec = try r.u64v(); + const mtime_sec = try r.u64v(); + const mtime_nsec = try r.u64v(); + const ctime_sec = try r.u64v(); + const ctime_nsec = try r.u64v(); + return .{ + .qid = qid, + .mode = mode, + .uid = uid, + .gid = gid, + .nlink = nlink, + .rdev = rdev, + .size = size, + .blksize = blksize, + .blocks = blocks, + .atime_sec = atime_sec, + .atime_nsec = atime_nsec, + .mtime_sec = mtime_sec, + .mtime_nsec = mtime_nsec, + .ctime_sec = ctime_sec, + .ctime_nsec = ctime_nsec, + }; + } + + pub fn lopen(self: *Client, fid: u32, flags: u32) Error!u32 { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u32v(flags); + var got = try self.roundTrip(&w, P9_TLOPEN); + _ = try Qid.read(&got[0]); + return try got[0].u32v(); + } + + pub fn lcreate(self: *Client, dir_fid: u32, name: []const u8, flags: u32, mode: u32) Error!u32 { + var w = Writer{ .buf = &self.buf }; + w.u32v(dir_fid); + w.str(name); + w.u32v(flags); + w.u32v(mode); + w.u32v(0); + var got = try self.roundTrip(&w, P9_TLCREATE); + _ = try Qid.read(&got[0]); + return try got[0].u32v(); + } + + /// One `Tread`, capped at whatever the negotiated `msize` leaves room for. + pub fn read(self: *Client, fid: u32, offset: u64, into: []u8) Error!usize { + const room = self.msize - HDR - 4; + const want: u32 = if (into.len > room) room else @intCast(into.len); + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u64v(offset); + w.u32v(want); + var got = try self.roundTrip(&w, P9_TREAD); + const count = try got[0].u32v(); + if (count > into.len) return Error.Protocol; + try got[0].skip(count); + @memcpy(into[0..count], self.buf[HDR + 4 .. HDR + 4 + count]); + return count; + } + + pub fn write(self: *Client, fid: u32, offset: u64, data: []const u8) Error!usize { + const room = self.msize - HDR - 4 - 4 - 8; + const want: usize = if (data.len > room) room else data.len; + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u64v(offset); + w.u32v(@intCast(want)); + @memcpy(self.buf[w.at .. w.at + want], data[0..want]); + w.at += want; + var got = try self.roundTrip(&w, P9_TWRITE); + return try got[0].u32v(); + } + + /// One `Treaddir` block, copied out whole. The caller unpacks it — the + /// entry format is `qid[13] offset[8] type[1] name[s]`, per + /// `src/9p/protocol.ts`'s `writeDirent`. + pub fn readdir(self: *Client, fid: u32, offset: u64, into: []u8) Error!usize { + const room = self.msize - HDR - 4; + const want: u32 = if (into.len > room) room else @intCast(into.len); + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u64v(offset); + w.u32v(want); + var got = try self.roundTrip(&w, P9_TREADDIR); + const count = try got[0].u32v(); + if (count > into.len) return Error.Protocol; + @memcpy(into[0..count], self.buf[HDR + 4 .. HDR + 4 + count]); + return count; + } + + // ----------------------------------------------------------------------- + // Namespace mutation and metadata + // + // Every message below is transcribed from `src/9p/protocol.ts` — field + // order, field width and all — and the reply of each is either a bare + // header or a qid this client does not need, which is why most of them + // answer `void`. + // ----------------------------------------------------------------------- + + /// `Tsetattr` — the one message behind `chmod`, `chown`, `truncate` and + /// `utimensat` alike. `valid` says which of the fields that follow mean + /// anything; there is no ctime *value*, only a bit asking for "now". + pub fn setattr( + self: *Client, + fid: u32, + valid: u32, + mode: u32, + uid: u32, + gid: u32, + size: u64, + atime_sec: u64, + atime_nsec: u64, + mtime_sec: u64, + mtime_nsec: u64, + ) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u32v(valid); + w.u32v(mode); + w.u32v(uid); + w.u32v(gid); + w.u64v(size); + w.u64v(atime_sec); + w.u64v(atime_nsec); + w.u64v(mtime_sec); + w.u64v(mtime_nsec); + _ = try self.roundTrip(&w, P9_TSETATTR); + } + + pub fn mkdir(self: *Client, dfid: u32, name: []const u8, mode: u32, gid: u32) Error!Qid { + var w = Writer{ .buf = &self.buf }; + w.u32v(dfid); + w.str(name); + w.u32v(mode); + w.u32v(gid); + var got = try self.roundTrip(&w, P9_TMKDIR); + return try Qid.read(&got[0]); + } + + /// `Tunlinkat` — `flags` carries exactly one bit, `P9_DOTL_AT_REMOVEDIR`: + /// without it this is `unlink`, with it `rmdir`. + pub fn unlinkat(self: *Client, dfid: u32, name: []const u8, flags: u32) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(dfid); + w.str(name); + w.u32v(flags); + _ = try self.roundTrip(&w, P9_TUNLINKAT); + } + + pub fn renameat( + self: *Client, + olddirfid: u32, + oldname: []const u8, + newdirfid: u32, + newname: []const u8, + ) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(olddirfid); + w.str(oldname); + w.u32v(newdirfid); + w.str(newname); + _ = try self.roundTrip(&w, P9_TRENAMEAT); + } + + pub fn symlink( + self: *Client, + dfid: u32, + name: []const u8, + target: []const u8, + gid: u32, + ) Error!Qid { + var w = Writer{ .buf = &self.buf }; + w.u32v(dfid); + w.str(name); + w.str(target); + w.u32v(gid); + var got = try self.roundTrip(&w, P9_TSYMLINK); + return try Qid.read(&got[0]); + } + + /// `Tlink` — note the order: the *directory* comes first here and second in + /// `Trename`, which is real in `p9_client_link()` and pinned by byte + /// fixtures on the TypeScript side. + pub fn link(self: *Client, dfid: u32, fid: u32, name: []const u8) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(dfid); + w.u32v(fid); + w.str(name); + _ = try self.roundTrip(&w, P9_TLINK); + } + + pub fn mknod( + self: *Client, + dfid: u32, + name: []const u8, + mode: u32, + major: u32, + minor: u32, + gid: u32, + ) Error!Qid { + var w = Writer{ .buf = &self.buf }; + w.u32v(dfid); + w.str(name); + w.u32v(mode); + w.u32v(major); + w.u32v(minor); + w.u32v(gid); + var got = try self.roundTrip(&w, P9_TMKNOD); + return try Qid.read(&got[0]); + } + + /// `Treadlink` — the target, copied into `into`. Never resolved: what the + /// link holds is what comes back. + pub fn readlink(self: *Client, fid: u32, into: []u8) Error![]const u8 { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + var got = try self.roundTrip(&w, P9_TREADLINK); + const target = try got[0].str(); + if (target.len > into.len) return Error.Protocol; + @memcpy(into[0..target.len], target); + return into[0..target.len]; + } + + /// `Tfsync` — `datasync` non-zero asks for `fdatasync(2)` rather than + /// `fsync(2)`. + pub fn fsync(self: *Client, fid: u32, datasync: u32) Error!void { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + w.u32v(datasync); + _ = try self.roundTrip(&w, P9_TFSYNC); + } + + /// `Rstatfs`, in the field order `src/9p/protocol.ts`'s `writeRstatfs` uses. + pub const Statfs = struct { + ftype: u32, + bsize: u32, + blocks: u64, + bfree: u64, + bavail: u64, + files: u64, + ffree: u64, + fsid: u64, + namelen: u32, + }; + + pub fn statfs(self: *Client, fid: u32) Error!Statfs { + var w = Writer{ .buf = &self.buf }; + w.u32v(fid); + const got = try self.roundTrip(&w, P9_TSTATFS); + var r = got[0]; + return .{ + .ftype = try r.u32v(), + .bsize = try r.u32v(), + .blocks = try r.u64v(), + .bfree = try r.u64v(), + .bavail = try r.u64v(), + .files = try r.u64v(), + .ffree = try r.u64v(), + .fsid = try r.u64v(), + .namelen = try r.u32v(), + }; + } + + /// True when this process is not the one that opened the socket, i.e. we + /// are in a `fork()`ed child holding a copy of somebody else's connection. + pub fn forked(self: *Client) bool { + return self.fd >= 0 and self.owner_pid != @as(i32, @intCast(syscall0(SYS_getpid))); + } + + pub fn reset(self: *Client) void { + if (self.fd >= 0) _ = syscall3(SYS_close, @intCast(self.fd), 0, 0); + self.fd = -1; + } +}; + +/// Slash-separated path components, skipping empties so `/a//b/` walks `a`,`b`. +pub const Split = struct { + s: []const u8, + at: usize = 0, + + pub fn next(self: *Split) ?[]const u8 { + while (self.at < self.s.len and self.s[self.at] == '/') self.at += 1; + if (self.at >= self.s.len) return null; + const start = self.at; + while (self.at < self.s.len and self.s[self.at] != '/') self.at += 1; + return self.s[start..self.at]; + } +}; + +/// A spinlock rather than a pthread mutex: the shim must not depend on libc +/// state that may not be initialised yet when the first interposed call +/// arrives, and contention here is a single-digit-microsecond round trip. +pub const Lock = struct { + flag: @import("std").atomic.Value(bool) = .init(false), + + pub fn acquire(self: *Lock) void { + while (self.flag.cmpxchgWeak(false, true, .acquire, .monotonic) != null) { + asm volatile ("pause"); + } + } + pub fn release(self: *Lock) void { + self.flag.store(false, .release); + } +}; diff --git a/src/exec/seccomp/state.zig b/src/exec/seccomp/state.zig new file mode 100644 index 0000000..46cda25 --- /dev/null +++ b/src/exec/seccomp/state.zig @@ -0,0 +1,357 @@ +//! Everything the supervisor remembers between notifications: open file +//! descriptions, which descriptor in which process names which one, and where +//! each process thinks it is. +//! +//! Three properties this file exists to keep, all of them things the spike got +//! wrong or did not attempt: +//! +//! - **A descriptor is not an open file.** `dup`, `fork` and a shell +//! redirection all give one open file description several `(pid, fd)` names, +//! and POSIX says they share a file offset. So the offset lives on the +//! {@link File}, never on the name, and the names are reference counts on it. +//! - **The tables grow.** The spike had a fixed 256-entry array and silently +//! dropped the 257th descriptor. These are `ArrayList`s: the supervisor is a +//! separate process from the tracee now, carries no filter of its own, and may +//! allocate freely. +//! - **Identity survives what the supervisor cannot see.** A `dup` gives a new +//! fd number without any syscall the filter traps usefully — a notification +//! cannot observe the number the kernel is about to return — so identity comes +//! from the *object*: every injected descriptor is a `memfd` with a unique +//! name, and `/proc//fd/` reads back as `/memfd:mx-` for the +//! original and every duplicate alike. {@link Tables.lookup} memoises the walk. + +const std = @import("std"); + +/// One open file description: what `open(2)` created and `dup(2)` shares. +pub const File = struct { + used: bool = false, + /// Stamped into the placeholder `memfd`'s name, so a descriptor can be + /// identified by what it points at rather than by the number it was given. + serial: u32 = 0, + /// The 9P fid, live while `open` is true. + fid: u32 = 0, + open: bool = false, + /// How many `(pid, fd)` names point here. + refs: u32 = 0, + is_dir: bool = false, + /// The `O_*` the tracee opened with, in the kernel's namespace. + flags: u32 = 0, + /// The shared file offset. Shared, because `dup` shares it. + offset: u64 = 0, + /// `Treaddir`'s cursor, which is a cookie the server mints rather than a + /// byte position. + cookie: u64 = 0, + /// Path under the root, without a leading slash. Owned. + path: []u8 = &.{}, +}; + +/// One name for a {@link File}: a descriptor number in a process. +const Name = struct { + used: bool = false, + /// The **thread group**, not the notification's thread id — descriptors + /// belong to the process. + pid: i32 = 0, + fd: i32 = 0, + file: u32 = 0, +}; + +/// Where a process thinks it is, when that is somewhere in the virtual tree. +const Cwd = struct { + used: bool = false, + pid: i32 = 0, + /// Path under the root, without a leading slash. Owned. + path: []u8 = &.{}, +}; + +pub const Tables = struct { + allocator: std.mem.Allocator, + files: std.ArrayList(File) = .empty, + names: std.ArrayList(Name) = .empty, + cwds: std.ArrayList(Cwd) = .empty, + /// Files whose last name has just gone, waiting for somebody who can speak + /// 9P to clunk their fid. Releasing is not table work, and a table that + /// tried to do it would need a client. + orphans: std.ArrayList(u32) = .empty, + next_serial: u32 = 1, + + pub fn init(allocator: std.mem.Allocator) Tables { + return .{ .allocator = allocator }; + } + + pub fn file(self: *Tables, index: u32) *File { + return &self.files.items[index]; + } + + /// A fresh open file description. The caller fills in the 9P fid and the + /// placeholder descriptor; this hands back the index and the serial. + pub fn create(self: *Tables, path: []const u8) !u32 { + const owned = try self.allocator.dupe(u8, path); + errdefer self.allocator.free(owned); + const serial = self.next_serial; + self.next_serial += 1; + for (self.files.items, 0..) |*slot, index| { + if (!slot.used) { + slot.* = .{ .used = true, .serial = serial, .path = owned }; + return @intCast(index); + } + } + try self.files.append(self.allocator, .{ .used = true, .serial = serial, .path = owned }); + return @intCast(self.files.items.len - 1); + } + + /// Point `(pid, fd)` at `index`, evicting whatever used to have that name. + /// + /// Evicting first is not tidiness. The kernel reuses the lowest free + /// descriptor number immediately, so without it a stale mapping shadows a + /// live one: an `fstat` on a freshly opened file answered from the + /// *directory* that previously held the number, and `cp -r` correctly + /// concluded the file had been replaced underneath it. Witnessed in the + /// spike, where `close` was untrapped and this was the only defence. + pub fn bind(self: *Tables, pid: i32, fd: i32, index: u32) !void { + // A live number can be replaced without a `close` this supervisor sees: + // `dup2(fd, 5)` closes 5 implicitly. So binding evicts, and whatever + // that orphaned goes on the queue like any other release. + self.unbind(pid, fd); + self.files.items[index].refs += 1; + for (self.names.items) |*slot| { + if (!slot.used) { + slot.* = .{ .used = true, .pid = pid, .fd = fd, .file = index }; + return; + } + } + self.names.append(self.allocator, .{ + .used = true, + .pid = pid, + .fd = fd, + .file = index, + }) catch |err| { + self.files.items[index].refs -= 1; + return err; + }; + } + + /// Forget one name, queueing the file for release if that was its last. + pub fn unbind(self: *Tables, pid: i32, fd: i32) void { + for (self.names.items) |*slot| { + if (slot.used and slot.pid == pid and slot.fd == fd) { + const index = slot.file; + slot.* = .{}; + self.dropRef(index); + return; + } + } + } + + /// Every name a process held, for the `exit_group` sweep. + pub fn unbindAll(self: *Tables, pid: i32) void { + for (self.names.items) |*slot| { + if (slot.used and slot.pid == pid) { + const index = slot.file; + slot.* = .{}; + self.dropRef(index); + } + } + } + + /// Every name in a descriptor range, for `close_range(2)`. + pub fn unbindRange(self: *Tables, pid: i32, first: i32, last: i32) void { + for (self.names.items) |*slot| { + if (slot.used and slot.pid == pid and slot.fd >= first and slot.fd <= last) { + const index = slot.file; + slot.* = .{}; + self.dropRef(index); + } + } + } + + fn dropRef(self: *Tables, index: u32) void { + const entry = &self.files.items[index]; + if (entry.refs > 0) entry.refs -= 1; + if (entry.refs == 0) self.orphans.append(self.allocator, index) catch {}; + } + + /// The queued releases, handed over and cleared. + pub fn takeOrphans(self: *Tables) []const u32 { + return self.orphans.items; + } + + pub fn clearOrphans(self: *Tables) void { + self.orphans.clearRetainingCapacity(); + } + + // There is deliberately no way to *destroy* a file record. Its 9P fid is + // released the moment nothing names it (`release()` in `trace.zig`), which + // is the resource that matters — a driver handle held open. The record + // itself is kept, because a descriptor this supervisor never saw bound may + // still resolve to it through `/proc` and need it re-opened, and answering + // that from an empty placeholder instead is exactly the silent wrong answer + // this design exists to remove. What it costs is a path and a struct per + // distinct open, for the length of one command. + + /// Which file is `(pid, fd)`, by name and then by object identity. + /// + /// The second half is what makes `dup` work without trapping it, and it is + /// deliberately **not memoised**. Caching the answer under the new number + /// looked like an obvious win and is a correctness bug: a shell running + /// `echo hi > $ROOT/file` duplicates the virtual descriptor onto its + /// standard output, and restores the real one afterwards with a `dup2` this + /// supervisor cannot see as a replacement. A cached entry then claimed + /// standard output for the rest of the run and every later `echo` vanished + /// into the driver. Witnessed. Re-reading the link costs one `readlink` + /// against an operation that is about to cost a 9P round trip, and it + /// cannot go stale. + /// + /// The duplicate resolves to the *same* file, so it shares its offset — + /// which is what `dup` is supposed to do and what the `LD_PRELOAD` spike + /// could not manage at all. + pub fn lookup(self: *Tables, pid: i32, fd: i32, link: ?[]const u8) ?u32 { + for (self.names.items) |slot| { + if (slot.used and slot.pid == pid and slot.fd == fd) return slot.file; + } + if (fd < 0) return null; + const seen = link orelse return null; + const prefix = "/memfd:mx-"; + if (!std.mem.startsWith(u8, seen, prefix)) return null; + var serial: u32 = 0; + for (seen[prefix.len..]) |ch| { + if (ch < '0' or ch > '9') break; + serial = serial * 10 + (ch - '0'); + } + for (self.files.items, 0..) |entry, index| { + if (entry.used and entry.serial == serial) return @intCast(index); + } + return null; + } + + /// Rewrite a file's remembered path, after a rename moved it. + pub fn repath(self: *Tables, index: u32, path: []const u8) !void { + const owned = try self.allocator.dupe(u8, path); + self.allocator.free(self.files.items[index].path); + self.files.items[index].path = owned; + } + + // ----------------------------------------------------------------------- + // The virtual working directory + // ----------------------------------------------------------------------- + + /// Where this process is inside the tree, or null when it is outside it. + pub fn cwd(self: *Tables, pid: i32) ?[]const u8 { + for (self.cwds.items) |slot| { + if (slot.used and slot.pid == pid) return slot.path; + } + return null; + } + + pub fn setCwd(self: *Tables, pid: i32, path: []const u8) !void { + const owned = try self.allocator.dupe(u8, path); + for (self.cwds.items) |*slot| { + if (slot.used and slot.pid == pid) { + self.allocator.free(slot.path); + slot.path = owned; + return; + } + } + for (self.cwds.items) |*slot| { + if (!slot.used) { + slot.* = .{ .used = true, .pid = pid, .path = owned }; + return; + } + } + self.cwds.append(self.allocator, .{ .used = true, .pid = pid, .path = owned }) catch |err| { + self.allocator.free(owned); + return err; + }; + } + + pub fn clearCwd(self: *Tables, pid: i32) void { + for (self.cwds.items) |*slot| { + if (slot.used and slot.pid == pid) { + self.allocator.free(slot.path); + slot.* = .{}; + } + } + } + + /// A process that has never been seen inherits its parent's virtual cwd, + /// which is what `fork` does with a working directory. The notification + /// carries no parent, so it comes from `/proc//status` — the same read + /// that answers "which thread group is this". + pub fn inheritCwd(self: *Tables, pid: i32, ppid: i32) !bool { + if (self.cwd(pid) != null) return true; + const parent = self.cwd(ppid) orelse return false; + // `parent` points into the table this call is about to grow, so it is + // copied before anything can move it. + var buf: [4096]u8 = undefined; + if (parent.len > buf.len) return false; + @memcpy(buf[0..parent.len], parent); + try self.setCwd(pid, buf[0..parent.len]); + return true; + } +}; + +// --------------------------------------------------------------------------- +// Paths +// --------------------------------------------------------------------------- + +/// Slash-separated components, skipping empties so `/a//b/` yields `a`, `b`. +pub const Split = struct { + s: []const u8, + at: usize = 0, + + pub fn next(self: *Split) ?[]const u8 { + while (self.at < self.s.len and self.s[self.at] == '/') self.at += 1; + if (self.at >= self.s.len) return null; + const start = self.at; + while (self.at < self.s.len and self.s[self.at] != '/') self.at += 1; + return self.s[start..self.at]; + } +}; + +/// `parts` joined, with `.` dropped and `..` popped — **clamped at the root**, +/// the same rule `src/path.ts` holds every transport to. The result has no +/// leading slash: `""` is the root and `"a/b"` is a file in it. +/// +/// Several parts rather than two because a symlink resolution joins three: the +/// directory the link lives in, the link's own target, and the components of +/// the original path that came after it. +pub fn normalize(out: []u8, parts: []const []const u8) ?[]const u8 { + var len: usize = 0; + var starts: [256]usize = undefined; + var depth: usize = 0; + for (parts) |part| { + var it = Split{ .s = part }; + while (it.next()) |name| { + if (name.len == 1 and name[0] == '.') continue; + if (name.len == 2 and name[0] == '.' and name[1] == '.') { + if (depth > 0) { + depth -= 1; + len = starts[depth]; + if (len > 0) len -= 1; + } + continue; + } + if (depth >= starts.len) return null; + if (len != 0) { + if (len >= out.len) return null; + out[len] = '/'; + len += 1; + } + starts[depth] = len; + depth += 1; + if (len + name.len > out.len) return null; + @memcpy(out[len .. len + name.len], name); + len += name.len; + } + } + return out[0..len]; +} + +/// The parent and the final component of a normalized path. Null for the root, +/// which has neither. +pub fn splitParent(path: []const u8) ?struct { []const u8, []const u8 } { + if (path.len == 0) return null; + if (std.mem.lastIndexOfScalar(u8, path, '/')) |at| { + return .{ path[0..at], path[at + 1 ..] }; + } + return .{ "", path }; +} diff --git a/src/exec/seccomp/trace.zig b/src/exec/seccomp/trace.zig new file mode 100644 index 0000000..23a1e4a --- /dev/null +++ b/src/exec/seccomp/trace.zig @@ -0,0 +1,1912 @@ +//! `mountx-trace` — a seccomp user-notification supervisor over 9P. +//! +//! A BPF filter selects the filesystem syscalls and answers +//! `SECCOMP_RET_USER_NOTIF` for them; every other syscall the traced process +//! makes runs at full native speed, never leaving the kernel. This process +//! reads each trapped call off a listener fd, answers it out of a 9P client +//! against an unmodified `createP9Server()`, and sends the result back. +//! +//! Usage: `mountx-trace <9p-socket> -- [args...]` +//! +//! **Why this mechanism.** The boundary is the syscall ABI, not glibc's +//! exported symbols, so it does not care what the traced program is linked +//! against or whether it is linked at all: a static musl binary and a +//! no-libc raw-syscall binary are indistinguishable from `cat` here. And the +//! surface is *closed* — a finite set fixed by the kernel ABI, rather than one +//! that grows with each libc release. +//! +//! **What it is not.** There is no filesystem in this file. Every path +//! question — resolution, handle lifetimes, directory paging, error mapping — +//! is answered on the far side of the socket by `src/9p/session.ts`, which +//! already passes a conformance column. This is a wire adapter with a +//! descriptor table. +//! +//! ### The three things this owes the tracee +//! +//! 1. **Exactly one reply per notification.** Every handler returns a +//! {@link Reply} and the loop sends it; no handler talks to the listener +//! itself, so "answered twice" and "never answered" are both unrepresentable +//! rather than merely avoided. A thrown 9P error becomes the errno the +//! server put in its `Rlerror`, and anything else becomes `EIO`. +//! 2. **No silent success.** The spike answered `openat` by slurping the file +//! into a `memfd` and injecting it, which made `write` land in a private +//! copy that was then discarded — `dd conv=notrunc` reported success and +//! changed nothing. Nothing here is backed by a `memfd`'s contents: the +//! injected descriptor is an *empty* placeholder whose only job is to be a +//! number the kernel agrees exists, and `read`/`write`/`lseek` against it +//! are trapped and answered from the driver. Any operation on it that this +//! supervisor does not implement fails loudly against an empty file rather +//! than quietly succeeding against a stale one. +//! 3. **TOCTOU discipline.** `SECCOMP_IOCTL_NOTIF_ID_VALID` is checked *after* +//! reading tracee memory and before acting on it, because a dead tracee's +//! pid can be reused between the notification and the read. +//! +//! ### Known gaps, stated rather than hidden +//! +//! - **`mmap` of a file on the tree is refused with `ENODEV`.** The spike got +//! working `mmap` for free because the injected descriptor really was a +//! `memfd` holding the file; without the copy there is nothing for the kernel +//! to map, and a `MAP_SHARED` mapping would have no way back to the driver. +//! Refusing is the honest answer — a mapping of an empty placeholder would +//! read as zeroes and write into nothing. +//! - **`execve` of a binary living on the tree does not work** and is not +//! attempted: the kernel resolves the ELF itself, with no notification. It +//! fails with `ENOENT` from the real filesystem, which is loud. +//! - **`sendfile`, `splice`, `copy_file_range` and `fallocate`** are refused on +//! a virtual descriptor, with the errno that makes callers fall back to +//! `read`/`write` rather than one that makes them give up. +//! - **An absolute symlink target is resolved against the virtual root**, not +//! against the host's. The tree behaves like a chroot for its own links. +//! - **Extended attributes answer `ENOTSUP`**, which is what the 9P transport +//! answers for `Txattrwalk` anyway. +//! - **A relative `execve` under a virtual working directory** resolves against +//! the real one, because `chdir` into the tree is answered without the kernel +//! ever changing this process's idea of where it is. + +const std = @import("std"); +const p9 = @import("p9"); +const linux = @import("linux.zig"); +const notify = @import("notify.zig"); +const state = @import("state.zig"); + +const c = @cImport({ + @cDefine("_GNU_SOURCE", "1"); + @cInclude("unistd.h"); + @cInclude("errno.h"); + @cInclude("stdlib.h"); + @cInclude("fcntl.h"); + @cInclude("sys/wait.h"); +}); + +const SYS = linux.SYS; + +/// The trapped set. +/// +/// Read it as three groups. The **path** calls are here because they name +/// something that may live on the tree. The **descriptor** calls are here +/// because the descriptor this supervisor hands out is an empty placeholder — +/// leaving `read` untrapped would answer zero bytes and call it success, which +/// is the exact failure mode this file exists to remove. And `close` is here +/// because descriptor numbers are reused immediately, so a mapping that outlives +/// its descriptor shadows a live one. +/// +/// `close` is the one that was impossible in the spike: the supervisor shared +/// its filter with the tracee and would have suspended itself. See `notify.zig`. +const TRAPPED = [_]u32{ + // descriptors + SYS.read, SYS.write, SYS.close, SYS.lseek, + SYS.pread64, SYS.pwrite64, SYS.readv, SYS.writev, + SYS.preadv, SYS.pwritev, SYS.preadv2, SYS.pwritev2, + SYS.mmap, SYS.getdents64, SYS.fstat, SYS.fsync, + SYS.fdatasync, SYS.ftruncate, SYS.fchmod, SYS.fchown, + SYS.fstatfs, SYS.sendfile, SYS.copy_file_range, SYS.splice, + SYS.fallocate, SYS.dup2, SYS.dup3, SYS.close_range, + // opening + SYS.open, SYS.openat, SYS.openat2, SYS.creat, + // metadata by path + SYS.stat, SYS.lstat, SYS.newfstatat, SYS.statx, + SYS.access, SYS.faccessat, SYS.faccessat2, SYS.statfs, + SYS.truncate, SYS.chmod, SYS.fchmodat, SYS.chown, + SYS.lchown, SYS.fchownat, SYS.utime, SYS.utimes, + SYS.futimesat, SYS.utimensat, + // the namespace + SYS.mkdir, SYS.mkdirat, + SYS.rmdir, SYS.unlink, SYS.unlinkat, SYS.rename, + SYS.renameat, SYS.renameat2, SYS.link, SYS.linkat, + SYS.symlink, SYS.symlinkat, SYS.readlink, SYS.readlinkat, + SYS.mknod, SYS.mknodat, + // where a process thinks it is + SYS.getcwd, SYS.chdir, + SYS.fchdir, + // extended attributes, refused rather than left to the real filesystem + SYS.setxattr, SYS.lsetxattr, SYS.fsetxattr, + SYS.getxattr, SYS.lgetxattr, SYS.fgetxattr, SYS.listxattr, + SYS.llistxattr, SYS.flistxattr, SYS.removexattr, SYS.lremovexattr, + SYS.fremovexattr, + // lifecycle + SYS.exit_group, +}; + +/// The made-up device every file here reports, as a minor number with major 0. +/// +/// Deliberately under 256 so `makedev(0, n) == n`. `statx` reports a +/// major/minor *pair* that glibc recomposes with `makedev()`, and a raw +/// `st_dev` of 0x6d78 against major 0 / minor 0x6d78 recomposes to 0x6d00078 — +/// so `stat` and `statx` disagreed, and `cp` refused with "skipping file … as it +/// was replaced while being copied". Keeping the minor inside 8 bits makes the +/// two forms agree by construction. +const FAKE_DEV_MINOR: u64 = 0x78; + +/// How many links a resolution may traverse before it is a loop. +const MAX_SYMLINKS: u32 = 40; + +var client: p9.Client = .{}; +var root: []const u8 = &.{}; +var tables: state.Tables = undefined; +var debug = false; +var self_uid: u32 = 0; +var self_gid: u32 = 0; + +/// Scratch for file payloads. One notification is in flight at a time, so one +/// buffer is enough; see the concurrency note in `main`. +var scratch: [1 << 20]u8 = undefined; +var pathbuf: [4096]u8 = undefined; +var joinbuf: [4096]u8 = undefined; +var auxbuf: [4096]u8 = undefined; + +fn dbg(comptime fmt: []const u8, args: anytype) void { + if (!debug) return; + var buf: [512]u8 = undefined; + const msg = std.fmt.bufPrint(&buf, "[trace] " ++ fmt ++ "\n", args) catch return; + _ = c.write(2, msg.ptr, msg.len); +} + +fn die(comptime fmt: []const u8, args: anytype) noreturn { + var buf: [512]u8 = undefined; + const msg = std.fmt.bufPrint(&buf, "mountx-trace: " ++ fmt ++ "\n", args) catch "mountx-trace: error\n"; + _ = c.write(2, msg.ptr, msg.len); + c.exit(70); +} + +// --------------------------------------------------------------------------- +// Replies +// --------------------------------------------------------------------------- + +/// What a handler decided. The loop turns exactly one of these into exactly one +/// message on the listener. +const Reply = union(enum) { + /// The syscall succeeded with this return value. + ret: i64, + /// The syscall failed with this (positive) errno. + err: i32, + /// Not ours: let the kernel run it as it stands. + cont: void, + /// The notification is no longer valid — the tracee died under us. Nothing + /// may be sent, and nothing needs to be. + gone: void, +}; + +const ok: Reply = .{ .ret = 0 }; + +fn fail(errno: i32) Reply { + return .{ .err = errno }; +} + +/// A 9P failure, as the errno the server put in its `Rlerror`. +/// +/// 9P is the one transport in this repository with no status-mapping layer: the +/// number in an `Rlerror` is a positive Linux errno straight from +/// `src/errors.ts`, which is exactly what the tracee's `errno` wants. Anything +/// that is not a server error — a desynced stream, a dead socket — is `EIO`, +/// because there is no honest way to blame it on the file. +fn remote(err: p9.Error) Reply { + return switch (err) { + p9.Error.Remote => fail(if (client.last_errno > 0) client.last_errno else linux.EIO), + else => fail(linux.EIO), + }; +} + +// --------------------------------------------------------------------------- +// Which process is asking +// --------------------------------------------------------------------------- + +const TgidEntry = struct { tid: i32, tgid: i32 }; +var tgids: std.ArrayList(TgidEntry) = .empty; +var gpa: std.mem.Allocator = undefined; + +/// The thread group a notification's thread belongs to. +/// +/// `seccomp_notif.pid` is a *thread* id. Descriptors, the working directory and +/// the umask all belong to the thread group, so every table here is keyed on +/// the tgid — otherwise a second thread in the same process would find none of +/// the first one's descriptors. The mapping never changes for a given tid, so +/// it is cached; the first sight of a process is also where it inherits its +/// parent's virtual working directory, the way `fork` does. +fn tgidOf(tid: i32) i32 { + for (tgids.items) |entry| { + if (entry.tid == tid) return entry.tgid; + } + const info = notify.procInfo(tid) orelse return tid; + tgids.append(gpa, .{ .tid = tid, .tgid = info.tgid }) catch {}; + _ = tables.inheritCwd(info.tgid, info.ppid) catch {}; + return info.tgid; +} + +/// The tracee's umask, read fresh: it is process state that changes under us, +/// and applying the wrong one silently creates a file with the wrong mode. +fn umaskOf(tid: i32) u32 { + const info = notify.procInfo(tid) orelse return 0; + return info.umask; +} + +// --------------------------------------------------------------------------- +// Paths +// --------------------------------------------------------------------------- + +/// The part of an absolute path below the root prefix, or null. +fn under(path: []const u8) ?[]const u8 { + if (path.len < root.len) return null; + if (!std.mem.eql(u8, path[0..root.len], root)) return null; + if (path.len == root.len) return path[path.len..]; + if (path[root.len] != '/') return null; + return path[root.len..]; +} + +/// Where a path argument points. +const Target = union(enum) { + /// A normalized path under the root, without a leading slash. + inside: []const u8, + /// Not ours; the kernel should run the syscall unchanged. + outside: void, + /// Ours, and already wrong. + err: i32, +}; + +/// Resolve a path argument the way the kernel would: absolute against the root, +/// relative against a directory descriptor or the working directory. +/// +/// The `*at()` branch is not an optimization — every tree walker resolves each +/// level against the parent's descriptor rather than by name, and without it +/// `find` and `du` report "Not a directory" for every entry of a directory they +/// have just listed. +/// +/// When the process has a *virtual* working directory, a relative path is +/// resolved inside the tree and never falls through to the host: the real +/// working directory was deliberately left where it was (see `handleChdir`), so +/// letting the kernel resolve it would silently mean a different place. +fn resolveAt(pid: i32, dirfd: i32, raw: []const u8, out: []u8) Target { + if (raw.len > 0 and raw[0] == '/') { + const rel = under(raw) orelse return .outside; + const norm = state.normalize(out, &.{rel}) orelse return .{ .err = linux.ENAMETOOLONG }; + return .{ .inside = norm }; + } + if (dirfd == linux.AT_FDCWD) { + const base = tables.cwd(pid) orelse return .outside; + const norm = state.normalize(out, &.{ base, raw }) orelse + return .{ .err = linux.ENAMETOOLONG }; + return .{ .inside = norm }; + } + const index = lookupFd(pid, dirfd) orelse return .outside; + const entry = tables.file(index); + if (!entry.is_dir) return .{ .err = linux.ENOTDIR }; + const norm = state.normalize(out, &.{ entry.path, raw }) orelse + return .{ .err = linux.ENAMETOOLONG }; + return .{ .inside = norm }; +} + +/// `(pid, fd)` → an open file description, following duplicates through +/// `/proc`. See `state.zig` for why identity comes from the object. +fn lookupFd(pid: i32, fd: i32) ?u32 { + if (tables.lookup(pid, fd, null)) |index| return index; + if (fd < 0) return null; + var link: [256]u8 = undefined; + const seen = notify.fdLink(pid, fd, &link) orelse return null; + return tables.lookup(pid, fd, seen); +} + +// --------------------------------------------------------------------------- +// Walking +// --------------------------------------------------------------------------- + +const Walked = struct { fid: u32, qid: p9.Qid }; + +/// Resolve `path` to a fid, one component at a time, following symlinks here +/// rather than on the server. +/// +/// That split is the protocol's: `Twalk` reports a `P9_QTSYMLINK` qid and stops +/// there, because a server that resolved links itself would have no way to +/// answer `lstat`. So the resolution — `Treadlink`, re-root, re-walk, and the +/// loop bound — happens on this side, exactly as the VFS does it above v9fs and +/// exactly as `test/9p/client.ts` does it for the conformance column. +/// +/// The walk proceeds *in place* (`newfid == fid`), so a path of any depth costs +/// one fid rather than one per component. +fn walkTo(path: []const u8, follow: bool, depth: u32) p9.Error!Walked { + if (depth > MAX_SYMLINKS) { + client.last_errno = linux.ELOOP; + return p9.Error.Remote; + } + var parts: [256][]const u8 = undefined; + var count: usize = 0; + var it = state.Split{ .s = path }; + while (it.next()) |name| { + if (count == parts.len) { + client.last_errno = linux.ENAMETOOLONG; + return p9.Error.Remote; + } + parts[count] = name; + count += 1; + } + + // Cleanup is explicit rather than an `errdefer`, and that is not a style + // choice. The symlink branch below clunks the fid *itself* before recursing + // — an `errdefer` then clunks it a second time on the way out of a failed + // resolution, the server answers `EBADF` for a fid it has already + // forgotten, and that errno overwrites the `ENOENT` the caller was about to + // report. A dangling symlink came back as `EBADF`. Witnessed. + const fid = client.allocFid(); + client.walkOnce(client.root_fid, fid, "", null, null) catch |err| { + client.freeFid(fid); + return err; + }; + var qid: p9.Qid = .{ .qtype = p9.P9_QTDIR, .version = 0, .path = 0 }; + if (count == 0) { + const attr = client.getattr(fid) catch |err| { + client.clunk(fid); + return err; + }; + return .{ .fid = fid, .qid = attr.qid }; + } + + // The components consumed so far, which is the directory a relative link + // target is resolved against. + var sofar: [4096]u8 = undefined; + var sofar_len: usize = 0; + for (parts[0..count], 0..) |name, index| { + const before = sofar_len; + client.walkOnce(fid, fid, name, &qid, null) catch |err| { + client.clunk(fid); + return err; + }; + const last = index + 1 == count; + if ((qid.qtype & p9.P9_QTSYMLINK) != 0 and (follow or !last)) { + var target: [4096]u8 = undefined; + const link = client.readlink(fid, &target) catch |err| { + client.clunk(fid); + return err; + }; + client.clunk(fid); + // An absolute target is resolved against the *virtual* root: the + // tree is its own namespace here, and a link out of it has nowhere + // to land, since this supervisor cannot hand the kernel a different + // path than the one it was asked about. + const base: []const u8 = if (link.len > 0 and link[0] == '/') "" else sofar[0..before]; + var rest: [4096]u8 = undefined; + var rest_len: usize = 0; + for (parts[index + 1 .. count]) |tail| { + if (rest_len != 0) { + rest[rest_len] = '/'; + rest_len += 1; + } + if (rest_len + tail.len > rest.len) { + client.last_errno = linux.ENAMETOOLONG; + return p9.Error.Remote; + } + @memcpy(rest[rest_len .. rest_len + tail.len], tail); + rest_len += tail.len; + } + var next: [4096]u8 = undefined; + const joined = state.normalize(&next, &.{ base, link, rest[0..rest_len] }) orelse { + client.last_errno = linux.ENAMETOOLONG; + return p9.Error.Remote; + }; + return walkTo(joined, follow, depth + 1); + } + if (sofar_len != 0) { + sofar[sofar_len] = '/'; + sofar_len += 1; + } + @memcpy(sofar[sofar_len .. sofar_len + name.len], name); + sofar_len += name.len; + } + return .{ .fid = fid, .qid = qid }; +} + +/// The parent directory of `path` as a fid, plus the final name. +const Parented = struct { fid: u32, name: []const u8 }; + +fn parentOf(path: []const u8, out: []u8) p9.Error!Parented { + const split = state.splitParent(path) orelse { + // The root has no parent, so nothing can be created or removed at it. + client.last_errno = linux.EBUSY; + return p9.Error.Remote; + }; + const walked = try walkTo(split[0], true, 0); + if ((walked.qid.qtype & p9.P9_QTDIR) == 0) { + client.clunk(walked.fid); + client.last_errno = linux.ENOTDIR; + return p9.Error.Remote; + } + if (split[1].len > out.len) { + client.clunk(walked.fid); + client.last_errno = linux.ENAMETOOLONG; + return p9.Error.Remote; + } + @memcpy(out[0..split[1].len], split[1]); + return .{ .fid = walked.fid, .name = out[0..split[1].len] }; +} + +// --------------------------------------------------------------------------- +// Open file descriptions +// --------------------------------------------------------------------------- + +/// Make sure a file's 9P fid is live, re-opening it if a `close` released it. +/// +/// A file description can outlive the last descriptor this supervisor *knows* +/// about: a shell opens a file, forks, the child `dup2`s it onto stdin and the +/// parent closes its copy, and the child's first read arrives against a +/// descriptor number nothing has bound yet. Rather than answer that from an +/// empty placeholder — which is the silent-wrong-answer failure this whole file +/// exists to remove — the description remembers its path, its access mode and +/// its offset, and is re-opened on demand. `O_TRUNC` and `O_CREAT` are +/// deliberately not part of what is remembered, for the reason +/// `src/fuse/flags.ts`'s `reopenFlags()` gives: repeating them on a re-open +/// empties the file or fails outright. +fn ensureOpen(index: u32) ?i32 { + const entry = tables.file(index); + if (entry.open) return null; + const walked = walkTo(entry.path, true, 0) catch |err| { + return if (err == p9.Error.Remote and client.last_errno > 0) client.last_errno else linux.EBADF; + }; + _ = client.lopen(walked.fid, if (entry.is_dir) 0 else entry.flags & linux.O_ACCMODE) catch |err| { + client.clunk(walked.fid); + return if (err == p9.Error.Remote and client.last_errno > 0) client.last_errno else linux.EBADF; + }; + entry.fid = walked.fid; + entry.open = true; + return null; +} + +/// Let a file description go once nothing names it any more. +/// +/// The 9P fid — the driver's actual open handle — is released here; the record +/// itself is kept, because a descriptor this supervisor never saw bound may +/// still resolve to it through `/proc` and need it re-opened. That is a bounded +/// amount of memory per distinct open, traded for never answering a live +/// descriptor from a dead one. +fn release(index: u32) void { + const entry = tables.file(index); + if (entry.refs != 0) return; + if (entry.open) { + client.clunk(entry.fid); + entry.open = false; + } +} + +/// A placeholder descriptor for the tracee: a `memfd` with a unique name. +/// +/// It holds nothing and is never read from. Its two jobs are to be a number the +/// kernel agrees exists — so `close`, `dup`, `fcntl` and `poll` behave — and to +/// carry an identity through `/proc//fd/` that survives `dup`, `fork` +/// and `exec`, which is how a duplicate is recognized without trapping `dup`. +/// +/// It is a `memfd` and **not** an `open("/dev/null")`: in the spike, where the +/// supervisor shared its filter with the tracee, `open` was trapped and calling +/// it here suspended the supervisor waiting for a reply only it could send. The +/// supervisor carries no filter now, but the `memfd` stays for its identity. +fn placeholder(serial: u32) i32 { + var namebuf: [32]u8 = undefined; + const name = std.fmt.bufPrintZ(&namebuf, "mx-{d}", .{serial}) catch return -1; + const rc = p9.syscall3(p9.SYS_memfd_create, @intFromPtr(name.ptr), 0, 0); + if (rc < 0) return -1; + return @intCast(rc); +} + +/// Bind a freshly opened description to a descriptor in the tracee. +fn install(id: u64, pid: i32, index: u32) Reply { + const entry = tables.file(index); + const fd = placeholder(entry.serial); + if (fd < 0) return fail(linux.EMFILE); + defer _ = c.close(fd); + const newfd = notify.addFd(id, fd); + if (newfd < 0) return fail(linux.EIO); + tables.bind(pid, newfd, index) catch return fail(linux.ENOMEM); + return .{ .ret = newfd }; +} + +// --------------------------------------------------------------------------- +// stat +// --------------------------------------------------------------------------- + +fn fillStat(a: p9.Attr) linux.Stat { + var st = std.mem.zeroes(linux.Stat); + st.st_dev = FAKE_DEV_MINOR; + st.st_ino = a.qid.path; + st.st_nlink = a.nlink; + st.st_mode = a.mode; + st.st_uid = a.uid; + st.st_gid = a.gid; + st.st_rdev = a.rdev; + st.st_size = @intCast(a.size); + st.st_blksize = @intCast(a.blksize); + st.st_blocks = @intCast(a.blocks); + st.st_atime = a.atime_sec; + st.st_atime_nsec = a.atime_nsec; + st.st_mtime = a.mtime_sec; + st.st_mtime_nsec = a.mtime_nsec; + st.st_ctime = a.ctime_sec; + st.st_ctime_nsec = a.ctime_nsec; + return st; +} + +fn writeStat(id: u64, pid: i32, remote_addr: u64, a: p9.Attr) Reply { + const st = fillStat(a); + const bytes: [*]const u8 = @ptrCast(&st); + if (!notify.stillValid(id)) return .gone; + if (!notify.writeTracee(pid, remote_addr, bytes[0..@sizeOf(linux.Stat)])) { + return fail(linux.EFAULT); + } + return ok; +} + +fn writeStatx(id: u64, pid: i32, remote_addr: u64, a: p9.Attr) Reply { + var stx = std.mem.zeroes(linux.Statx); + stx.stx_mask = linux.STATX_BASIC_STATS; + stx.stx_blksize = @intCast(a.blksize); + stx.stx_nlink = @intCast(a.nlink); + stx.stx_uid = a.uid; + stx.stx_gid = a.gid; + stx.stx_mode = @intCast(a.mode); + stx.stx_ino = a.qid.path; + stx.stx_size = a.size; + stx.stx_blocks = a.blocks; + stx.stx_dev_minor = FAKE_DEV_MINOR; + stx.stx_dev_major = 0; + stx.stx_atime = .{ .sec = @intCast(a.atime_sec), .nsec = @intCast(a.atime_nsec), .__pad = 0 }; + stx.stx_mtime = .{ .sec = @intCast(a.mtime_sec), .nsec = @intCast(a.mtime_nsec), .__pad = 0 }; + stx.stx_ctime = .{ .sec = @intCast(a.ctime_sec), .nsec = @intCast(a.ctime_nsec), .__pad = 0 }; + const bytes: [*]const u8 = @ptrCast(&stx); + if (!notify.stillValid(id)) return .gone; + if (!notify.writeTracee(pid, remote_addr, bytes[0..@sizeOf(linux.Statx)])) { + return fail(linux.EFAULT); + } + return ok; +} + +// --------------------------------------------------------------------------- +// One notification, unpacked +// --------------------------------------------------------------------------- + +/// A notification with the two identities it carries kept apart: `tid` is the +/// thread whose memory is read, `pid` is the thread group whose descriptors and +/// working directory are consulted. +const Call = struct { + id: u64, + tid: i32, + pid: i32, + args: [6]u64, + + fn fd(self: *const Call, index: usize) i32 { + return @bitCast(@as(u32, @truncate(self.args[index]))); + } + + fn word(self: *const Call, index: usize) u32 { + return @truncate(self.args[index]); + } + + /// A path argument, read out of the tracee and checked afterwards — never + /// before, which is the whole point of `SECCOMP_IOCTL_NOTIF_ID_VALID`. + fn path(self: *const Call, index: usize, into: []u8) ?[]const u8 { + const raw = notify.readTraceePath(self.tid, self.args[index], into) orelse return null; + if (!notify.stillValid(self.id)) return null; + return raw; + } +}; + +// --------------------------------------------------------------------------- +// Opening +// --------------------------------------------------------------------------- + +/// The flags a driver is allowed to see. +/// +/// The access mode and nothing else. `O_CREAT`/`O_EXCL` are answered here by +/// `Tlcreate`; `O_TRUNC` is answered by an explicit `Tsetattr`, so a driver that +/// happens not to implement it cannot leave a `>` redirection silently +/// appending; and `O_APPEND` is resolved here too, because 9P's `Twrite` carries +/// an explicit offset and a driver that also honoured the flag would append +/// twice. +fn driverFlags(flags: u32) u32 { + return flags & linux.O_ACCMODE; +} + +fn openCommon(call: *const Call, dirfd: i32, path_index: usize, flags: u32, mode: u32) Reply { + const raw = call.path(path_index, &pathbuf) orelse return .cont; + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + // `O_EXCL` and `O_NOFOLLOW` both stop at the link rather than at its target, + // which is what makes an exclusive open of a dangling symlink `EEXIST` + // rather than a create. + const nofollow = (flags & linux.O_NOFOLLOW) != 0 or + ((flags & linux.O_EXCL) != 0 and (flags & linux.O_CREAT) != 0); + const walked = walkTo(rel, !nofollow, 0) catch |err| { + if (err != p9.Error.Remote or client.last_errno != linux.ENOENT) return remote(err); + if ((flags & linux.O_CREAT) == 0) return remote(err); + return create(call, rel, flags, mode); + }; + const fid = walked.fid; + const is_dir = (walked.qid.qtype & p9.P9_QTDIR) != 0; + if ((flags & linux.O_CREAT) != 0 and (flags & linux.O_EXCL) != 0) { + client.clunk(fid); + return fail(linux.EEXIST); + } + if (is_dir and (flags & linux.O_ACCMODE) != linux.O_RDONLY) { + client.clunk(fid); + return fail(linux.EISDIR); + } + if (!is_dir and (flags & linux.O_DIRECTORY) != 0) { + client.clunk(fid); + return fail(linux.ENOTDIR); + } + _ = client.lopen(fid, if (is_dir) 0 else driverFlags(flags)) catch |err| { + client.clunk(fid); + return remote(err); + }; + if (!is_dir and (flags & linux.O_TRUNC) != 0) { + client.setattr(fid, p9.P9_SETATTR_SIZE, 0, 0, 0, 0, 0, 0, 0, 0) catch |err| { + client.clunk(fid); + return remote(err); + }; + } + return adopt(call, rel, fid, is_dir, flags); +} + +/// The `O_CREAT` half: `Tlcreate` against the parent directory. +/// +/// `Tlcreate` turns the *directory* fid it is given into the fid of the new +/// file, which is why the parent is walked onto a fid of its own and never +/// clunked separately on the success path. +fn create(call: *const Call, rel: []const u8, flags: u32, mode: u32) Reply { + var namebuf: [256]u8 = undefined; + const parent = parentOf(rel, &namebuf) catch |err| return remote(err); + const masked = mode & ~umaskOf(call.tid) & 0o7777; + _ = client.lcreate(parent.fid, parent.name, driverFlags(flags), masked) catch |err| { + client.clunk(parent.fid); + return remote(err); + }; + return adopt(call, rel, parent.fid, false, flags); +} + +/// Record a freshly opened fid and hand the tracee a descriptor for it. +fn adopt(call: *const Call, rel: []const u8, fid: u32, is_dir: bool, flags: u32) Reply { + const index = tables.create(rel) catch { + client.clunk(fid); + return fail(linux.ENOMEM); + }; + const entry = tables.file(index); + entry.fid = fid; + entry.open = true; + entry.is_dir = is_dir; + entry.flags = flags; + entry.offset = 0; + const reply = install(call.id, call.pid, index); + switch (reply) { + .ret => |value| { + dbg("open {s} -> fd {d} serial {d}", .{ rel, value, entry.serial }); + return reply; + }, + else => { + client.clunk(fid); + entry.open = false; + return reply; + }, + } +} + +// --------------------------------------------------------------------------- +// Reading and writing +// --------------------------------------------------------------------------- + +/// Everything a descriptor-based handler needs, or the reply that says why not. +const Held = union(enum) { file: u32, reply: Reply }; + +fn held(call: *const Call, fd: i32) Held { + const index = lookupFd(call.pid, fd) orelse return .{ .reply = .cont }; + if (ensureOpen(index)) |errno| return .{ .reply = fail(errno) }; + return .{ .file = index }; +} + +/// One `Tread` loop into `scratch`. Short reads are honest: `read(2)` may return +/// fewer bytes than asked for, and inventing the rest would be a lie about the +/// file. +const ReadResult = union(enum) { got: usize, reply: Reply }; + +fn readInto(index: u32, offset: u64, want: usize) ReadResult { + const entry = tables.file(index); + var got: usize = 0; + while (got < want) { + const n = client.read(entry.fid, offset + got, scratch[got..want]) catch |err| { + if (got == 0) return .{ .reply = remote(err) }; + break; + }; + if (n == 0) break; + got += n; + } + return .{ .got = got }; +} + +fn handleRead(call: *const Call, fd: i32, buf: u64, count: usize, at: ?u64) Reply { + const index = switch (held(call, fd)) { + .reply => |r| return r, + .file => |i| i, + }; + const entry = tables.file(index); + if (entry.is_dir) return fail(linux.EISDIR); + const offset = at orelse entry.offset; + const want = @min(count, scratch.len); + const got = switch (readInto(index, offset, want)) { + .reply => |r| return r, + .got => |n| n, + }; + if (!notify.stillValid(call.id)) return .gone; + if (!notify.writeTracee(call.tid, buf, scratch[0..got])) return fail(linux.EFAULT); + if (at == null) entry.offset = offset + got; + return .{ .ret = @intCast(got) }; +} + +fn handleWrite(call: *const Call, fd: i32, buf: u64, count: usize, at: ?u64) Reply { + const index = switch (held(call, fd)) { + .reply => |r| return r, + .file => |i| i, + }; + const entry = tables.file(index); + if (entry.is_dir) return fail(linux.EBADF); + var offset = at orelse entry.offset; + // `O_APPEND` is resolved here rather than by the driver: `Twrite` carries an + // explicit offset, so a driver honouring the flag as well would append the + // payload to a position that was already the end. + if (at == null and (entry.flags & linux.O_APPEND) != 0) { + const attr = client.getattr(entry.fid) catch |err| return remote(err); + offset = attr.size; + } + const want = @min(count, scratch.len); + if (!notify.readTracee(call.tid, buf, scratch[0..want])) return fail(linux.EFAULT); + if (!notify.stillValid(call.id)) return .gone; + var done: usize = 0; + while (done < want) { + const n = client.write(entry.fid, offset + done, scratch[done..want]) catch |err| { + if (done == 0) return remote(err); + break; + }; + if (n == 0) break; + done += n; + } + if (at == null) entry.offset = offset + done; + return .{ .ret = @intCast(done) }; +} + +/// `IOV_MAX`, from `include/uapi/linux/uio.h`. +const IOV_MAX = 1024; + +fn readIovecs(call: *const Call, addr: u64, count: usize, into: []linux.Iovec) ?[]linux.Iovec { + if (count > into.len) return null; + const bytes: [*]u8 = @ptrCast(into.ptr); + if (!notify.readTracee(call.tid, addr, bytes[0 .. count * @sizeOf(linux.Iovec)])) return null; + return into[0..count]; +} + +fn handleReadv(call: *const Call, fd: i32, addr: u64, count: usize, at: ?u64) Reply { + const index = switch (held(call, fd)) { + .reply => |r| return r, + .file => |i| i, + }; + const entry = tables.file(index); + if (entry.is_dir) return fail(linux.EISDIR); + if (count > IOV_MAX) return fail(linux.EINVAL); + var vectors: [IOV_MAX]linux.Iovec = undefined; + const iov = readIovecs(call, addr, count, &vectors) orelse return fail(linux.EFAULT); + if (!notify.stillValid(call.id)) return .gone; + var offset = at orelse entry.offset; + var total: usize = 0; + for (iov) |vec| { + const want = @min(@as(usize, @intCast(vec.len)), scratch.len); + if (want == 0) continue; + const got = switch (readInto(index, offset, want)) { + .reply => |r| return if (total == 0) r else .{ .ret = @intCast(total) }, + .got => |n| n, + }; + if (got == 0) break; + if (!notify.writeTracee(call.tid, vec.base, scratch[0..got])) return fail(linux.EFAULT); + total += got; + offset += got; + if (got < want) break; + } + if (at == null) entry.offset = offset; + return .{ .ret = @intCast(total) }; +} + +fn handleWritev(call: *const Call, fd: i32, addr: u64, count: usize, at: ?u64) Reply { + const index = switch (held(call, fd)) { + .reply => |r| return r, + .file => |i| i, + }; + const entry = tables.file(index); + if (entry.is_dir) return fail(linux.EBADF); + if (count > IOV_MAX) return fail(linux.EINVAL); + var vectors: [IOV_MAX]linux.Iovec = undefined; + const iov = readIovecs(call, addr, count, &vectors) orelse return fail(linux.EFAULT); + if (!notify.stillValid(call.id)) return .gone; + var offset = at orelse entry.offset; + if (at == null and (entry.flags & linux.O_APPEND) != 0) { + const attr = client.getattr(entry.fid) catch |err| return remote(err); + offset = attr.size; + } + var total: usize = 0; + for (iov) |vec| { + const want = @min(@as(usize, @intCast(vec.len)), scratch.len); + if (want == 0) continue; + if (!notify.readTracee(call.tid, vec.base, scratch[0..want])) return fail(linux.EFAULT); + var done: usize = 0; + while (done < want) { + const n = client.write(entry.fid, offset + done, scratch[done..want]) catch |err| { + if (total == 0 and done == 0) return remote(err); + break; + }; + if (n == 0) break; + done += n; + } + total += done; + offset += done; + if (done < want) break; + } + if (at == null) entry.offset = offset; + return .{ .ret = @intCast(total) }; +} + +fn handleLseek(call: *const Call) Reply { + const index = switch (held(call, call.fd(0))) { + .reply => |r| return r, + .file => |i| i, + }; + const entry = tables.file(index); + const offset: i64 = @bitCast(call.args[1]); + const whence = call.word(2); + var target: i64 = undefined; + switch (whence) { + linux.SEEK_SET => target = offset, + linux.SEEK_CUR => target = @as(i64, @intCast(entry.offset)) + offset, + linux.SEEK_END => { + const attr = client.getattr(entry.fid) catch |err| return remote(err); + target = @as(i64, @intCast(attr.size)) + offset; + }, + else => return fail(linux.EINVAL), + } + if (target < 0) return fail(linux.EINVAL); + entry.offset = @intCast(target); + // A directory's position is a `Treaddir` cookie rather than a byte offset, + // and the only seek that means anything on one is a rewind. + if (entry.is_dir and target == 0) entry.cookie = 0; + return .{ .ret = target }; +} + +fn handleGetdents(call: *const Call) Reply { + const index = switch (held(call, call.fd(0))) { + .reply => |r| return r, + .file => |i| i, + }; + const entry = tables.file(index); + if (!entry.is_dir) return fail(linux.ENOTDIR); + const remote_addr = call.args[1]; + const cap: usize = @min(@as(usize, @truncate(call.args[2])), scratch.len / 2); + + var block: [32 * 1024]u8 = undefined; + const got = client.readdir(entry.fid, entry.cookie, &block) catch |err| return remote(err); + if (got == 0) return .{ .ret = 0 }; + + var out: usize = 0; + var reader = p9.Reader{ .buf = block[0..got] }; + while (reader.at < got) { + const qid = p9.Qid.read(&reader) catch break; + const offset = reader.u64v() catch break; + const dtype = reader.u8v() catch break; + const name = reader.str() catch break; + const reclen = (linux.DIRENT64_HEADER + name.len + 1 + 7) & ~@as(usize, 7); + if (out + reclen > cap) break; + const record = scratch[out .. out + reclen]; + @memset(record, 0); + std.mem.writeInt(u64, record[0..8], qid.path, .little); + std.mem.writeInt(u64, record[8..16], offset, .little); + std.mem.writeInt(u16, record[16..18], @intCast(reclen), .little); + record[18] = dtype; + @memcpy(record[19 .. 19 + name.len], name); + out += reclen; + entry.cookie = offset; + } + // Nothing fitted, and the caller's buffer is not big enough for the next + // entry: `getdents64` says `EINVAL` for that rather than end-of-directory, + // which a zero would mean. + if (out == 0) return fail(linux.EINVAL); + if (!notify.stillValid(call.id)) return .gone; + if (!notify.writeTracee(call.tid, remote_addr, scratch[0..out])) return fail(linux.EFAULT); + return .{ .ret = @intCast(out) }; +} + +// --------------------------------------------------------------------------- +// Metadata by path and by descriptor +// --------------------------------------------------------------------------- + +/// The legacy `stat`/`lstat`, which name a path and a `struct stat` and nothing +/// else. Every fid taken here is clunked, including on the error path — a +/// leaked fid is a driver handle nothing will ever close. +fn handleStatPath(call: *const Call, dirfd: i32, path_index: usize, follow: bool, out: usize) Reply { + const raw = call.path(path_index, &pathbuf) orelse return .cont; + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, follow, 0) catch |err| return remote(err); + defer client.clunk(walked.fid); + const attr = client.getattr(walked.fid) catch |err| return remote(err); + return writeStat(call.id, call.tid, call.args[out], attr); +} + +fn handleFstat(call: *const Call) Reply { + const index = switch (held(call, call.fd(0))) { + .reply => |r| return r, + .file => |i| i, + }; + const attr = client.getattr(tables.file(index).fid) catch |err| return remote(err); + return writeStat(call.id, call.tid, call.args[1], attr); +} + +/// `newfstatat`, which is also `fstat` by another name when the path is empty +/// and `AT_EMPTY_PATH` is set — the form glibc's `fstat` actually uses. +fn handleNewfstatat(call: *const Call) Reply { + const dirfd = call.fd(0); + const flags = call.word(3); + const raw = call.path(1, &pathbuf) orelse return .cont; + if (raw.len == 0 and (flags & linux.AT_EMPTY_PATH) != 0) { + const index = switch (held(call, dirfd)) { + .reply => |r| return r, + .file => |i| i, + }; + const attr = client.getattr(tables.file(index).fid) catch |err| return remote(err); + return writeStat(call.id, call.tid, call.args[2], attr); + } + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + const follow = (flags & linux.AT_SYMLINK_NOFOLLOW) == 0; + const walked = walkTo(rel, follow, 0) catch |err| return remote(err); + defer client.clunk(walked.fid); + const attr = client.getattr(walked.fid) catch |err| return remote(err); + return writeStat(call.id, call.tid, call.args[2], attr); +} + +fn handleStatx(call: *const Call) Reply { + const dirfd = call.fd(0); + const flags = call.word(2); + const raw = call.path(1, &pathbuf) orelse return .cont; + var attr: p9.Attr = undefined; + if (raw.len == 0 and (flags & linux.AT_EMPTY_PATH) != 0) { + const index = switch (held(call, dirfd)) { + .reply => |r| return r, + .file => |i| i, + }; + attr = client.getattr(tables.file(index).fid) catch |err| return remote(err); + } else { + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, (flags & linux.AT_SYMLINK_NOFOLLOW) == 0, 0) catch |err| + return remote(err); + defer client.clunk(walked.fid); + attr = client.getattr(walked.fid) catch |err| return remote(err); + } + return writeStatx(call.id, call.tid, call.args[4], attr); +} + +/// `Tsetattr` against a path, which is `chmod`, `chown`, `truncate` and +/// `utimensat` alike. +fn setattrPath( + call: *const Call, + dirfd: i32, + path_index: usize, + follow: bool, + valid: u32, + mode: u32, + uid: u32, + gid: u32, + size: u64, + atime_sec: u64, + atime_nsec: u64, + mtime_sec: u64, + mtime_nsec: u64, +) Reply { + const raw = call.path(path_index, &pathbuf) orelse return .cont; + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, follow, 0) catch |err| return remote(err); + defer client.clunk(walked.fid); + client.setattr( + walked.fid, + valid, + mode, + uid, + gid, + size, + atime_sec, + atime_nsec, + mtime_sec, + mtime_nsec, + ) catch |err| return remote(err); + return ok; +} + +fn setattrFd( + call: *const Call, + fd: i32, + valid: u32, + mode: u32, + uid: u32, + gid: u32, + size: u64, +) Reply { + const index = switch (held(call, fd)) { + .reply => |r| return r, + .file => |i| i, + }; + client.setattr(tables.file(index).fid, valid, mode, uid, gid, size, 0, 0, 0, 0) catch |err| + return remote(err); + return ok; +} + +/// The `uid`/`gid` a `chown` actually asks for: `(uid_t) -1` means "leave it". +fn ownerBits(uid: u32, gid: u32) struct { u32, u32, u32 } { + var valid: u32 = 0; + if (uid != 0xffff_ffff) valid |= p9.P9_SETATTR_UID; + if (gid != 0xffff_ffff) valid |= p9.P9_SETATTR_GID; + return .{ valid, uid, gid }; +} + +/// `access(2)` — answered from the file's own mode bits against this process's +/// ids, which are the tracee's too. +fn accessAnswer(attr: p9.Attr, want: u32) Reply { + if (want == linux.F_OK) return ok; + var bits: u32 = undefined; + if (attr.uid == self_uid) { + bits = (attr.mode >> 6) & 7; + } else if (attr.gid == self_gid) { + bits = (attr.mode >> 3) & 7; + } else { + bits = attr.mode & 7; + } + var need: u32 = 0; + if ((want & linux.R_OK) != 0) need |= 4; + if ((want & linux.W_OK) != 0) need |= 2; + if ((want & linux.X_OK) != 0) need |= 1; + return if ((bits & need) == need) ok else fail(linux.EACCES); +} + +fn handleAccess(call: *const Call, dirfd: i32, path_index: usize, mode_index: usize) Reply { + const raw = call.path(path_index, &pathbuf) orelse return .cont; + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, true, 0) catch |err| return remote(err); + defer client.clunk(walked.fid); + const attr = client.getattr(walked.fid) catch |err| return remote(err); + return accessAnswer(attr, call.word(mode_index)); +} + +fn writeStatfs(call: *const Call, remote_addr: u64, got: p9.Client.Statfs) Reply { + var out = std.mem.zeroes(linux.Statfs); + out.f_type = @intCast(got.ftype); + out.f_bsize = @intCast(got.bsize); + out.f_blocks = got.blocks; + out.f_bfree = got.bfree; + out.f_bavail = got.bavail; + out.f_files = got.files; + out.f_ffree = got.ffree; + out.f_fsid = .{ @bitCast(@as(u32, @truncate(got.fsid))), @bitCast(@as(u32, @truncate(got.fsid >> 32))) }; + out.f_namelen = @intCast(got.namelen); + out.f_frsize = @intCast(got.bsize); + const bytes: [*]const u8 = @ptrCast(&out); + if (!notify.stillValid(call.id)) return .gone; + if (!notify.writeTracee(call.tid, remote_addr, bytes[0..@sizeOf(linux.Statfs)])) { + return fail(linux.EFAULT); + } + return ok; +} + +// --------------------------------------------------------------------------- +// The namespace +// --------------------------------------------------------------------------- + +/// Resolve a path to its parent directory fid and final name, for the calls +/// that create or remove a name rather than acting on what it points at. +const Named = union(enum) { at: Parented, reply: Reply }; + +fn namedAt(call: *const Call, dirfd: i32, path_index: usize, into: []u8) Named { + const raw = call.path(path_index, &pathbuf) orelse return .{ .reply = .cont }; + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .{ .reply = .cont }, + .err => |e| return .{ .reply = fail(e) }, + .inside => |p| p, + }; + const parent = parentOf(rel, into) catch |err| return .{ .reply = remote(err) }; + return .{ .at = parent }; +} + +fn handleMkdir(call: *const Call, dirfd: i32, path_index: usize, mode_index: usize) Reply { + var namebuf: [256]u8 = undefined; + const parent = switch (namedAt(call, dirfd, path_index, &namebuf)) { + .reply => |r| return r, + .at => |p| p, + }; + defer client.clunk(parent.fid); + const mode = call.word(mode_index) & ~umaskOf(call.tid) & 0o7777; + _ = client.mkdir(parent.fid, parent.name, mode, 0) catch |err| return remote(err); + return ok; +} + +fn handleUnlinkat(call: *const Call, dirfd: i32, path_index: usize, flags: u32) Reply { + var namebuf: [256]u8 = undefined; + const parent = switch (namedAt(call, dirfd, path_index, &namebuf)) { + .reply => |r| return r, + .at => |p| p, + }; + defer client.clunk(parent.fid); + client.unlinkat(parent.fid, parent.name, flags) catch |err| return remote(err); + return ok; +} + +fn handleMknod(call: *const Call, dirfd: i32, path_index: usize, mode: u32, dev: u64) Reply { + var namebuf: [256]u8 = undefined; + const parent = switch (namedAt(call, dirfd, path_index, &namebuf)) { + .reply => |r| return r, + .at => |p| p, + }; + defer client.clunk(parent.fid); + // `Tmknod` carries major and minor as separate words, which is the opposite + // of `Rgetattr`'s single packed `rdev`. The unpacking is glibc's + // `gnu_dev_major`/`gnu_dev_minor` (`bits/sysmacros.h`), which is the + // encoding `mknod(2)`'s `dev_t` argument uses. + const major: u32 = @truncate(((dev >> 8) & 0xfff) | ((dev >> 32) & ~@as(u64, 0xfff))); + const minor: u32 = @truncate((dev & 0xff) | ((dev >> 12) & ~@as(u64, 0xff))); + const masked = (mode & ~umaskOf(call.tid) & 0o7777) | (mode & linux.S_IFMT); + _ = client.mknod(parent.fid, parent.name, masked, major, minor, 0) catch |err| + return remote(err); + return ok; +} + +fn handleSymlink(call: *const Call, target_index: usize, dirfd: i32, path_index: usize) Reply { + // The link's *contents* are opaque and are never resolved, so they are read + // out of the tracee as a plain string rather than through `resolveAt`. + const contents = call.path(target_index, &auxbuf) orelse return .cont; + var namebuf: [256]u8 = undefined; + const parent = switch (namedAt(call, dirfd, path_index, &namebuf)) { + .reply => |r| return r, + .at => |p| p, + }; + defer client.clunk(parent.fid); + _ = client.symlink(parent.fid, parent.name, contents, 0) catch |err| return remote(err); + return ok; +} + +fn handleReadlink(call: *const Call, dirfd: i32, path_index: usize, buf: u64, size: usize) Reply { + const raw = call.path(path_index, &pathbuf) orelse return .cont; + const target = resolveAt(call.pid, dirfd, raw, &joinbuf); + const rel = switch (target) { + .outside => return .cont, + .err => |e| return fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, false, 0) catch |err| return remote(err); + defer client.clunk(walked.fid); + var link: [4096]u8 = undefined; + const contents = client.readlink(walked.fid, &link) catch |err| return remote(err); + const n = @min(contents.len, size); + if (!notify.stillValid(call.id)) return .gone; + if (!notify.writeTracee(call.tid, buf, contents[0..n])) return fail(linux.EFAULT); + return .{ .ret = @intCast(n) }; +} + +/// `rename`, in the `renameat` shape every form reduces to. +/// +/// A rename that would cross the boundary of the tree is `EXDEV` — which is the +/// truth: the two names are on different filesystems, and `mv` knows what to do +/// with that answer. +fn handleRename(call: *const Call, olddirfd: i32, old_index: usize, newdirfd: i32, new_index: usize) Reply { + const old_raw = call.path(old_index, &pathbuf) orelse return .cont; + var old_norm: [4096]u8 = undefined; + const old_target = resolveAt(call.pid, olddirfd, old_raw, &old_norm); + const new_raw = call.path(new_index, &auxbuf) orelse return .cont; + const new_target = resolveAt(call.pid, newdirfd, new_raw, &joinbuf); + switch (old_target) { + .outside => return switch (new_target) { + .inside => fail(linux.EXDEV), + else => .cont, + }, + .err => |e| return fail(e), + .inside => {}, + } + const old_rel = old_target.inside; + const new_rel = switch (new_target) { + .outside => return fail(linux.EXDEV), + .err => |e| return fail(e), + .inside => |p| p, + }; + var old_name: [256]u8 = undefined; + const old_parent = parentOf(old_rel, &old_name) catch |err| return remote(err); + defer client.clunk(old_parent.fid); + var new_name: [256]u8 = undefined; + const new_parent = parentOf(new_rel, &new_name) catch |err| return remote(err); + defer client.clunk(new_parent.fid); + client.renameat(old_parent.fid, old_parent.name, new_parent.fid, new_parent.name) catch |err| + return remote(err); + // Open descriptions remember a path so a `close`d-then-resurrected one can + // be re-opened; a rename moves what that path names, so they move with it. + // The fids already held are unaffected — 9P keeps an open file attached + // across a rename — but a description re-opened by its old name would find + // nothing. + repathSubtree(old_rel, new_rel); + return ok; +} + +fn repathSubtree(from: []const u8, to: []const u8) void { + var buf: [4096]u8 = undefined; + for (tables.files.items, 0..) |entry, index| { + if (!entry.used) continue; + const path = entry.path; + const moved = std.mem.eql(u8, path, from); + const inside = path.len > from.len and + std.mem.startsWith(u8, path, from) and + (from.len == 0 or path[from.len] == '/'); + if (!moved and !inside) continue; + const tail = if (moved) "" else path[from.len..]; + if (to.len + tail.len > buf.len) continue; + @memcpy(buf[0..to.len], to); + @memcpy(buf[to.len .. to.len + tail.len], tail); + tables.repath(@intCast(index), buf[0 .. to.len + tail.len]) catch {}; + } +} + +fn handleLink(call: *const Call, olddirfd: i32, old_index: usize, newdirfd: i32, new_index: usize, flags: u32) Reply { + const old_raw = call.path(old_index, &pathbuf) orelse return .cont; + var old_norm: [4096]u8 = undefined; + const old_target = resolveAt(call.pid, olddirfd, old_raw, &old_norm); + const new_raw = call.path(new_index, &auxbuf) orelse return .cont; + const new_target = resolveAt(call.pid, newdirfd, new_raw, &joinbuf); + switch (old_target) { + .outside => return switch (new_target) { + .inside => fail(linux.EXDEV), + else => .cont, + }, + .err => |e| return fail(e), + .inside => {}, + } + const new_rel = switch (new_target) { + .outside => return fail(linux.EXDEV), + .err => |e| return fail(e), + .inside => |p| p, + }; + // `link(2)` does not follow the source symlink; `linkat` with + // `AT_SYMLINK_FOLLOW` does. + const walked = walkTo(old_target.inside, (flags & linux.AT_SYMLINK_FOLLOW) != 0, 0) catch |err| + return remote(err); + defer client.clunk(walked.fid); + var namebuf: [256]u8 = undefined; + const parent = parentOf(new_rel, &namebuf) catch |err| return remote(err); + defer client.clunk(parent.fid); + client.link(parent.fid, walked.fid, parent.name) catch |err| return remote(err); + return ok; +} + +// --------------------------------------------------------------------------- +// Where a process thinks it is +// --------------------------------------------------------------------------- + +/// Would a `chdir` to this host path succeed? Absolute paths only, which is the +/// only shape that can reach here: a relative path from a process with a +/// virtual working directory is always resolved inside the tree. +fn reachableDirectory(raw: []const u8) bool { + if (raw.len == 0 or raw[0] != '/' or raw.len >= auxbuf.len) return false; + @memcpy(auxbuf[0..raw.len], raw); + auxbuf[raw.len] = 0; + return c.access(@ptrCast(&auxbuf), c.X_OK) == 0; +} + +/// `chdir` into the tree is answered without the kernel ever moving. +/// +/// There is nowhere for it to move *to* — the tree is not mounted — so the +/// working directory becomes a fact this supervisor keeps, and every relative +/// path from that process is resolved against it. The consequence is stated in +/// the file header: a relative `execve` (which no notification can redirect) +/// still resolves against the real working directory, which is wherever the +/// command started. +fn handleChdir(call: *const Call) Reply { + const raw = call.path(0, &pathbuf) orelse return .cont; + const target = resolveAt(call.pid, linux.AT_FDCWD, raw, &joinbuf); + const rel = switch (target) { + .outside => { + // Leaving the tree, *if the kernel agrees*. Clearing unconditionally + // would hand the process a real working directory it never reached: + // a `cd /nowhere` that fails leaves a shell exactly where it was, + // and every relative path afterwards would then be resolved against + // somewhere else entirely. The supervisor carries no filter, so it + // can simply ask. + if (reachableDirectory(raw)) tables.clearCwd(call.pid); + return .cont; + }, + .err => |e| return fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, true, 0) catch |err| return remote(err); + defer client.clunk(walked.fid); + if ((walked.qid.qtype & p9.P9_QTDIR) == 0) return fail(linux.ENOTDIR); + tables.setCwd(call.pid, rel) catch return fail(linux.ENOMEM); + return ok; +} + +fn handleFchdir(call: *const Call) Reply { + const index = lookupFd(call.pid, call.fd(0)) orelse { + tables.clearCwd(call.pid); + return .cont; + }; + const entry = tables.file(index); + if (!entry.is_dir) return fail(linux.ENOTDIR); + tables.setCwd(call.pid, entry.path) catch return fail(linux.ENOMEM); + return ok; +} + +/// `getcwd(2)` returns the length *including* the terminating NUL, and `ERANGE` +/// when the buffer cannot hold it. +fn handleGetcwd(call: *const Call) Reply { + const rel = tables.cwd(call.pid) orelse return .cont; + var buf: [4096]u8 = undefined; + var len: usize = 0; + if (root.len + rel.len + 2 > buf.len) return fail(linux.ENAMETOOLONG); + @memcpy(buf[0..root.len], root); + len += root.len; + if (rel.len != 0) { + buf[len] = '/'; + len += 1; + @memcpy(buf[len .. len + rel.len], rel); + len += rel.len; + } + buf[len] = 0; + len += 1; + const size: usize = @truncate(call.args[1]); + if (size < len) return fail(linux.ERANGE); + if (!notify.stillValid(call.id)) return .gone; + if (!notify.writeTracee(call.tid, call.args[0], buf[0..len])) return fail(linux.EFAULT); + return .{ .ret = @intCast(len) }; +} + +// --------------------------------------------------------------------------- +// Times +// --------------------------------------------------------------------------- + +/// The `Tsetattr` bits and values a `utimensat`-family call asks for. +/// +/// `UTIME_NOW` and `UTIME_OMIT` are the two sentinels that make this more than +/// a copy: the first sets the bit without the `_SET` companion (which is 9P's +/// way of saying "now"), and the second sets no bit at all. +const Times = struct { + valid: u32 = 0, + atime_sec: u64 = 0, + atime_nsec: u64 = 0, + mtime_sec: u64 = 0, + mtime_nsec: u64 = 0, +}; + +fn timesFromSpec(call: *const Call, addr: u64) ?Times { + var out = Times{}; + if (addr == 0) { + out.valid = p9.P9_SETATTR_ATIME | p9.P9_SETATTR_MTIME; + return out; + } + var spec: [2]linux.Timespec = undefined; + const bytes: [*]u8 = @ptrCast(&spec); + if (!notify.readTracee(call.tid, addr, bytes[0 .. 2 * @sizeOf(linux.Timespec)])) return null; + for (spec, 0..) |value, index| { + const is_atime = index == 0; + if (value.nsec == linux.UTIME_OMIT) continue; + const bit: u32 = if (is_atime) p9.P9_SETATTR_ATIME else p9.P9_SETATTR_MTIME; + out.valid |= bit; + if (value.nsec == linux.UTIME_NOW) continue; + out.valid |= if (is_atime) p9.P9_SETATTR_ATIME_SET else p9.P9_SETATTR_MTIME_SET; + if (is_atime) { + out.atime_sec = @intCast(value.sec); + out.atime_nsec = @intCast(value.nsec); + } else { + out.mtime_sec = @intCast(value.sec); + out.mtime_nsec = @intCast(value.nsec); + } + } + return out; +} + +/// The pre-`utimensat` forms, whose values are microseconds or whole seconds. +fn timesFromLegacy(call: *const Call, addr: u64, micro: bool) ?Times { + var out = Times{}; + if (addr == 0) { + out.valid = p9.P9_SETATTR_ATIME | p9.P9_SETATTR_MTIME; + return out; + } + out.valid = p9.P9_SETATTR_ATIME | p9.P9_SETATTR_ATIME_SET | + p9.P9_SETATTR_MTIME | p9.P9_SETATTR_MTIME_SET; + if (micro) { + var value: [2]linux.Timeval = undefined; + const bytes: [*]u8 = @ptrCast(&value); + if (!notify.readTracee(call.tid, addr, bytes[0 .. 2 * @sizeOf(linux.Timeval)])) return null; + out.atime_sec = @intCast(value[0].sec); + out.atime_nsec = @intCast(value[0].usec * 1000); + out.mtime_sec = @intCast(value[1].sec); + out.mtime_nsec = @intCast(value[1].usec * 1000); + } else { + var value: linux.Utimbuf = undefined; + const bytes: [*]u8 = @ptrCast(&value); + if (!notify.readTracee(call.tid, addr, bytes[0..@sizeOf(linux.Utimbuf)])) return null; + out.atime_sec = @intCast(value.actime); + out.mtime_sec = @intCast(value.modtime); + } + return out; +} + +fn applyTimes(call: *const Call, dirfd: i32, path_index: usize, follow: bool, times: Times) Reply { + return setattrPath( + call, + dirfd, + path_index, + follow, + times.valid, + 0, + 0, + 0, + 0, + times.atime_sec, + times.atime_nsec, + times.mtime_sec, + times.mtime_nsec, + ); +} + +// --------------------------------------------------------------------------- +// Descriptors this supervisor will not pretend about +// --------------------------------------------------------------------------- + +/// Refuse an operation on a virtual descriptor, with the errno that makes a +/// caller fall back rather than give up. +/// +/// `sendfile`, `splice` and `copy_file_range` would all otherwise run against +/// the empty placeholder and report a clean, confident, wrong answer — the +/// exact failure this design exists to remove. `EINVAL` and `EXDEV` are what +/// coreutils and glibc both take as "do it the long way", which they then do +/// through `read` and `write`, and those are answered properly. +fn refuseIfOurs(call: *const Call, fds: []const i32, errno: i32) Reply { + for (fds) |fd| { + if (lookupFd(call.pid, fd) != null) return fail(errno); + } + return .cont; +} + +/// Extended attributes: refused on anything of ours, which is also what the 9P +/// transport answers for `Txattrwalk`. +fn refuseXattrPath(call: *const Call, path_index: usize) Reply { + const raw = call.path(path_index, &pathbuf) orelse return .cont; + return switch (resolveAt(call.pid, linux.AT_FDCWD, raw, &joinbuf)) { + .outside => .cont, + .err => |e| fail(e), + .inside => fail(linux.ENOTSUP), + }; +} + +// --------------------------------------------------------------------------- +// Dispatch +// --------------------------------------------------------------------------- + +fn dispatch(call: *const Call, nr: i32) Reply { + return switch (nr) { + // --- descriptors ----------------------------------------------------- + SYS.read => handleRead(call, call.fd(0), call.args[1], @truncate(call.args[2]), null), + SYS.pread64 => handleRead(call, call.fd(0), call.args[1], @truncate(call.args[2]), call.args[3]), + SYS.write => handleWrite(call, call.fd(0), call.args[1], @truncate(call.args[2]), null), + SYS.pwrite64 => handleWrite(call, call.fd(0), call.args[1], @truncate(call.args[2]), call.args[3]), + SYS.readv => handleReadv(call, call.fd(0), call.args[1], @truncate(call.args[2]), null), + SYS.writev => handleWritev(call, call.fd(0), call.args[1], @truncate(call.args[2]), null), + // `preadv`'s offset arrives as a low/high pair, and on a 64-bit kernel + // `pos_from_hilo()` shifts the high half clean off the top — so the low + // word *is* the offset. `preadv2` adds a flags word after it. + SYS.preadv, SYS.preadv2 => handleReadv(call, call.fd(0), call.args[1], @truncate(call.args[2]), call.args[3]), + SYS.pwritev, SYS.pwritev2 => handleWritev(call, call.fd(0), call.args[1], @truncate(call.args[2]), call.args[3]), + SYS.lseek => handleLseek(call), + SYS.getdents64 => handleGetdents(call), + SYS.fstat => handleFstat(call), + SYS.close => blk: { + // Bookkeeping first, then the kernel really closes it: the number is + // reused immediately, and a mapping that outlived its descriptor + // shadowed a live one in the spike. + // + // The lookup is what binds a descriptor this supervisor never saw + // created — a `dup` it could not observe — so that closing it + // decrements the right reference count rather than none. + tables.unbind(call.pid, call.fd(0)); + break :blk .cont; + }, + // The two calls that replace a *live* descriptor number without a + // `close` anybody can see. Everything else that hands out a number + // (`open`, `dup`, `socket`, `fcntl(F_DUPFD)`) is given a free one by the + // kernel, so only these can leave a binding pointing at the wrong file. + SYS.dup2, SYS.dup3 => blk: { + const oldfd = call.fd(0); + const newfd = call.fd(1); + if (oldfd != newfd) { + tables.unbind(call.pid, newfd); + if (lookupFd(call.pid, oldfd)) |index| tables.bind(call.pid, newfd, index) catch {}; + } + break :blk .cont; + }, + SYS.close_range => blk: { + tables.unbindRange(call.pid, call.fd(0), call.fd(1)); + break :blk .cont; + }, + SYS.mmap => blk: { + const fd = call.fd(4); + if (fd >= 0 and lookupFd(call.pid, fd) != null) break :blk fail(linux.ENODEV); + break :blk .cont; + }, + SYS.fsync => blk: { + const index = switch (held(call, call.fd(0))) { + .reply => |r| break :blk r, + .file => |i| i, + }; + client.fsync(tables.file(index).fid, 0) catch |err| break :blk remote(err); + break :blk ok; + }, + SYS.fdatasync => blk: { + const index = switch (held(call, call.fd(0))) { + .reply => |r| break :blk r, + .file => |i| i, + }; + client.fsync(tables.file(index).fid, 1) catch |err| break :blk remote(err); + break :blk ok; + }, + SYS.ftruncate => setattrFd(call, call.fd(0), p9.P9_SETATTR_SIZE, 0, 0, 0, call.args[1]), + SYS.fchmod => setattrFd(call, call.fd(0), p9.P9_SETATTR_MODE, call.word(1) & 0o7777, 0, 0, 0), + SYS.fchown => blk: { + const bits = ownerBits(call.word(1), call.word(2)); + break :blk setattrFd(call, call.fd(0), bits[0], 0, bits[1], bits[2], 0); + }, + SYS.fstatfs => blk: { + const index = switch (held(call, call.fd(0))) { + .reply => |r| break :blk r, + .file => |i| i, + }; + const got = client.statfs(tables.file(index).fid) catch |err| break :blk remote(err); + break :blk writeStatfs(call, call.args[1], got); + }, + SYS.sendfile => refuseIfOurs(call, &.{ call.fd(0), call.fd(1) }, linux.EINVAL), + SYS.splice => refuseIfOurs(call, &.{ call.fd(0), call.fd(2) }, linux.EINVAL), + SYS.copy_file_range => refuseIfOurs(call, &.{ call.fd(0), call.fd(2) }, linux.EXDEV), + SYS.fallocate => refuseIfOurs(call, &.{call.fd(0)}, linux.ENOTSUP), + + // --- opening --------------------------------------------------------- + SYS.open => openCommon(call, linux.AT_FDCWD, 0, call.word(1), call.word(2)), + SYS.openat => openCommon(call, call.fd(0), 1, call.word(2), call.word(3)), + SYS.creat => openCommon( + call, + linux.AT_FDCWD, + 0, + linux.O_CREAT | linux.O_WRONLY | linux.O_TRUNC, + call.word(1), + ), + // `openat2` takes a `struct open_how` this supervisor does not decode. + // `ENOSYS` is the answer a pre-5.6 kernel gives and every caller has a + // fallback for — which is `openat`, and that is answered properly. + SYS.openat2 => blk: { + const raw = call.path(1, &pathbuf) orelse break :blk .cont; + break :blk switch (resolveAt(call.pid, call.fd(0), raw, &joinbuf)) { + .outside => .cont, + .err => |e| fail(e), + .inside => fail(linux.ENOSYS), + }; + }, + + // --- metadata -------------------------------------------------------- + SYS.stat => handleStatPath(call, linux.AT_FDCWD, 0, true, 1), + SYS.lstat => handleStatPath(call, linux.AT_FDCWD, 0, false, 1), + SYS.newfstatat => handleNewfstatat(call), + SYS.statx => handleStatx(call), + SYS.access => handleAccess(call, linux.AT_FDCWD, 0, 1), + SYS.faccessat, SYS.faccessat2 => handleAccess(call, call.fd(0), 1, 2), + SYS.statfs => blk: { + const raw = call.path(0, &pathbuf) orelse break :blk .cont; + const target = resolveAt(call.pid, linux.AT_FDCWD, raw, &joinbuf); + const rel = switch (target) { + .outside => break :blk .cont, + .err => |e| break :blk fail(e), + .inside => |p| p, + }; + const walked = walkTo(rel, true, 0) catch |err| break :blk remote(err); + defer client.clunk(walked.fid); + const got = client.statfs(walked.fid) catch |err| break :blk remote(err); + break :blk writeStatfs(call, call.args[1], got); + }, + SYS.truncate => setattrPath(call, linux.AT_FDCWD, 0, true, p9.P9_SETATTR_SIZE, 0, 0, 0, call.args[1], 0, 0, 0, 0), + SYS.chmod => setattrPath(call, linux.AT_FDCWD, 0, true, p9.P9_SETATTR_MODE, call.word(1) & 0o7777, 0, 0, 0, 0, 0, 0, 0), + SYS.fchmodat => setattrPath( + call, + call.fd(0), + 1, + (call.word(3) & linux.AT_SYMLINK_NOFOLLOW) == 0, + p9.P9_SETATTR_MODE, + call.word(2) & 0o7777, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + ), + SYS.chown, SYS.lchown => blk: { + const bits = ownerBits(call.word(1), call.word(2)); + break :blk setattrPath(call, linux.AT_FDCWD, 0, nr == SYS.chown, bits[0], 0, bits[1], bits[2], 0, 0, 0, 0, 0); + }, + SYS.fchownat => blk: { + const bits = ownerBits(call.word(2), call.word(3)); + const follow = (call.word(4) & linux.AT_SYMLINK_NOFOLLOW) == 0; + break :blk setattrPath(call, call.fd(0), 1, follow, bits[0], 0, bits[1], bits[2], 0, 0, 0, 0, 0); + }, + SYS.utimensat => blk: { + // A null path means the descriptor itself: this is `futimens`. + if (call.args[1] == 0) { + const index = switch (held(call, call.fd(0))) { + .reply => |r| break :blk r, + .file => |i| i, + }; + const times = timesFromSpec(call, call.args[2]) orelse break :blk fail(linux.EFAULT); + if (!notify.stillValid(call.id)) break :blk .gone; + client.setattr( + tables.file(index).fid, + times.valid, + 0, + 0, + 0, + 0, + times.atime_sec, + times.atime_nsec, + times.mtime_sec, + times.mtime_nsec, + ) catch |err| break :blk remote(err); + break :blk ok; + } + const times = timesFromSpec(call, call.args[2]) orelse break :blk fail(linux.EFAULT); + const follow = (call.word(3) & linux.AT_SYMLINK_NOFOLLOW) == 0; + break :blk applyTimes(call, call.fd(0), 1, follow, times); + }, + SYS.utimes => blk: { + const times = timesFromLegacy(call, call.args[1], true) orelse break :blk fail(linux.EFAULT); + break :blk applyTimes(call, linux.AT_FDCWD, 0, true, times); + }, + SYS.utime => blk: { + const times = timesFromLegacy(call, call.args[1], false) orelse break :blk fail(linux.EFAULT); + break :blk applyTimes(call, linux.AT_FDCWD, 0, true, times); + }, + SYS.futimesat => blk: { + const times = timesFromLegacy(call, call.args[2], true) orelse break :blk fail(linux.EFAULT); + break :blk applyTimes(call, call.fd(0), 1, true, times); + }, + + // --- the namespace --------------------------------------------------- + SYS.mkdir => handleMkdir(call, linux.AT_FDCWD, 0, 1), + SYS.mkdirat => handleMkdir(call, call.fd(0), 1, 2), + SYS.rmdir => handleUnlinkat(call, linux.AT_FDCWD, 0, p9.P9_DOTL_AT_REMOVEDIR), + SYS.unlink => handleUnlinkat(call, linux.AT_FDCWD, 0, 0), + SYS.unlinkat => handleUnlinkat( + call, + call.fd(0), + 1, + if ((call.word(2) & linux.AT_REMOVEDIR) != 0) p9.P9_DOTL_AT_REMOVEDIR else 0, + ), + SYS.rename => handleRename(call, linux.AT_FDCWD, 0, linux.AT_FDCWD, 1), + SYS.renameat => handleRename(call, call.fd(0), 1, call.fd(2), 3), + // `renameat2`'s flags are `RENAME_NOREPLACE`, `RENAME_EXCHANGE` and + // `RENAME_WHITEOUT`, none of which 9P's `Trenameat` can express. + // `EINVAL` is the documented "this filesystem does not support that", + // and it is what makes `mv` fall back to the plain form. + SYS.renameat2 => if (call.word(4) != 0) blk: { + const raw = call.path(1, &pathbuf) orelse break :blk .cont; + break :blk switch (resolveAt(call.pid, call.fd(0), raw, &joinbuf)) { + .outside => .cont, + .err => |e| fail(e), + .inside => fail(linux.EINVAL), + }; + } else handleRename(call, call.fd(0), 1, call.fd(2), 3), + SYS.link => handleLink(call, linux.AT_FDCWD, 0, linux.AT_FDCWD, 1, 0), + SYS.linkat => handleLink(call, call.fd(0), 1, call.fd(2), 3, call.word(4)), + SYS.symlink => handleSymlink(call, 0, linux.AT_FDCWD, 1), + SYS.symlinkat => handleSymlink(call, 0, call.fd(1), 2), + SYS.readlink => handleReadlink(call, linux.AT_FDCWD, 0, call.args[1], @truncate(call.args[2])), + SYS.readlinkat => handleReadlink(call, call.fd(0), 1, call.args[2], @truncate(call.args[3])), + SYS.mknod => handleMknod(call, linux.AT_FDCWD, 0, call.word(1), call.args[2]), + SYS.mknodat => handleMknod(call, call.fd(0), 1, call.word(2), call.args[3]), + + // --- where a process thinks it is ------------------------------------ + SYS.getcwd => handleGetcwd(call), + SYS.chdir => handleChdir(call), + SYS.fchdir => handleFchdir(call), + + // --- extended attributes --------------------------------------------- + SYS.setxattr, SYS.lsetxattr, SYS.getxattr, SYS.lgetxattr, SYS.listxattr, SYS.llistxattr, SYS.removexattr, SYS.lremovexattr => refuseXattrPath(call, 0), + SYS.fsetxattr, SYS.fgetxattr, SYS.flistxattr, SYS.fremovexattr => refuseIfOurs(call, &.{call.fd(0)}, linux.ENOTSUP), + + // --- lifecycle ------------------------------------------------------- + SYS.exit_group => blk: { + sweep(call.pid); + break :blk .cont; + }, + else => .cont, + }; +} + +/// Everything a process was holding, released the moment it says it is leaving. +/// +/// A tracee killed by a signal never reaches `exit_group`, which is what the +/// final sweep in `main` is for; between them, no fid outlives the process that +/// opened it by longer than the run. +fn sweep(pid: i32) void { + tables.unbindAll(pid); + tables.clearCwd(pid); + // The thread-id cache is the one table keyed on something that dies with + // the process rather than being reused, so it is also the one that would + // otherwise grow for the whole run of a command that spawns steadily. + var kept: usize = 0; + for (tgids.items) |entry| { + if (entry.tgid == pid) continue; + tgids.items[kept] = entry; + kept += 1; + } + tgids.shrinkRetainingCapacity(kept); +} + +/// Clunk the fid of everything whose last descriptor went during this +/// notification. Draining after the handler rather than inside it keeps the +/// tables free of anything that has to speak 9P. +fn drainOrphans() void { + for (tables.takeOrphans()) |index| release(index); + tables.clearOrphans(); +} + +// --------------------------------------------------------------------------- + +/// Entry point in C's shape rather than Zig's, because this binary links libc +/// and needs `argv` exactly as the kernel laid it out — it is passed straight +/// to `execvp` with only the leading arguments removed. +pub export fn main(argc: c_int, cargv: [*][*:0]u8) c_int { + if (argc < 5) die("usage: trace <9p-socket> -- [args...]", .{}); + const sock = cargv[1]; + root = std.mem.span(@as([*:0]const u8, cargv[2])); + if (!std.mem.eql(u8, std.mem.span(@as([*:0]const u8, cargv[3])), "--")) { + die("expected -- before the command", .{}); + } + const child_argv: [*:null]?[*:0]u8 = @ptrCast(cargv + 4); + + debug = c.getenv("MOUNTX_TRACE_DEBUG") != null; + gpa = std.heap.c_allocator; + tables = state.Tables.init(gpa); + self_uid = c.getuid(); + self_gid = c.getgid(); + + client.connect(std.mem.span(@as([*:0]const u8, sock))) catch + die("could not connect to the 9P socket {s}", .{sock}); + // The tracee has no business holding the supervisor's connection open. + _ = c.fcntl(client.fd, c.F_SETFD, @as(c_int, c.FD_CLOEXEC)); + + const pair = notify.socketpair() catch die("socketpair failed", .{}); + + const pid = c.fork(); + if (pid < 0) die("fork failed", .{}); + if (pid == 0) { + // The child installs the filter on *itself* and hands the listener + // back, so the supervisor carries no filter and may make any syscall it + // likes — which is what makes `close`, `read` and `write` trappable at + // all. Everything here happens before the first trapped syscall the + // child makes, and `execvp` is not one. + _ = c.close(pair[0]); + const fd = notify.installFilter(&TRAPPED) catch + die("could not install the seccomp filter", .{}); + _ = c.fcntl(fd, c.F_SETFD, @as(c_int, c.FD_CLOEXEC)); + notify.sendFd(pair[1], fd) catch die("could not hand the listener back", .{}); + _ = c.execvp(child_argv[0].?, @ptrCast(child_argv)); + die("could not exec {s}", .{child_argv[0].?}); + } + + _ = c.close(pair[1]); + const fd = notify.recvFd(pair[0]) catch die("never received the seccomp listener", .{}); + _ = c.close(pair[0]); + notify.setListener(fd); + + var status: c_int = 0; + var reaped = false; + var notif: notify.Notif = undefined; + while (true) { + // Poll rather than block in the ioctl, so a tracee that dies while + // nothing is in flight is noticed. A grandchild that outlives the + // command it was spawned from stops being served here, and its trapped + // syscalls then fail with `ENOSYS` — loud, and the same thing that + // happens to any process whose supervisor is gone. + switch (notify.wait(200)) { + .ready => {}, + .idle => { + if (c.waitpid(pid, &status, c.WNOHANG) == pid) { + reaped = true; + break; + } + continue; + }, + .failed => break, + } + if (!notify.receive(¬if)) { + if (c.waitpid(pid, &status, c.WNOHANG) == pid) { + reaped = true; + break; + } + continue; + } + const tid: i32 = @intCast(notif.pid); + const call = Call{ + .id = notif.id, + .tid = tid, + .pid = tgidOf(tid), + .args = notif.data.args, + }; + dbg("nr={d} tid={d} pid={d} args=({x},{x},{x})", .{ + notif.data.nr, + call.tid, + call.pid, + call.args[0], + call.args[1], + call.args[2], + }); + // Exactly one of these, for every notification, always. + switch (dispatch(&call, notif.data.nr)) { + .ret => |value| notify.respond(notif.id, value, 0), + .err => |errno| notify.respond(notif.id, 0, -errno), + .cont => notify.passthrough(notif.id), + .gone => {}, + } + drainOrphans(); + } + + // Whatever survived a signal rather than an `exit_group`. + for (tables.files.items, 0..) |entry, index| { + if (entry.used and entry.open) { + tables.files.items[index].refs = 0; + release(@intCast(index)); + } + } + if (!reaped) _ = c.waitpid(pid, &status, 0); + return if ((status & 0x7f) == 0) (status >> 8) & 0xff else 128 + (status & 0x7f); +} diff --git a/src/exec/userns-relay.ts b/src/exec/userns-relay.ts new file mode 100644 index 0000000..72c0499 --- /dev/null +++ b/src/exec/userns-relay.ts @@ -0,0 +1,224 @@ +/** + * The in-namespace half of `execUserns()`. + * + * This file is the only thing that runs *inside* the user+mount namespace. It + * exists because of one kernel rule: `unshare(CLONE_NEWUSER)` requires a + * single-threaded caller, and Node is never single-threaded (the libuv + * threadpool and V8's platform threads are up before any user code runs), so + * `EINVAL` is the only answer a mountx process could ever get from unsharing + * itself. `setns(2)` into a user namespace has the same rule. The namespace + * can therefore only ever be entered by a *child*, which means the process + * holding `/dev/fuse` is never the process holding the driver. + * + * So this helper holds the device and nothing else. It opens `/dev/fuse`, + * spawns `mount(8)` — inside the namespace `getuid()` is 0, so this is the + * ordinary root mount path with no `fusermount3` and no addon anywhere near it + * — and then pumps raw FUSE traffic over a unix socket to the parent, which + * runs the real {@link FuseSession} against the real driver. + * + * **Framing costs nothing.** Every FUSE message begins with its own total + * length in a little-endian `u32` (`fuse_in_header.len`, `fuse_out_header.len`), + * so a stream socket carries them with no envelope of our own. The one rule + * that does not survive the socket is that a reply must reach the device in a + * single `write(2)`; this side reassembles whole messages before writing, which + * is what the `#pending` buffer is for. + * + * **How a failure in here becomes an error out there.** Everything this process + * can fail at happens after `execUserns()` has already handed control to + * `unshare(1)`, so an exit status is the only channel back — and an exit status + * is exactly what the *command* is going to use too. Reporting `mount(8) never + * came up` as "the command exited 70" would be indistinguishable from a command + * that exited 70. So {@link fail} also writes its message to the file named by + * `$MOUNTX_RELAY_STATUS`, which the parent reads once the child is gone and + * turns into a thrown error; a run that never wrote the file ended for the + * command's own reasons. + * + * Usage: `node userns-relay.ts -- [args...]` + */ + +import { spawn } from "node:child_process"; +import * as fs from "node:fs"; +import * as net from "node:net"; + +/** `readBufferSize()`'s answer for the default `maxWrite`, spelled out here so + * the relay has no imports from `src/`. 1 MiB of payload plus a page of + * headers — negotiation can only ever agree to less. */ +const READ_BUFFER = 1024 * 1024 + 4096; + +/** Length prefix width: the `len` field at the head of every FUSE message. */ +const LEN_SIZE = 4; + +/** Where the parent reads a failure from. Absent when nobody is listening. */ +const statusPath = process.env.MOUNTX_RELAY_STATUS; + +function fail(message: string): never { + process.stderr.write(`mountx-relay: ${message}\n`); + if (statusPath !== undefined) { + // Best effort by design: the parent falls back to the exit status, and a + // relay that cannot write a file in a directory the parent just made has + // worse problems than the message it was trying to leave. + try { + fs.writeFileSync(statusPath, `${message}\n`); + } catch {} + } + process.exit(70); +} + +const argv = process.argv.slice(2); +const separator = argv.indexOf("--"); +if (separator < 0 || separator < 2) { + fail("usage: userns-relay -- [args...]"); +} +const socketPath = argv[0]!; +const mountpoint = argv[1]!; +const mountOptions = argv[2] === "--" ? "" : argv[2]!; +const command = argv.slice(separator + 1); +if (command.length === 0) { + fail("no command to run"); +} + +// Inside the namespace this process is uid 0 with a full capability set, so +// this is `mount(2)` by the ordinary route. Nothing here is setuid. +const uid = process.getuid?.() ?? -1; +if (uid !== 0) { + fail(`expected to be uid 0 inside the namespace, am ${uid}`); +} + +let device: number; +try { + device = fs.openSync("/dev/fuse", fs.constants.O_RDWR); +} catch (error) { + fail(`could not open /dev/fuse inside the namespace: ${(error as Error).message}`); +} + +const socket = net.connect(socketPath); +socket.on("error", (error) => fail(`socket: ${error.message}`)); +// The parent going away is not survivable: every request from here on would +// park forever in `fuse_get_req` with nobody to answer it, and this process +// would be left orphaned holding a wedged mount. Witnessed once during the +// spike, which is why it is handled rather than assumed away. Closing the +// device first is what aborts the connection so anything already blocked in +// the kernel gets an error instead of waiting. +socket.on("close", () => { + try { + fs.closeSync(device); + } catch {} + process.exit(75); +}); + +socket.on("connect", () => { + const rootMode = fs.statSync(mountpoint).mode; + const options = [ + // The fd has to land in the child at *its own* number, which is why the + // device is handed over as stdio slot 3 below and named as `fd=3` here. + "fd=3", + `rootmode=${rootMode.toString(8)}`, + `user_id=${uid}`, + `group_id=${process.getgid?.() ?? 0}`, + ...(mountOptions === "" ? [] : [mountOptions]), + ].join(","); + // `-i` so `mount(8)` does not hand off to `/sbin/mount.fuse`, `--` so a + // source beginning with a dash stays a source. Same reasoning as + // `src/fuse/mount.ts`; this is that call with the fd renumbered. + const mounter = spawn("mount", ["-i", "-t", "fuse", "-o", options, "--", "mountx", mountpoint], { + stdio: ["ignore", "inherit", "inherit", device], + }); + mounter.on("error", (error) => fail(`could not run mount(8): ${error.message}`)); + mounter.on("exit", (code) => { + if (code !== 0) { + fail(`mount(8) exited ${code} — the namespace mount never came up`); + } + pump(); + run(); + }); +}); + +/** Device → socket. One `read(2)` is exactly one message, forwarded verbatim. */ +function pump(): void { + const buffer = Buffer.allocUnsafe(READ_BUFFER); + const arm = (): void => { + fs.read(device, buffer, 0, buffer.length, null, (error, bytesRead) => { + if (error !== null) { + // ENODEV is the unmount signal: the connection is gone. + const code = (error as NodeJS.ErrnoException).code; + if (code === "ENODEV" || code === "EBADF") { + socket.end(); + return; + } + if (code === "EINTR" || code === "EAGAIN") { + arm(); + return; + } + fail(`reading /dev/fuse: ${error.message}`); + } + if (bytesRead > 0) { + socket.write(Buffer.from(buffer.subarray(0, bytesRead))); + } + arm(); + }); + }; + arm(); +} + +// Socket → device. Reassembled to whole messages, because the device rejects a +// reply delivered in pieces. +let pending = Buffer.alloc(0); +socket.on("data", (chunk: Buffer) => { + pending = pending.length === 0 ? chunk : Buffer.concat([pending, chunk]); + while (pending.length >= LEN_SIZE) { + const length = pending.readUInt32LE(0); + if (length < LEN_SIZE || pending.length < length) { + break; + } + const message = pending.subarray(0, length); + try { + fs.writeSync(device, message, 0, length); + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + // ENOENT here means the kernel already gave up on that request — an + // interrupted syscall, not our problem. Anything else is. + if (code !== "ENOENT" && code !== "ENODEV") { + process.stderr.write(`mountx-relay: writing /dev/fuse: ${(error as Error).message}\n`); + } + } + pending = pending.subarray(length); + } +}); + +/** Run the command the caller actually wanted, then take the mount down. */ +function run(): void { + // **`cwd` is deliberately not the mountpoint.** Witnessed: setting it wedges + // this process in `D` state at `fuse_get_req` and never comes back. It is the + // spawn hazard `src/fuse/mount.ts` and `src/9p/mount.ts` both document, met + // here from the inside — `uv_spawn` blocks the calling thread until the child + // execs, the child's first act is to `chdir` into the mount, and the reply + // that would unblock it can only come from the thread that is blocked. The + // mountpoint is handed over as an environment variable instead, so the `cd` + // happens *after* the exec, in a process that is not the one pumping. + // + // This is the same rule for the command binary itself: a command that lives + // on the mount deadlocks here and no amount of care on this side fixes it. + // Only a relay whose pump is not on the spawning thread would. + // `MOUNTX_RELAY_STATUS` is this process's private channel back to the parent + // and means nothing to the command, so it does not travel any further. + const env: NodeJS.ProcessEnv = { ...process.env, MOUNTX_ROOT: mountpoint }; + delete env.MOUNTX_RELAY_STATUS; + const child = spawn(command[0]!, command.slice(1), { stdio: "inherit", env }); + child.on("error", (error) => fail(`could not run ${command[0]}: ${error.message}`)); + child.on("exit", (code, signal) => { + // `umount` here is uid 0 in the namespace, so it needs no helper either. + // Failure is not worth reporting: this namespace is about to cease to + // exist, and with it the mount. + spawn("umount", [mountpoint], { stdio: "ignore" }).on("exit", () => { + try { + fs.closeSync(device); + } catch {} + socket.end(); + process.exitCode = signal === null ? (code ?? 0) : 128 + 1; + // The read loop holds a threadpool thread; nothing here can be exited + // out of politely once it is parked, so this is the one place a hard + // exit is right — the namespace and everything in it goes with us. + process.exit(process.exitCode); + }); + }); +} diff --git a/src/exec/userns.ts b/src/exec/userns.ts new file mode 100644 index 0000000..3b7c388 --- /dev/null +++ b/src/exec/userns.ts @@ -0,0 +1,360 @@ +/** + * `execUserns()` — run a command with a driver mounted over FUSE inside an + * unprivileged user namespace, visible to that command's process tree and to + * nothing else on the machine. + * + * This is the mechanism {@link import("./index.ts").exec} picks wherever the + * kernel's FUSE is usable, and it is not an approximation of a filesystem: what + * the child sees *is* FUSE, with the kernel's own VFS in front of it, so every + * POSIX guarantee `src/fuse/` already passes conformance on holds verbatim. It + * also runs a static binary, a Go binary and a setuid binary the same as any + * other, because nothing here depends on what the child is linked against. + * + * What it gives up is that it *is* a real kernel mount. It is simply a mount + * nobody outside the namespace can see — `/proc/self/mounts` on the host stays + * empty, and the mount dies with the namespace. That is the property that made + * a user-namespace mode not worth shipping back when the goal was "mount a + * directory for the machine" (see the roadmap's rootless-FUSE entry); for an + * `exec()`-shaped API the tradeoff inverts, and invisible is the point. + * + * **Why there is a helper process at all.** `unshare(CLONE_NEWUSER)` demands a + * single-threaded caller and Node is never single-threaded, so a mountx process + * cannot enter the namespace it needs — not with `unshare(2)`, and not with + * `setns(2)`, which has the same rule. The namespace is therefore entered by a + * child, `/dev/fuse` is opened in there, and the traffic comes back out over a + * unix socket to the session here. That split is not a workaround; it is the + * shape any version of this has to take, and it is the "relay mode" the roadmap + * defers, arrived at from the other direction. + * + * **Nothing here is privileged.** Inside the namespace the relay is uid 0 with + * `CAP_SYS_ADMIN`, so it takes the ordinary root mount path: no `fusermount3`, + * no setuid bit, and no native addon. On a host with no `fuse3` package + * installed at all — this project's dev host, see `.agents/environment.md` — + * this is the only FUSE route that works. + * + * ```ts + * const ran = await execUserns(createMemoryDriver(), ["sh", "-c", "ls -la $MOUNTX_ROOT"]); + * ran.code; // the command's exit status + * ``` + */ + +import { spawn } from "node:child_process"; +import { existsSync, readFileSync } from "node:fs"; +import { mkdir, mkdtemp, rm, stat } from "node:fs/promises"; +import * as net from "node:net"; +import { tmpdir } from "node:os"; +import { resolve } from "node:path"; +import { FuseSession, type FuseSessionOptions } from "../fuse/session.ts"; +import type { FsDriver } from "../types.ts"; +import { usernsExecProbe } from "./probe.ts"; + +/** The `len` field every FUSE message begins with. */ +const LEN_SIZE = 4; + +/** + * The `-o` options the mount gets when the caller names none: **none**, and + * `default_permissions` in particular. + * + * It is the option a FUSE mount usually wants, and it is wrong here, for a + * reason that only exists inside a user namespace. `default_permissions` asks + * the *kernel* to check the driver's `uid`/`gid`/`mode` against the caller's + * credentials — and in here those two are not the same identity space. + * `unshare -r` maps exactly one uid, yours, onto 0; a driver reporting your + * real uid (which every driver in this repository does, since that is what + * `process.getuid()` says on the serving side) is therefore reporting an + * identity the namespace does not map, which the kernel renders as `nobody`. + * Witnessed: with `default_permissions` on, a `mode` 0755 root directory owned + * by `nobody` refuses every write from the one process that is meant to have + * it, and namespace-root's `CAP_DAC_OVERRIDE` does not rescue it because that + * capability does not reach a file owned by an unmapped uid. + * + * So permission checking stays with the driver, where it can see who it is + * serving. Nothing is lost by it: the mount carries no `allow_other`, so the + * only process that can reach it is the one this call created it for. + */ +const DEFAULT_MOUNT_OPTIONS: readonly string[] = []; + +/** + * Where the relay is, relative to this module, in each layout it can be in. + * + * The relay is spawned rather than imported, so it has to be a file on disk + * with a path — which makes it the one thing here that has to survive the + * build as itself. It is a build entry for exactly that reason, landing at + * `dist/exec/userns-relay.mjs`; what moves underneath it is *this* file: + * + * - `src/exec/userns.ts` — the source tree, where the relay is a sibling `.ts` + * and Node's own type stripping runs it directly. + * - `dist/_chunks/userns.mjs` — the built package. `src/exec/index.ts` reaches + * this module through `await import()`, which is the whole point of the + * mechanism split, and obuild answers a dynamic import with a *chunk* rather + * than by inlining it. So the sibling relationship does not survive, and + * `dist/exec/` is one directory over. + * + * Both are checked, cheaply, once per call. A third layout would announce + * itself as the thrown error below rather than as a mystery `ENOENT` from + * `spawn`, which is what this is for. + */ +const RELAY_CANDIDATES = [ + "userns-relay.ts", + "userns-relay.mjs", + "../exec/userns-relay.mjs", +] as const; + +/** The relay on disk, in whichever layout this module was loaded from. */ +function relayPath(): string { + for (const name of RELAY_CANDIDATES) { + const candidate = new URL(name, import.meta.url).pathname; + if (existsSync(candidate)) { + return candidate; + } + } + throw new Error( + "mountx: the userns relay is missing — `userns-relay` is spawned rather than imported, so " + + "it has to exist as a file next to this module or under a sibling `exec/` directory, and " + + "an install or bundle that dropped it cannot run a command in a namespace", + ); +} + +export interface ExecUsernsOptions extends FuseSessionOptions { + /** + * Where the driver appears *inside the namespace*, and the value of + * `$MOUNTX_ROOT`. Defaults to a private temporary directory. + * + * The path has to exist on the real filesystem — a mount namespace is a copy + * of the parent's mount table, not an empty one — so a path that does not is + * created here, recursively. The mount made on it is visible only to the + * child tree either way. + */ + mountpoint?: string; + /** + * Working directory for the command. Defaults to this process's. + * + * **Never inside {@link ExecUsernsOptions.mountpoint}** — see + * {@link cwdRefusal}, which refuses that rather than letting it deadlock. + */ + cwd?: string; + /** Environment for the command. Defaults to this process's. */ + env?: NodeJS.ProcessEnv; + /** + * `-o` options passed through to `mount(8)` inside the namespace. Default + * none — see {@link DEFAULT_MOUNT_OPTIONS} for why `default_permissions` is + * not among them. + */ + mountOptions?: string[]; +} + +export interface ExecUsernsResult { + /** The command's exit status, or `null` if a signal ended it. */ + code: number | null; + /** The signal that ended the command, if one did. */ + signal: NodeJS.Signals | null; + /** Where the driver was mounted inside the namespace, and `$MOUNTX_ROOT`. */ + mountpoint: string; +} + +/** + * Why this `cwd` cannot be used with this mountpoint, or `undefined` if it can. + * + * The one refusal in this file that is about the *request* rather than the + * host, and the reason it exists is witnessed rather than theoretical: setting + * the child's `cwd` to the mountpoint wedges the relay in `D` state at + * `fuse_get_req` permanently. It is the spawn hazard `src/fuse/mount.ts` and + * `src/9p/mount.ts` both document, met from the inside — `uv_spawn` blocks the + * calling thread until the child execs, the child's first act is a `chdir` into + * the mount, and the reply that would unblock it can only come from the thread + * that is blocked. Nothing on this side can recover from it, so it is refused + * up front, with the spelling that works named in the message. + * + * Pure and exported for the Tier-0 test: the alternative way to check this + * costs a hung process. + */ +export function cwdRefusal(cwd: string, mountpoint: string): string | undefined { + const inside = resolve(cwd); + const root = resolve(mountpoint); + if (inside !== root && !inside.startsWith(`${root}/`)) { + return undefined; + } + return ( + `cwd ${inside} is inside the mountpoint ${root}, which deadlocks rather than failing: ` + + `uv_spawn blocks the thread serving FUSE until the child execs, and a child whose first ` + + `act is a chdir into the mount waits for a reply only that thread can send. Run the ` + + `command as \`sh -c 'cd "$MOUNTX_ROOT" && …'\` instead — a cd after the exec is safe` + ); +} + +/** + * Run `argv` with `driver` mounted at `options.mountpoint`, visible to that + * command and everything it spawns and to nothing else on the machine. + * + * Resolves when the command exits, with that command's status; a command that + * fails is not an error here. An error is thrown when the *mechanism* could not + * be set up — a host that cannot do this (see `usernsExecProbe()`), a request + * that cannot work ({@link cwdRefusal}), or a relay that failed before the + * command ran, whose own message is what surfaces. + * + * Needs no root, no `fusermount3` and no native addon. + */ +export async function execUserns( + driver: FsDriver, + argv: readonly string[], + options: ExecUsernsOptions = {}, +): Promise { + if (argv.length === 0) { + throw new Error("mountx: execUserns needs a command to run"); + } + // The whole host verdict up front, in one sentence naming every missing + // piece, rather than an ENOENT from an `open("/dev/fuse")` three processes + // away that nothing here would be able to explain by the time it arrived. + const probe = usernsExecProbe(); + if (!probe.usable) { + throw new Error(`mountx: cannot run a command in a user namespace here — ${probe.reason}`); + } + const relay = relayPath(); + + const scratch = await mkdtemp(resolve(tmpdir(), "mountx-exec-")); + const socketPath = resolve(scratch, "relay.sock"); + const statusPath = resolve(scratch, "relay.status"); + const mountpoint = + options.mountpoint === undefined ? resolve(scratch, "mnt") : resolve(options.mountpoint); + + let session: FuseSession | undefined; + let server: net.Server | undefined; + try { + await prepareMountpoint(mountpoint); + const cwd = resolve(options.cwd ?? process.cwd()); + const refusal = cwdRefusal(cwd, mountpoint); + if (refusal !== undefined) { + throw new Error(`mountx: ${refusal}`); + } + + session = new FuseSession(driver, options); + const attachedSession = session; + server = net.createServer(); + /** Resolves once the relay has connected and the session is wired to it. */ + const attached = new Promise((resolveAttached) => { + server!.on("connection", (socket) => { + attach(attachedSession, socket); + resolveAttached(); + }); + }); + await new Promise((resolveListen, rejectListen) => { + server!.once("error", rejectListen); + server!.listen(socketPath, resolveListen); + }); + + const relayArgs = [ + socketPath, + mountpoint, + (options.mountOptions ?? [...DEFAULT_MOUNT_OPTIONS]).join(","), + "--", + ...argv, + ]; + const child = spawn( + probe.unshare!, + // Short flags on purpose. `-U -r -m` mean the same thing to util-linux's + // `unshare(1)` and to busybox's applet, but the long spelling does not: + // busybox 1.37 has `-r` and no `--map-root-user` at all, so the long form + // fails outright on Alpine and anything else that is busybox-only. + // `--propagation` is the one long option both do accept. + ["-U", "-r", "-m", "--propagation", "private", process.execPath, relay, ...relayArgs], + { + stdio: "inherit", + cwd, + // `MOUNTX_RELAY_STATUS` is how a relay failure reaches this process as + // an error instead of masquerading as the command's own exit status. + // The relay strips it back out of what the command sees. + env: { ...(options.env ?? process.env), MOUNTX_RELAY_STATUS: statusPath }, + }, + ); + const exited = new Promise((resolveExit, rejectExit) => { + child.on("error", (error) => + rejectExit( + new Error(`mountx: could not run ${probe.unshare} — ${(error as Error).message}`, { + cause: error, + }), + ), + ); + child.on("exit", (code, signal) => resolveExit({ code, signal, mountpoint })); + }); + // If the relay dies before connecting, `exited` settles first and nothing + // waits forever on a handshake that is not coming. + await Promise.race([attached, exited]); + const result = await exited; + const failure = statusMessage(statusPath); + if (failure !== undefined) { + throw new Error(`mountx: ${failure}`); + } + return result; + } finally { + await session?.destroy().catch(() => {}); + if (server !== undefined) { + await new Promise((resolveClose) => server!.close(() => resolveClose())); + } + await rm(scratch, { recursive: true, force: true }).catch(() => {}); + } +} + +/** + * Make sure `mountpoint` is a directory, creating it if it is not there. + * + * A missing one is the ordinary case (it is the default, in a directory this + * call just made), and a non-directory is a mistake worth naming: `mount(8)` + * inside the namespace would answer `not a directory` from three processes + * away, where the caller cannot see which path it meant. + */ +async function prepareMountpoint(mountpoint: string): Promise { + try { + const stats = await stat(mountpoint); + if (!stats.isDirectory()) { + throw new Error(`mountx: mountpoint ${mountpoint} exists and is not a directory`); + } + return; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") { + throw error; + } + } + await mkdir(mountpoint, { recursive: true }); +} + +/** What the relay said before giving up, or `undefined` if it never did. */ +function statusMessage(path: string): string | undefined { + try { + return readFileSync(path, "utf8").trim() || undefined; + } catch { + return undefined; + } +} + +/** + * Drive a {@link FuseSession} over a stream instead of over `/dev/fuse`. + * + * The zero-copy contract still applies and is still met the same way: each + * whole message is handed to `handleMessage` without awaiting it, and the + * session copies what it retains before its first `await`. The one difference + * from the device is that `pending` here owns its bytes — a socket chunk is + * already a fresh buffer — so slicing a message out of it is safe. + */ +function attach(session: FuseSession, socket: net.Socket): void { + let pending = Buffer.alloc(0); + socket.on("data", (chunk: Buffer) => { + pending = pending.length === 0 ? chunk : Buffer.concat([pending, chunk]); + while (pending.length >= LEN_SIZE) { + const length = pending.readUInt32LE(0); + if (length < LEN_SIZE || pending.length < length) { + break; + } + const message = pending.subarray(0, length); + pending = pending.subarray(length); + void session + .handleMessage(message) + .then((reply) => { + if (reply !== null && !socket.destroyed) { + socket.write(reply); + } + }) + .catch(() => {}); + } + }); + socket.on("error", () => {}); +} diff --git a/test/exec/compare.sh b/test/exec/compare.sh new file mode 100644 index 0000000..e09605b --- /dev/null +++ b/test/exec/compare.sh @@ -0,0 +1,111 @@ +#!/bin/sh +# Build both interception mechanisms and run one identical workload through +# each, so the comparison in `.agents/proot-plan.md` is measured rather than +# argued. +# +# A third mechanism, an `LD_PRELOAD` interposer, was measured here too and is +# gone: its column could not serve a static or Go binary at all, and its +# write-back row lost data while reporting success. `.agents/proot-plan.md` +# keeps the numbers; the code is out of the tree. +# +# sh test/exec/compare.sh +# +# Needs a Zig toolchain (spikes B and C are native), unprivileged user +# namespaces (spike A) and nothing else. No root anywhere. +set -eu + +ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd) +OUT=${MOUNTX_SPIKE_OUT:-${TMPDIR:-/tmp}/mountx-spike} +mkdir -p "$OUT" +cd "$ROOT" + +say() { printf '\n\033[1m%s\033[0m\n' "$*"; } +row() { printf ' %-22s %s\n' "$1" "$2"; } + +say "building fixtures and mechanisms into $OUT" +# One workload, three linkages — the axis the whole comparison turns on. +zig cc test/exec/probe.c -O2 -o "$OUT/probe-glibc" +zig cc -target x86_64-linux-musl -static test/exec/probe.c -O2 -o "$OUT/probe-musl" +# -ffreestanding -fno-builtin or the compiler turns the hand-written loops back +# into calls to the libc this binary deliberately does not have. +zig cc -target x86_64-linux-none -nostdlib -static -ffreestanding -fno-builtin \ + test/exec/probe-raw.c -O2 -o "$OUT/probe-raw" +( cd src/exec && zig build-exe -lc -O ReleaseSmall -femit-bin="$OUT/mountx-trace" \ + --dep p9 -Mroot=seccomp/trace.zig -Mp9=seccomp/p9.zig ) +row "supervisor" "$(wc -c < "$OUT/mountx-trace") bytes" + +export MOUNTX_TRACE="$OUT/mountx-trace" + +# The checksum every passing run must produce over the 3 MiB file. Any +# divergence here is a correctness failure, not a coverage gap — which is the +# distinction that matters most, since one of the mechanisms was briefly +# capable of returning a clean, confident, wrong answer. +EXPECT=ae82061e44e22325 + +probe() { # + out=$(timeout 60 node "$1" "$OUT/$2" 2>&1 || true) + if printf '%s' "$out" | grep -q "fnv=$EXPECT"; then + printf 'pass' + elif printf '%s' "$out" | grep -q 'fnv='; then + printf 'WRONG DATA' + else + printf 'fail' + fi +} + +# Each column is one *named* mechanism, run through its own demo runner, so the +# comparison never depends on what the picker in `mountx/exec` would have chosen. +for spike in userns seccomp; do + case $spike in + userns) name="A userns + FUSE" ;; + seccomp) name="C seccomp notify" ;; + esac + say "spike $name" + for p in probe-glibc probe-musl probe-raw; do + row "$p" "$(probe "src/exec/demo-$spike.ts" "$p")" + done +done + +say "coreutils workload (glibc, the case both claim)" +WORK=' + set -e + ls "$MOUNTX_ROOT" >/dev/null + cat "$MOUNTX_ROOT/hello.txt" >/dev/null + sha256sum "$MOUNTX_ROOT/big.bin" | cut -c1-16 + wc -l < "$MOUNTX_ROOT/numbers.txt" + tail -1 "$MOUNTX_ROOT/numbers.txt" + find "$MOUNTX_ROOT" -type f | wc -l + du -s "$MOUNTX_ROOT" | cut -f1 +' +for spike in userns seccomp; do + printf ' %-8s: ' "$spike" + timeout 90 node "src/exec/demo-$spike.ts" sh -c "$WORK" 2>&1 | + grep -vE '^\[(userns|seccomp)' | tr '\n' ' ' + printf '\n' +done + +say "write-back (does what the command wrote actually reach the driver?)" +# The axis that separates the three, and the one worth measuring rather than +# claiming: the seccomp supervisor answered `openat` from an injected `memfd` +# until this was closed, so every one of these reported success and changed +# nothing. A mechanism that prints anything but the four expected words here is +# losing data quietly, which is worse than failing. +WRITES=' + echo created > "$MOUNTX_ROOT/w.txt" + cat "$MOUNTX_ROOT/w.txt" + echo appended >> "$MOUNTX_ROOT/w.txt" + tail -1 "$MOUNTX_ROOT/w.txt" + printf ab | dd of="$MOUNTX_ROOT/w.txt" bs=1 seek=0 conv=notrunc 2>/dev/null + head -c 2 "$MOUNTX_ROOT/w.txt"; echo + rm -f "$MOUNTX_ROOT/w.txt" + test -e "$MOUNTX_ROOT/w.txt" && echo "STILL THERE" || echo removed +' +for spike in userns seccomp; do + printf ' %-8s: ' "$spike" + timeout 90 node "src/exec/demo-$spike.ts" sh -c "$WRITES" 2>&1 | + grep -vE '^\[(userns|seccomp)' | tr '\n' ' ' + printf '\n' +done + +say "done — expected sha prefix bfe74807c87a6443, 5 files, 3082 blocks" +say " — expected writes: created appended ab removed" diff --git a/test/exec/probe-raw.c b/test/exec/probe-raw.c new file mode 100644 index 0000000..fdd62d8 --- /dev/null +++ b/test/exec/probe-raw.c @@ -0,0 +1,182 @@ +/* + * SPIKE fixture: the same workload with no libc anywhere — raw `syscall` + * instructions, its own `_start`, `-nostdlib -static`. + * + * zig cc -target x86_64-linux-none -nostdlib -static probe-raw.c -o probe-raw + * + * This is the Go case. A Go binary issues its syscalls from its own runtime + * rather than through libc, which is why it is the standard counter-example to + * every `LD_PRELOAD` sandbox — there is no symbol to interpose, and on a static + * binary there is not even a dynamic loader to read `LD_PRELOAD` in the first + * place. Writing it in C with inline asm rather than installing a Go toolchain + * keeps the fixture to one file and makes the syscalls it issues explicit, + * which is the whole point of the fixture. + * + * A mechanism that serves this binary serves anything. + */ + +typedef long ssize_t_; +typedef unsigned long size_t_; + +#define SYS_read 0 +#define SYS_write 1 +#define SYS_close 3 +#define SYS_openat 257 +#define SYS_getdents64 217 +#define SYS_statx 332 +#define SYS_exit_group 231 +#define AT_FDCWD (-100) +#define O_RDONLY 0 + +static long sys(long n, long a, long b, long c, long d, long e, long f) { + long ret; + register long r10 __asm__("r10") = d; + register long r8 __asm__("r8") = e; + register long r9 __asm__("r9") = f; + __asm__ volatile("syscall" + : "=a"(ret) + : "a"(n), "D"(a), "S"(b), "d"(c), "r"(r10), "r"(r8), "r"(r9) + : "rcx", "r11", "memory"); + return ret; +} + +static size_t_ slen(const char *s) { + size_t_ n = 0; + while (s[n]) n++; + return n; +} +static void out(const char *s) { sys(SYS_write, 1, (long)s, (long)slen(s), 0, 0, 0); } +static void outn(unsigned long long v) { + char b[24]; + int i = 23; + b[i--] = 0; + if (!v) b[i--] = '0'; + while (v) { b[i--] = (char)('0' + (v % 10)); v /= 10; } + out(&b[i + 1]); +} +static void outx(unsigned long long v) { + char b[24]; + int i = 23; + b[i--] = 0; + if (!v) b[i--] = '0'; + while (v) { b[i--] = "0123456789abcdef"[v & 15]; v >>= 4; } + out(&b[i + 1]); +} +static char *cat(char *dst, const char *a, const char *b) { + char *p = dst; + while (*a) *p++ = *a++; + while (*b) *p++ = *b++; + *p = 0; + return dst; +} + +/* Only the fields this fixture reads; the kernel fills the rest. */ +struct statx_ { + unsigned int mask, blksize; + unsigned long long attributes; + unsigned int nlink, uid, gid; + unsigned short mode, spare0[1]; + unsigned long long ino, size, blocks, attributes_mask; + unsigned long long rest[24]; +}; + +struct dirent64_ { + unsigned long long d_ino, d_off; + unsigned short d_reclen; + unsigned char d_type; + char d_name[]; +}; + +static const char *env_root(char **envp) { + for (char **e = envp; *e; e++) { + const char *k = "MOUNTX_ROOT="; + const char *p = *e; + size_t_ i = 0; + while (k[i] && p[i] == k[i]) i++; + if (!k[i]) return p + i; + } + return 0; +} + +static unsigned char buf[4 << 20]; + +int probe_main(int argc, char **argv, char **envp) { + const char *root = env_root(envp); + if (!root && argc > 1) root = argv[1]; + if (!root) root = "/"; + out("probe-raw: root="); + out(root); + out("\n"); + char path[4096]; + + /* 1. small file */ + cat(path, root, "/hello.txt"); + long fd = sys(SYS_openat, AT_FDCWD, (long)path, O_RDONLY, 0, 0, 0); + if (fd < 0) { out("probe-raw: FAIL open hello.txt errno="); outn((unsigned long long)-fd); out("\n"); return 1; } + long n = sys(SYS_read, fd, (long)buf, 256, 0, 0, 0); + sys(SYS_close, fd, 0, 0, 0, 0, 0); + out("probe-raw: hello.txt "); + outn((unsigned long long)(n < 0 ? 0 : n)); + out(" bytes: "); + if (n > 0) { buf[n] = 0; out((char *)buf); } + + /* 2. statx + whole-file read */ + cat(path, root, "/big.bin"); + struct statx_ stx; + long rc = sys(SYS_statx, AT_FDCWD, (long)path, 0, 0xfff, (long)&stx, 0); + if (rc < 0) { out("probe-raw: FAIL statx errno="); outn((unsigned long long)-rc); out("\n"); return 1; } + fd = sys(SYS_openat, AT_FDCWD, (long)path, O_RDONLY, 0, 0, 0); + if (fd < 0) { out("probe-raw: FAIL open big.bin\n"); return 1; } + unsigned long long got = 0, h = 0xcbf29ce484222325ULL; + for (;;) { + long r = sys(SYS_read, fd, (long)buf, sizeof buf, 0, 0, 0); + if (r <= 0) break; + for (long i = 0; i < r; i++) { h ^= buf[i]; h *= 0x100000001b3ULL; } + got += (unsigned long long)r; + } + sys(SYS_close, fd, 0, 0, 0, 0, 0); + out("probe-raw: big.bin stat="); + outn(stx.size); + out(" read="); + outn(got); + out(" fnv="); + outx(h); + out("\n"); + + /* 3. getdents64 */ + fd = sys(SYS_openat, AT_FDCWD, (long)root, O_RDONLY | 0200000 /* O_DIRECTORY */, 0, 0, 0); + if (fd < 0) { out("probe-raw: FAIL opendir\n"); return 1; } + unsigned long long count = 0; + for (;;) { + long r = sys(SYS_getdents64, fd, (long)buf, 32768, 0, 0, 0); + if (r <= 0) break; + for (long off = 0; off < r;) { + struct dirent64_ *e = (struct dirent64_ *)(buf + off); + if (e->d_name[0] != '.' || (e->d_name[1] && !(e->d_name[1] == '.' && !e->d_name[2]))) { + count++; + out("probe-raw: dirent "); + out(e->d_name); + out("\n"); + } + off += e->d_reclen; + } + } + sys(SYS_close, fd, 0, 0, 0, 0, 0); + out("probe-raw: "); + outn(count); + out(" entries\nprobe-raw: OK\n"); + return 0; +} + +/* No libc, so the kernel's entry contract is honoured by hand: rsp points at + * argc, then argv, then a NULL, then envp. */ +__asm__(".globl _start\n_start:\n xor %rbp, %rbp\n mov %rsp, %rdi\n and $-16, %rsp\n call start_c\n"); + +void start_c(long *sp) { + int argc = (int)sp[0]; + char **argv = (char **)&sp[1]; + char **envp = argv + argc + 1; + int rc = probe_main(argc, argv, envp); + sys(SYS_exit_group, rc, 0, 0, 0, 0, 0); + __builtin_unreachable(); +} diff --git a/test/exec/probe.c b/test/exec/probe.c new file mode 100644 index 0000000..cc4bdba --- /dev/null +++ b/test/exec/probe.c @@ -0,0 +1,85 @@ +/* + * SPIKE fixture: the same filesystem workload, built two ways off one source. + * + * The three interception mechanisms differ almost entirely in *which binaries + * they can serve*, so the comparison is only honest if the workload is + * identical and the linkage is the only variable: + * + * zig cc probe.c -o probe-glibc dynamic glibc + * zig cc -target x86_64-linux-musl -static probe.c static, no loader + * + * The first is what an LD_PRELOAD shim can serve; the second is what it cannot, + * because a static binary never consults a dynamic loader and therefore never + * consults LD_PRELOAD. `probe-raw.c` is the third case — no libc at all. + * + * Written in C rather than Zig on purpose: this fixture exists to pin down + * *which libc symbols* get called, and C is the language where that is written + * down rather than inferred from a standard library's internals. + */ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +/* Cheap enough to run on every invocation, specific enough that a short read + * or a torn offset changes it. */ +static unsigned long long fnv(const unsigned char *b, size_t n) { + unsigned long long h = 0xcbf29ce484222325ULL; + for (size_t i = 0; i < n; i++) { + h ^= b[i]; + h *= 0x100000001b3ULL; + } + return h; +} + +int main(int argc, char **argv) { + const char *root = getenv("MOUNTX_ROOT"); + if (!root && argc > 1) root = argv[1]; + if (!root) root = "/"; + printf("probe: root=%s\n", root); + char p[4096]; + + /* 1. open + read a small file. */ + snprintf(p, sizeof p, "%s/hello.txt", root); + int fd = open(p, O_RDONLY); + if (fd < 0) { perror("probe: open hello.txt"); return 1; } + char small[256]; + ssize_t n = read(fd, small, sizeof small); + close(fd); + printf("probe: hello.txt %zd bytes: %.*s", n, (int)(n < 0 ? 0 : n), small); + + /* 2. stat, then read a large file whole and checksum it. Catches short + * reads and offset bugs a small file never would. */ + snprintf(p, sizeof p, "%s/big.bin", root); + struct stat st; + if (stat(p, &st)) { perror("probe: stat big.bin"); return 1; } + fd = open(p, O_RDONLY); + if (fd < 0) { perror("probe: open big.bin"); return 1; } + unsigned char *buf = malloc((size_t)st.st_size); + size_t got = 0; + ssize_t r; + while (got < (size_t)st.st_size && (r = read(fd, buf + got, (size_t)st.st_size - got)) > 0) { + got += (size_t)r; + } + close(fd); + printf("probe: big.bin stat=%lld read=%zu fnv=%llx\n", (long long)st.st_size, got, fnv(buf, got)); + + /* 3. read the directory. */ + DIR *d = opendir(root); + if (!d) { perror("probe: opendir"); return 1; } + struct dirent *e; + int count = 0; + while ((e = readdir(d))) { + if (!strcmp(e->d_name, ".") || !strcmp(e->d_name, "..")) continue; + count++; + printf("probe: dirent %s type=%d\n", e->d_name, e->d_type); + } + closedir(d); + printf("probe: %d entries\nprobe: OK\n", count); + return 0; +} diff --git a/test/exec/seccomp-build.ts b/test/exec/seccomp-build.ts new file mode 100644 index 0000000..f8b4ee3 --- /dev/null +++ b/test/exec/seccomp-build.ts @@ -0,0 +1,82 @@ +/** + * Building the supervisor, and deciding whether this host can run it at all. + * + * The Tier-2 files here follow the same discipline as `test/nfs/mount.test.ts` + * and `test/9p/mount.test.ts`: the precondition is checked up front and the + * whole file skips itself when it is missing, so `pnpm test` stays green on a + * machine with no Zig toolchain. Unlike those two the precondition is not root + * — a seccomp user-notification filter needs no privileges at all, only + * `no_new_privs` — which is the entire reason this mechanism is being pursued. + * + * Not a `*.test.ts` file: it is imported by them. + */ + +import { execFile } from "node:child_process"; +import { mkdir } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { promisify } from "node:util"; + +const run = promisify(execFile); + +/** Where the built supervisor lands. One copy, reused by every file here. */ +const OUT = join(tmpdir(), "mountx-exec-test"); + +/** + * Why this host cannot run the supervisor, or `undefined` if it can. + * + * x86-64 Linux only, because the BPF filter compares against one syscall table + * and refuses to interpret any other — see `src/exec/seccomp/linux.zig`. + */ +export async function supervisorRefusal(): Promise { + if (process.platform !== "linux") { + return `seccomp user notification is Linux-only, and this is ${process.platform}`; + } + if (process.arch !== "x64") { + return `the filter is transcribed for x86-64, and this is ${process.arch}`; + } + try { + await run("zig", ["version"]); + } catch { + return "no zig on PATH, so the supervisor cannot be built"; + } + return undefined; +} + +/** One build per process, however many tests ask for it. */ +let building: Promise | undefined; + +/** + * Build `mountx-trace` and answer with its path. + * + * The build is `zig build-exe` with the two modules the binary is made of, the + * same command `test/exec/compare.sh` uses. Memoised because the conformance + * column asks once per case: Zig's own cache makes a repeat build cheap, but + * "cheap" is still a process spawn, and sixty-six of them is most of that + * file's runtime. + */ +export function buildSupervisor(): Promise { + building ??= build(); + return building; +} + +async function build(): Promise { + await mkdir(OUT, { recursive: true }); + const binary = join(OUT, "mountx-trace"); + await run( + "zig", + [ + "build-exe", + "-lc", + "-O", + "ReleaseSmall", + `-femit-bin=${binary}`, + "--dep", + "p9", + "-Mroot=seccomp/trace.zig", + "-Mp9=seccomp/p9.zig", + ], + { cwd: new URL("../../src/exec/", import.meta.url).pathname }, + ); + return binary; +} diff --git a/test/exec/seccomp-client.ts b/test/exec/seccomp-client.ts new file mode 100644 index 0000000..3f817fc --- /dev/null +++ b/test/exec/seccomp-client.ts @@ -0,0 +1,251 @@ +/** + * An `FsDriver` whose every call is a syscall made by a *traced process*. + * + * This is the other end of `seccomp-helper.ts`, and together they are what lets + * `test/conformance.ts` run unmodified as the supervisor's column of the + * matrix. A call arrives here, crosses a pipe as one line of NDJSON, becomes a + * `node:fs/promises` call in a process running under the seccomp filter, is + * trapped by the kernel, answered by the supervisor out of a 9P client against + * a `P9Session` over the driver under test, and comes back the same way. + * + * Nothing is short-circuited anywhere along that path — which is the point, and + * also why this column is slower than every other one. + * + * Not a `*.test.ts` file: it is imported by one. + */ + +import { Buffer } from "node:buffer"; +import type { ChildProcess } from "node:child_process"; +import { createInterface } from "node:readline"; +import { + S_IFBLK, + S_IFCHR, + S_IFDIR, + S_IFIFO, + S_IFLNK, + S_IFMT, + S_IFREG, + S_IFSOCK, +} from "../../src/types.ts"; +import type { + DirentLike, + FileHandleLike, + FsDriver, + MkdirOptions, + StatsFsLike, + StatsLike, + TimeLike, +} from "../../src/types.ts"; + +interface Reply { + i: number; + v?: unknown; + e?: { code: string; errno?: number; syscall?: string; message: string }; +} + +/** + * The pipe, with one request outstanding at a time. + * + * Serialized deliberately: the suite is written as a sequence of awaited calls, + * and pipelining them would test the helper's request ordering rather than the + * supervisor's behaviour. The supervisor's own concurrency is exercised + * elsewhere — `node:fs/promises` puts every one of these on a threadpool + * thread, so the notifications already arrive from a thread that is not the + * one that started the process. + */ +export class HelperLink { + #child: ChildProcess; + #pending = new Map< + number, + { resolve: (value: unknown) => void; reject: (error: unknown) => void } + >(); + #next = 1; + #closed: Error | undefined; + + constructor(child: ChildProcess) { + this.#child = child; + const stdout = child.stdout; + if (stdout === null || child.stdin === null) { + throw new Error("mountx: the helper needs piped stdio"); + } + const lines = createInterface({ input: stdout }); + lines.on("line", (line) => { + if (line.trim() === "") return; + const reply = JSON.parse(line) as Reply; + const waiter = this.#pending.get(reply.i); + if (waiter === undefined) return; + this.#pending.delete(reply.i); + if (reply.e === undefined) { + waiter.resolve(reply.v); + } else { + waiter.reject( + Object.assign(new Error(reply.e.message), { + code: reply.e.code, + errno: reply.e.errno, + syscall: reply.e.syscall, + }), + ); + } + }); + child.on("exit", () => { + this.#closed = new Error("mountx: the traced helper exited"); + for (const waiter of this.#pending.values()) waiter.reject(this.#closed); + this.#pending.clear(); + }); + } + + call(op: string, ...a: unknown[]): Promise { + if (this.#closed !== undefined) return Promise.reject(this.#closed); + const i = this.#next++; + return new Promise((resolve, reject) => { + this.#pending.set(i, { resolve, reject }); + this.#child.stdin?.write(`${JSON.stringify({ i, op, a })}\n`); + }); + } + + /** Ask the helper to leave, which is what ends the traced command. */ + finish(): void { + this.#child.stdin?.write(`${JSON.stringify({ i: 0, op: "exit", a: [] })}\n`); + this.#child.stdin?.end(); + } +} + +/** The seven predicates, rebuilt from the one number they all come from. */ +function typedBy(mode: number): Omit> { + const is = (bits: number): boolean => (mode & S_IFMT) === bits; + return { + isFile: () => is(S_IFREG), + isDirectory: () => is(S_IFDIR), + isSymbolicLink: () => is(S_IFLNK), + isBlockDevice: () => is(S_IFBLK), + isCharacterDevice: () => is(S_IFCHR), + isFIFO: () => is(S_IFIFO), + isSocket: () => is(S_IFSOCK), + } as Omit>; +} + +function toStats(value: Record): StatsLike { + return { ...value, ...typedBy(value.mode ?? 0) } as unknown as StatsLike; +} + +function toDirent(entry: { name: string; type: string }): DirentLike { + const is = (letter: string): boolean => entry.type === letter; + return { + name: entry.name, + isFile: () => is("f"), + isDirectory: () => is("d"), + isSymbolicLink: () => is("l"), + isBlockDevice: () => is("b"), + isCharacterDevice: () => is("c"), + isFIFO: () => is("p"), + isSocket: () => is("s"), + }; +} + +/** A `TimeLike` the far side can parse: `Date`s do not survive JSON as dates. */ +function toTime(value: TimeLike): number { + return value instanceof Date ? value.getTime() / 1000 : value; +} + +export function seccompDriver(link: HelperLink): FsDriver { + const handle = (id: number): FileHandleLike => ({ + async read(buffer, offset, length, position) { + const result = (await link.call( + "read", + id, + buffer.length, + offset ?? 0, + length ?? buffer.length, + position ?? null, + )) as { bytesRead: number; data: string }; + const bytes = Buffer.from(result.data, "base64"); + buffer.set(bytes.subarray(0, buffer.length)); + return { bytesRead: result.bytesRead, buffer }; + }, + async write(buffer, offset, length, position) { + const bytesWritten = (await link.call( + "write", + id, + Buffer.from(buffer).toString("base64"), + offset ?? 0, + length ?? buffer.length, + position ?? null, + )) as number; + return { bytesWritten, buffer }; + }, + async stat() { + return toStats((await link.call("fstat", id)) as Record); + }, + async truncate(length) { + await link.call("ftruncate", id, length ?? 0); + }, + async close() { + await link.call("close", id); + }, + async sync() { + await link.call("fsync", id); + }, + async datasync() { + await link.call("fdatasync", id); + }, + }); + + return { + async stat(path) { + return toStats((await link.call("stat", path)) as Record); + }, + async lstat(path) { + return toStats((await link.call("lstat", path)) as Record); + }, + async statfs(path) { + return (await link.call("statfs", path)) as StatsFsLike; + }, + async readdir(path) { + const entries = (await link.call("readdir", path)) as { name: string; type: string }[]; + return entries.map(toDirent); + }, + async open(path, flags, mode) { + const id = (await link.call("open", path, flags ?? "r", mode ?? 0o666)) as number; + return handle(id); + }, + async mkdir(path, options?: MkdirOptions) { + return ((await link.call("mkdir", path, options ?? {})) as string | null) ?? undefined; + }, + async rmdir(path) { + await link.call("rmdir", path); + }, + async unlink(path) { + await link.call("unlink", path); + }, + async rename(oldPath, newPath) { + await link.call("rename", oldPath, newPath); + }, + async link(existingPath, newPath) { + await link.call("link", existingPath, newPath); + }, + async symlink(target, path) { + await link.call("symlink", target, path); + }, + async readlink(path) { + return (await link.call("readlink", path)) as string; + }, + async chmod(path, mode) { + await link.call("chmod", path, mode); + }, + async chown(path, uid, gid) { + await link.call("chown", path, uid, gid); + }, + async lchown(path, uid, gid) { + await link.call("lchown", path, uid, gid); + }, + async truncate(path, length) { + await link.call("truncate", path, length ?? 0); + }, + async utimes(path, atime, mtime) { + await link.call("utimes", path, toTime(atime), toTime(mtime)); + }, + async lutimes(path, atime, mtime) { + await link.call("lutimes", path, toTime(atime), toTime(mtime)); + }, + }; +} diff --git a/test/exec/seccomp-conformance.test.ts b/test/exec/seccomp-conformance.test.ts new file mode 100644 index 0000000..48093e5 --- /dev/null +++ b/test/exec/seccomp-conformance.test.ts @@ -0,0 +1,124 @@ +/** + * The conformance matrix, seccomp-supervisor column. + * + * The organizing idea again: *one* suite written against the driver interface, + * run every way the library can carry it. `drivers.test.ts` runs it through the + * loopback harness, `test/9p/conformance.test.ts` through a whole 9P stack, + * `test/fuse/conformance-mount.test.ts` through a real kernel mount — and this + * file runs it through a **traced process**, with no mount anywhere. + * + * The path a single `fs.stat()` in this file takes: the suite calls the + * loopback, which calls the driver in `seccomp-client.ts`, which writes a line + * to a pipe; a `node` process on the far side of that pipe — running under a + * seccomp filter it installed on itself — makes the `statx(2)` call; the kernel + * suspends it and hands the supervisor a notification; the supervisor resolves + * the path, walks it over 9P against a `P9Session`, reads `Rgetattr`, writes a + * `struct statx` into the traced process's memory and answers the notification. + * Nothing is short-circuited at any step. + * + * Tier 2, and unusually for a Tier-2 file in this repository it needs **no + * root**: an unprivileged process may install a seccomp filter as long as it + * sets `no_new_privs`. What it does need is a Zig toolchain to build the + * supervisor, and x86-64 Linux, and it skips itself cleanly without either. + */ + +import { afterEach, describe, expect, it } from "vitest"; +import { createMemoryDriver } from "../../src/drivers/memory.ts"; +import { execSeccomp } from "../../src/exec/seccomp.ts"; +import { createLoopback, type ResolvedCapabilities } from "../../src/harness.ts"; +import { conformance } from "../conformance.ts"; +import { buildSupervisor, supervisorRefusal } from "./seccomp-build.ts"; +import { HelperLink, seccompDriver } from "./seccomp-client.ts"; + +const refusal = await supervisorRefusal(); + +/** + * What survives the trip through the syscall boundary. + * + * Declared rather than derived, because this is the column's *claim*. It is the + * 9P column's claim with nothing removed, which is the interesting part: the + * supervisor speaks 9P to the same `P9Session` the kernel's v9fs client speaks + * to, so everything that crosses one crosses the other. + * + * - **`handles: true`** — a `Tlopen` really opens the driver's file and the + * fid outlives an `unlink`, so a descriptor a traced process holds keeps + * working after the name is gone. The supervisor's own descriptor table + * holds the fid; the placeholder it injected has nothing in it. + * - **`extensions: []`**, as in every transport column: the `mountx.*` + * namespace is a driver-to-session channel with no wire representation. + * + * Nothing here is faked to make a case pass. The gaps this transport really + * has — `mmap` of a file on the tree, `execve` of a binary on it, extended + * attributes, `sendfile`/`splice`/`copy_file_range` — are all things the driver + * interface has no way to ask for, so the suite never asks. + */ +const THROUGH_SECCOMP: ResolvedCapabilities = { + handles: true, + atomicRename: true, + hardlinks: true, + symlinks: true, + permissions: true, + times: true, + truncate: true, + caseSensitive: true, + statfs: true, + readOnly: false, + extensions: [], +}; + +const helper = new URL("./seccomp-helper.ts", import.meta.url).pathname; + +/** Every traced process this file started, so a failure cannot leave one. */ +const running: (() => Promise)[] = []; + +afterEach(async () => { + for (const stop of running.splice(0)) await stop(); +}); + +describe.skipIf(refusal !== undefined)("through a seccomp-traced process", () => { + conformance({ + name: "memory driver, through the supervisor", + capabilities: THROUGH_SECCOMP, + setup: async () => { + const trace = await buildSupervisor(); + const driver = createMemoryDriver(); + let ready: (link: HelperLink) => void = () => {}; + const linked = new Promise((resolve) => { + ready = resolve; + }); + const exited = execSeccomp(driver, [process.execPath, helper], { + trace, + // stderr stays inherited: a helper that dies of something unexpected + // should say so rather than hang the suite in silence. + stdio: ["pipe", "pipe", "inherit"], + onSpawn: (child) => ready(new HelperLink(child)), + }); + const link = await linked; + const stop = async (): Promise => { + link.finish(); + await exited; + }; + running.push(stop); + return { fs: createLoopback(seccompDriver(link)), cleanup: stop }; + }, + }); + + it("really is running under a filter it cannot escape", async () => { + const trace = await buildSupervisor(); + let status = ""; + const result = await execSeccomp(createMemoryDriver(), ["sh", "-c", "cat /proc/self/status"], { + trace, + stdio: ["ignore", "pipe", "inherit"], + onSpawn: (child) => { + child.stdout?.on("data", (chunk: Buffer) => { + status += chunk.toString("utf8"); + }); + }, + }); + expect(result.code).toBe(0); + // Mode 2 is `SECCOMP_MODE_FILTER`, and `NoNewPrivs` is what an + // unprivileged process must set before it is allowed to install one. + expect(status).toMatch(/^Seccomp:\s*2$/m); + expect(status).toMatch(/^NoNewPrivs:\s*1$/m); + }); +}); diff --git a/test/exec/seccomp-helper.ts b/test/exec/seccomp-helper.ts new file mode 100644 index 0000000..38a3bd4 --- /dev/null +++ b/test/exec/seccomp-helper.ts @@ -0,0 +1,215 @@ +/** + * The tracee half of the conformance column: a filesystem REPL that runs + * **under the supervisor** and does what it is told with `node:fs/promises`. + * + * This is the piece that makes a conformance column possible at all. Every + * other column in this repository reaches its transport through a client + * library — `test/9p/client.ts` speaks 9P, `test/nfs/v4/client.ts` speaks + * COMPOUND — but the seccomp supervisor has no client: its interface *is* the + * syscall ABI, and the only way to drive it is to be a process making syscalls. + * So the driver under test lives on the other side of a pipe, in a process + * whose `openat`, `read`, `write` and `getdents64` are answered by the + * supervisor out of an `FsDriver`. + * + * `node:fs/promises` rather than the sync API on purpose: it is exactly what + * the loopback column of the matrix runs against, so anything this column + * disagrees about is a difference the supervisor introduced and not one the API + * has. It also means every request travels the libuv threadpool, which is a + * second tracee thread by construction — the multi-threaded case the supervisor + * has to key its tables on a thread *group* to survive. + * + * Requests and replies are NDJSON, one per line, and payloads are base64. Not a + * `*.test.ts` file: it is spawned by one. + */ + +import { Buffer } from "node:buffer"; +import type { Stats } from "node:fs"; +import * as fs from "node:fs/promises"; +import { createInterface } from "node:readline"; + +/** Where the driver appears in this process's namespace. */ +const root = process.env.MOUNTX_ROOT ?? "/mountx"; + +/** Absolute paths only, and never one that escapes the tree. */ +function at(path: string): string { + return path === "/" ? root : root + path; +} + +const handles = new Map(); +let nextHandle = 1; + +function need(id: number): fs.FileHandle { + const handle = handles.get(id); + if (handle === undefined) { + throw Object.assign(new Error("EBADF: no such handle"), { code: "EBADF", errno: -9 }); + } + return handle; +} + +/** A `Stats` flattened to what `StatsLike` needs, plus the mode to rebuild it. */ +function stats(value: Stats): Record { + return { + dev: Number(value.dev), + ino: Number(value.ino), + mode: Number(value.mode), + nlink: Number(value.nlink), + uid: Number(value.uid), + gid: Number(value.gid), + rdev: Number(value.rdev), + size: Number(value.size), + blksize: Number(value.blksize), + blocks: Number(value.blocks), + atimeMs: value.atimeMs, + mtimeMs: value.mtimeMs, + ctimeMs: value.ctimeMs, + birthtimeMs: value.birthtimeMs, + }; +} + +// eslint-disable-next-line complexity +async function perform(op: string, args: unknown[]): Promise { + // A fixed-length tuple rather than an array, so that indexing it is not + // `T | undefined` under `noUncheckedIndexedAccess`: the shape of each + // request's arguments is decided by its `op`, and the switch below is where + // that is checked. + const a = args as [never, never, never, never, never]; + switch (op) { + case "stat": + return stats(await fs.stat(at(a[0]))); + case "lstat": + return stats(await fs.lstat(at(a[0]))); + case "statfs": { + const value = await fs.statfs(at(a[0])); + return { + type: Number(value.type), + bsize: Number(value.bsize), + blocks: Number(value.blocks), + bfree: Number(value.bfree), + bavail: Number(value.bavail), + files: Number(value.files), + ffree: Number(value.ffree), + }; + } + case "readdir": { + const entries = await fs.readdir(at(a[0]), { withFileTypes: true }); + return entries.map((entry) => ({ + name: entry.name, + // The type bits, rebuilt on the far side. A `Dirent` cannot cross a + // pipe, and its seven predicates all come from one value. + type: entry.isDirectory() + ? "d" + : entry.isSymbolicLink() + ? "l" + : entry.isFile() + ? "f" + : entry.isBlockDevice() + ? "b" + : entry.isCharacterDevice() + ? "c" + : entry.isFIFO() + ? "p" + : entry.isSocket() + ? "s" + : "?", + })); + } + case "open": { + const handle = await fs.open(at(a[0]), a[1], a[2]); + const id = nextHandle++; + handles.set(id, handle); + return id; + } + case "close": { + const handle = need(a[0]); + handles.delete(a[0]); + await handle.close(); + return null; + } + case "read": { + const buffer = Buffer.alloc(a[1]); + const { bytesRead } = await need(a[0]).read(buffer, a[2], a[3], a[4]); + return { bytesRead, data: buffer.toString("base64") }; + } + case "write": { + const buffer = Buffer.from(a[1] as string, "base64"); + const { bytesWritten } = await need(a[0]).write(buffer, a[2], a[3], a[4]); + return bytesWritten; + } + case "fstat": + return stats(await need(a[0]).stat()); + case "ftruncate": + await need(a[0]).truncate(a[1]); + return null; + case "fsync": + await need(a[0]).sync(); + return null; + case "fdatasync": + await need(a[0]).datasync(); + return null; + case "mkdir": + return (await fs.mkdir(at(a[0]), a[1])) ?? null; + case "rmdir": + await fs.rmdir(at(a[0])); + return null; + case "unlink": + await fs.unlink(at(a[0])); + return null; + case "rename": + await fs.rename(at(a[0]), at(a[1])); + return null; + case "link": + await fs.link(at(a[0]), at(a[1])); + return null; + case "symlink": + // The target is opaque: it is stored as given, never rooted. + await fs.symlink(a[0], at(a[1])); + return null; + case "readlink": + return await fs.readlink(at(a[0])); + case "chmod": + await fs.chmod(at(a[0]), a[1]); + return null; + case "chown": + await fs.chown(at(a[0]), a[1], a[2]); + return null; + case "lchown": + await fs.lchown(at(a[0]), a[1], a[2]); + return null; + case "truncate": + await fs.truncate(at(a[0]), a[1]); + return null; + case "utimes": + await fs.utimes(at(a[0]), a[1], a[2]); + return null; + case "lutimes": + await fs.lutimes(at(a[0]), a[1], a[2]); + return null; + default: + throw new Error(`unknown op ${op}`); + } +} + +/** An error, flattened to the fields `node:fs`'s shape is made of. */ +function flatten(error: unknown): Record { + const value = error as { code?: string; errno?: number; syscall?: string; message?: string }; + return { + code: value.code ?? "EIO", + errno: value.errno, + syscall: value.syscall, + message: value.message ?? String(error), + }; +} + +const lines = createInterface({ input: process.stdin }); +for await (const line of lines) { + if (line.trim() === "") continue; + const request = JSON.parse(line) as { i: number; op: string; a: unknown[] }; + if (request.op === "exit") break; + let reply: string; + try { + reply = JSON.stringify({ i: request.i, v: await perform(request.op, request.a) }); + } catch (error) { + reply = JSON.stringify({ i: request.i, e: flatten(error) }); + } + process.stdout.write(`${reply}\n`); +} diff --git a/test/exec/seccomp-run.test.ts b/test/exec/seccomp-run.test.ts new file mode 100644 index 0000000..9f07789 --- /dev/null +++ b/test/exec/seccomp-run.test.ts @@ -0,0 +1,366 @@ +/** + * Tier 2 for the seccomp supervisor: real commands, really traced, against a + * driver that is not on any disk. + * + * The suite next door (`seccomp-conformance.test.ts`) drives the whole + * conformance matrix through one traced process and is the broader check. This + * file is the narrower and more literal one: it runs the shell, `cat`, `mv`, + * `chmod` and a statically linked binary with no libc calls at all, and then + * asserts on the **driver** rather than on what the command printed. + * + * That distinction is the whole point. The spike this replaces answered + * `openat` by slurping the file into a `memfd`, so a program's writes landed in + * a private copy that was thrown away: `dd conv=notrunc` reported success and + * changed nothing, `rm -f` reported success and the file was still there. Every + * command below that changes something is checked against the driver + * afterwards, because "the command exited 0" is exactly what that failure + * looked like. + * + * Skips itself with no Zig toolchain, off x86-64 Linux, or where a filter + * cannot be installed. It needs **no root**: an unprivileged process may + * install a seccomp filter as long as it sets `no_new_privs`, which is the + * property this whole mechanism exists for. + */ + +import { execFile } from "node:child_process"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { createMemoryDriver } from "../../src/drivers/memory.ts"; +import { execSeccomp } from "../../src/exec/seccomp.ts"; +import { createLoopback, type Loopback } from "../../src/harness.ts"; +import type { FsDriver } from "../../src/types.ts"; +import { buildSupervisor, supervisorRefusal } from "./seccomp-build.ts"; + +const run = promisify(execFile); +const refusal = await supervisorRefusal(); +const decoder = new TextDecoder(); + +describe.skipIf(refusal !== undefined)("a command traced by the seccomp supervisor", () => { + let trace = ""; + let scratch = ""; + + beforeAll(async () => { + trace = await buildSupervisor(); + scratch = await mkdtemp(join(tmpdir(), "mountx-exec-run-")); + }, 120_000); + + afterAll(async () => { + await rm(scratch, { recursive: true, force: true }); + }); + + /** A fresh driver with one file in it, and a loopback to inspect it with. */ + async function fixture(): Promise<{ driver: FsDriver; fs: Loopback }> { + const driver = createMemoryDriver(); + const fs = createLoopback(driver); + await fs.writeFile("/hello.txt", "hello from a driver\n"); + await fs.mkdir("/docs"); + await fs.writeFile("/docs/a.txt", "alpha\n"); + return { driver, fs }; + } + + /** Run `argv` under the supervisor and collect what it printed. */ + async function traced( + driver: FsDriver, + argv: readonly string[], + ): Promise<{ code: number | null; stdout: string; stderr: string }> { + let stdout = ""; + let stderr = ""; + const result = await execSeccomp(driver, argv, { + trace, + stdio: ["ignore", "pipe", "pipe"], + onSpawn: (child) => { + child.stdout?.on("data", (chunk: Buffer) => { + stdout += chunk.toString("utf8"); + }); + child.stderr?.on("data", (chunk: Buffer) => { + stderr += chunk.toString("utf8"); + }); + }, + }); + return { code: result.code, stdout, stderr }; + } + + const shell = (driver: FsDriver, script: string) => traced(driver, ["sh", "-c", script]); + + it("reads a file the driver holds", async () => { + const { driver } = await fixture(); + const result = await shell(driver, 'cat "$MOUNTX_ROOT/hello.txt"'); + expect(result.stderr).toBe(""); + expect(result.code).toBe(0); + expect(result.stdout).toBe("hello from a driver\n"); + }); + + it("lists a directory, with types", async () => { + const { driver } = await fixture(); + const result = await shell(driver, 'ls -1 "$MOUNTX_ROOT"; ls -d "$MOUNTX_ROOT/docs"'); + expect(result.stdout.split("\n").filter(Boolean).sort()).toEqual([ + "/mountx/docs", + "docs", + "hello.txt", + ]); + }); + + // --- the whole reason this file exists --------------------------------- + + it("a shell redirection reaches the driver", async () => { + const { driver, fs } = await fixture(); + const result = await shell(driver, 'echo written > "$MOUNTX_ROOT/new.txt"'); + expect(result.code).toBe(0); + expect(decoder.decode(await fs.readFile("/new.txt"))).toBe("written\n"); + }); + + it("appends rather than replacing", async () => { + const { driver, fs } = await fixture(); + await shell(driver, 'echo more >> "$MOUNTX_ROOT/hello.txt"'); + expect(decoder.decode(await fs.readFile("/hello.txt"))).toBe("hello from a driver\nmore\n"); + }); + + it("truncates on a plain redirection", async () => { + const { driver, fs } = await fixture(); + await shell(driver, 'echo short > "$MOUNTX_ROOT/hello.txt"'); + expect(decoder.decode(await fs.readFile("/hello.txt"))).toBe("short\n"); + }); + + it("writes in the middle of a file without disturbing the rest", async () => { + const { driver, fs } = await fixture(); + await fs.writeFile("/data", "0123456789"); + const result = await shell( + driver, + 'printf ab | dd of="$MOUNTX_ROOT/data" bs=1 seek=3 conv=notrunc 2>/dev/null', + ); + expect(result.code).toBe(0); + // The spike answered this one with a clean exit and an unchanged file. + expect(decoder.decode(await fs.readFile("/data"))).toBe("012ab56789"); + }); + + it("removes a file, and the driver agrees it is gone", async () => { + const { driver, fs } = await fixture(); + const result = await shell(driver, 'rm -f "$MOUNTX_ROOT/hello.txt"'); + expect(result.code).toBe(0); + await expect(fs.stat("/hello.txt")).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it("creates and removes directories", async () => { + const { driver, fs } = await fixture(); + await shell(driver, 'mkdir -p "$MOUNTX_ROOT/a/b/c" && rmdir "$MOUNTX_ROOT/a/b/c"'); + expect((await fs.stat("/a/b")).isDirectory()).toBe(true); + await expect(fs.stat("/a/b/c")).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it("renames", async () => { + const { driver, fs } = await fixture(); + await shell(driver, 'mv "$MOUNTX_ROOT/hello.txt" "$MOUNTX_ROOT/docs/moved.txt"'); + await expect(fs.stat("/hello.txt")).rejects.toMatchObject({ code: "ENOENT" }); + expect(decoder.decode(await fs.readFile("/docs/moved.txt"))).toBe("hello from a driver\n"); + }); + + it("changes permissions", async () => { + const { driver, fs } = await fixture(); + await shell(driver, 'chmod 600 "$MOUNTX_ROOT/hello.txt"'); + expect((await fs.stat("/hello.txt")).mode & 0o777).toBe(0o600); + }); + + it("creates and reads symlinks", async () => { + const { driver, fs } = await fixture(); + const result = await shell( + driver, + 'ln -s hello.txt "$MOUNTX_ROOT/link" && readlink "$MOUNTX_ROOT/link" && cat "$MOUNTX_ROOT/link"', + ); + expect(result.stdout).toBe("hello.txt\nhello from a driver\n"); + expect((await fs.lstat("/link")).isSymbolicLink()).toBe(true); + }); + + it("copies a whole subtree out of the tree", async () => { + const { driver } = await fixture(); + const target = join(scratch, "copied"); + const result = await shell( + driver, + `cp -r "$MOUNTX_ROOT/docs" ${target} && cat ${target}/a.txt`, + ); + expect(result.stderr).toBe(""); + expect(result.stdout).toBe("alpha\n"); + }); + + // --- the working directory --------------------------------------------- + + it("honours a working directory inside the tree", async () => { + const { driver, fs } = await fixture(); + const result = await shell( + driver, + 'cd "$MOUNTX_ROOT" && pwd && cat hello.txt && echo relative > made.txt && ls -1 .', + ); + expect(result.code).toBe(0); + expect(result.stdout).toBe("/mountx\nhello from a driver\ndocs\nhello.txt\nmade.txt\n"); + expect(decoder.decode(await fs.readFile("/made.txt"))).toBe("relative\n"); + }); + + it("stays inside the tree when a working directory outside it cannot be reached", async () => { + const { driver } = await fixture(); + // A `cd` that fails leaves a shell exactly where it was. A supervisor that + // forgot the virtual working directory on the *attempt* would resolve every + // relative path afterwards against somewhere else entirely. + const result = await shell( + driver, + 'cd "$MOUNTX_ROOT" && { cd /nonexistent-mountx-xyz 2>/dev/null || true; }; pwd; cat hello.txt; cd /tmp && pwd', + ); + expect(result.stdout).toBe("/mountx\nhello from a driver\n/tmp\n"); + }); + + it("resolves `..` inside the tree and clamps it at the root", async () => { + const { driver } = await fixture(); + const result = await shell( + driver, + 'cd "$MOUNTX_ROOT/docs" && cat ../hello.txt && cat ../../hello.txt', + ); + expect(result.stdout).toBe("hello from a driver\nhello from a driver\n"); + }); + + // --- more than one of everything --------------------------------------- + + it("serves several traced processes at once", async () => { + const { driver, fs } = await fixture(); + // Eight subshells, each its own process with its own descriptors, all + // writing at the same time. The supervisor answers notifications one at a + // time, so what this checks is that a *second* process's descriptors and + // working directory are not the first one's — which is what keying every + // table on the thread group behind `seccomp_notif.pid` is for. + const result = await shell( + driver, + 'cd "$MOUNTX_ROOT" && for i in 1 2 3 4 5 6 7 8; do (echo "worker $i" > "w$i.txt") & done; wait', + ); + expect(result.stderr).toBe(""); + for (let index = 1; index <= 8; index++) { + expect(decoder.decode(await fs.readFile(`/w${index}.txt`))).toBe(`worker ${index}\n`); + } + }); + + it("keeps a descriptor a child inherited working after the parent closes it", async () => { + const { driver } = await fixture(); + // The shape that breaks a supervisor which frees an open file the moment + // the last descriptor it *knows about* goes: the shell opens the file, + // forks, the child moves it onto its standard input and the parent lets go. + const result = await shell(driver, 'wc -c < "$MOUNTX_ROOT/hello.txt"'); + expect(result.stdout.trim()).toBe("20"); + }); + + it("walks a path deeper than one Twalk can carry", async () => { + const { driver, fs } = await fixture(); + const deep = Array.from({ length: 20 }, (_, index) => `d${index}`).join("/"); + const result = await shell( + driver, + `mkdir -p "$MOUNTX_ROOT/${deep}" && echo bottom > "$MOUNTX_ROOT/${deep}/f" && cat "$MOUNTX_ROOT/${deep}/f"`, + ); + // `P9_MAXWELEM` is 16, so this path takes two walks and the second one has + // to continue from where the first stopped. + expect(result.stdout).toBe("bottom\n"); + expect(decoder.decode(await fs.readFile(`/${deep}/f`))).toBe("bottom\n"); + }); + + it("survives more opens than any fid table could hold at once", async () => { + const { driver } = await fixture(); + const result = await shell( + driver, + 'cd "$MOUNTX_ROOT" && i=0; while [ $i -lt 400 ]; do echo $i > "f$i"; i=$((i+1)); done; cat f399; ls -1 | wc -l', + ); + expect(result.stdout).toBe("399\n402\n"); + }, 120_000); + + // --- the properties that make this mechanism worth having --------------- + + it("serves a statically linked binary that never calls libc", async () => { + const { driver, fs } = await fixture(); + await fs.writeFile("/payload.bin", "abcdefghij"); + const source = join(scratch, "raw.c"); + const binary = join(scratch, "raw"); + // No libc, no dynamic loader, nothing for an `LD_PRELOAD` interposer to + // interpose on: every one of these is a bare `syscall` instruction. This is + // the case the whole seccomp approach exists for. + await writeFile( + source, + ` +static long sys(long n, long a, long b, long c) { + long r; + __asm__ volatile("syscall" : "=a"(r) : "a"(n), "D"(a), "S"(b), "d"(c) : "rcx", "r11", "memory"); + return r; +} +/* The kernel enters at _start with the stack 16-byte aligned and argc on top, + which is not what a C function signature says; re-aligning here is what a + libc start file would otherwise do. */ +__asm__(".globl _start\\n_start:\\n xor %rbp, %rbp\\n and $-16, %rsp\\n call start_c\\n"); +void start_c(void) { + char buf[64]; + long fd = sys(2, (long) "/mountx/payload.bin", 0, 0); + long got = fd < 0 ? -1 : sys(0, fd, (long) buf, sizeof buf); + if (got > 0) sys(1, 1, (long) buf, got); + sys(231, got == 10 ? 0 : 9, 0, 0); + __builtin_unreachable(); +} +`, + ); + await run("zig", [ + "cc", + "-target", + "x86_64-linux-none", + "-nostdlib", + "-static", + "-ffreestanding", + "-fno-builtin", + "-O2", + source, + "-o", + binary, + ]); + const result = await traced(driver, [binary]); + expect(result.code).toBe(0); + expect(result.stdout).toBe("abcdefghij"); + }, 120_000); + + it("carries a payload bigger than one 9P message byte for byte", async () => { + const { driver, fs } = await fixture(); + const big = Buffer.allocUnsafe(3 * 1024 * 1024); + for (let index = 0; index < big.length; index++) big[index] = (index * 31 + 7) & 0xff; + await fs.writeFile("/big.bin", big); + const result = await shell( + driver, + 'cat "$MOUNTX_ROOT/big.bin" > "$MOUNTX_ROOT/copy.bin"; cmp "$MOUNTX_ROOT/big.bin" "$MOUNTX_ROOT/copy.bin" && echo identical', + ); + expect(result.stdout).toBe("identical\n"); + expect(Buffer.from(await fs.readFile("/copy.bin")).equals(big)).toBe(true); + }, 120_000); + + // --- the answers that must be errors rather than lies ------------------- + + it("refuses to map a file on the tree instead of mapping an empty one", async () => { + const { driver } = await fixture(); + // Without the refusal this reads as an all-zero file of the right length, + // which is the silent wrong answer the injected `memfd` used to give. + const result = await shell( + driver, + 'dd if="$MOUNTX_ROOT/hello.txt" of=/dev/null 2>/dev/null; echo done', + ); + expect(result.stdout).toBe("done\n"); + }); + + it("reports a missing file as ENOENT and leaves the exit status alone", async () => { + const { driver } = await fixture(); + const result = await shell(driver, 'cat "$MOUNTX_ROOT/nope" 2>&1; echo "status=$?"'); + expect(result.stdout).toMatch(/No such file or directory/); + expect(result.stdout).toMatch(/status=1/); + }); + + it("leaves everything outside the tree to the real filesystem", async () => { + const { driver } = await fixture(); + const outside = join(scratch, "outside.txt"); + await writeFile(outside, "on the real disk\n"); + const result = await shell(driver, `cat ${outside}`); + expect(result.stdout).toBe("on the real disk\n"); + }); + + it("passes the command's exit status through", async () => { + const { driver } = await fixture(); + expect((await shell(driver, "exit 42")).code).toBe(42); + }); +}); diff --git a/test/exec/seccomp.test.ts b/test/exec/seccomp.test.ts new file mode 100644 index 0000000..9a853ed --- /dev/null +++ b/test/exec/seccomp.test.ts @@ -0,0 +1,38 @@ +/** + * Tier 0 for `execSeccomp()` — the part that has an opinion before anything is + * built, spawned or trapped. + * + * Everything else about this transport needs a supervisor binary and a kernel + * that will install a filter, and lives in `seccomp-run.test.ts` and + * `seccomp-conformance.test.ts`, both of which skip themselves without a Zig + * toolchain. This file runs everywhere, including on a machine that could never + * run the mechanism at all. + */ + +import { describe, expect, it } from "vitest"; +import { createMemoryDriver } from "../../src/drivers/memory.ts"; +import { execSeccomp } from "../../src/exec/seccomp.ts"; + +describe("execSeccomp", () => { + it("refuses to run nothing", async () => { + await expect(execSeccomp(createMemoryDriver(), [], { trace: "/nonexistent" })).rejects.toThrow( + /needs a command/, + ); + }); + + it("says which supervisor it could not find", async () => { + const saved = process.env.MOUNTX_TRACE; + delete process.env.MOUNTX_TRACE; + try { + await expect(execSeccomp(createMemoryDriver(), ["true"])).rejects.toThrow(/MOUNTX_TRACE/); + } finally { + if (saved !== undefined) process.env.MOUNTX_TRACE = saved; + } + }); + + it("cleans up the private socket when the supervisor cannot be spawned", async () => { + await expect( + execSeccomp(createMemoryDriver(), ["true"], { trace: "/nonexistent/mountx-trace" }), + ).rejects.toMatchObject({ code: "ENOENT" }); + }); +}); diff --git a/test/exec/strategy.test.ts b/test/exec/strategy.test.ts new file mode 100644 index 0000000..a7b0094 --- /dev/null +++ b/test/exec/strategy.test.ts @@ -0,0 +1,168 @@ +/** + * Tier 0: what `mountx/exec` decides, and what it refuses to decide. + * + * Nothing here runs a command — `test/exec/userns.test.ts` does that. This is + * the choice itself: the preference order, the reasons a mechanism is ruled + * out, and the named-mechanism paths that must *not* consult the probe at all. + * + * The `platform`/`arch`/`supervisor` overrides are what make it a Tier-0 suite + * instead of a host-dependent one: darwin, win32 and arm64 are answered from + * any host, and only the assertions about *this* host's real capabilities are + * gated. + */ + +import { describe, expect, it } from "vitest"; +import { createMemoryDriver } from "../../src/drivers/memory.ts"; +import { exec, probeExec } from "../../src/exec/index.ts"; +import { seccompExecProbe, usernsExecProbe } from "../../src/exec/probe.ts"; +import { cwdRefusal } from "../../src/exec/userns.ts"; + +const here = probeExec(); + +describe("probeExec", () => { + it("prefers the user namespace, then seccomp — one order on every host", () => { + // There is no second order to have: off Linux neither mechanism can work, + // so nothing is decided by what follows what. + expect(here.preference).toEqual(["userns", "seccomp"]); + expect(probeExec("darwin").preference).toEqual(["userns", "seccomp"]); + }); + + it("rules both out on macOS, each in its own words", () => { + const probe = probeExec("darwin"); + expect(probe.chosen).toBeUndefined(); + expect(probe.userns.usable).toBe(false); + expect(probe.seccomp.usable).toBe(false); + // Not one shared "needs Linux": a reader on macOS is entitled to know that + // macFUSE does not help with the first and that nothing at all plays + // seccomp's role for the second. + expect(probe.userns.reason).toContain("macFUSE"); + expect(probe.seccomp.reason).toContain("Linux facility"); + // One sentence naming both, rather than whichever failed last. + expect(probe.reason).toContain("user namespace:"); + expect(probe.reason).toContain("seccomp:"); + expect(probe.reason).toContain("darwin"); + }); + + it("rules both out on Windows too", () => { + const probe = probeExec("win32"); + expect(probe.chosen).toBeUndefined(); + expect(probe.reason).toContain("win32"); + }); + + it("always pairs usability with a reason, and a choice with a usable mechanism", () => { + for (const probe of [here.userns, here.seccomp]) { + expect(probe.usable).toBe(probe.reason === undefined); + } + expect(here.platform).toBe(process.platform); + if (here.chosen === undefined) { + expect(here.reason).toBeTruthy(); + } else { + expect(here.reason).toBeUndefined(); + expect(here[here.chosen].usable).toBe(true); + // The choice is the first usable one in preference order, not any usable one. + expect(here.preference.find((mechanism) => here[mechanism].usable)).toBe(here.chosen); + } + }); + + it("delegates each question to the mechanism's own probe rather than re-deciding", () => { + // Same probes `src/exec/probe.ts` publishes, not a second opinion — which + // is what keeps "what does the user-namespace mechanism need" one fact in + // one file. + expect(here.userns).toEqual(usernsExecProbe()); + expect(here.seccomp).toEqual(seccompExecProbe()); + }); + + it("honours a supervisor the caller names, and refuses one that is not there", () => { + // The supervisor is not shipped in the package, so "where is it" is a real + // input to the decision and not a host fact this file can read. + expect(seccompExecProbe("linux", "x64", undefined).reason).toContain("$MOUNTX_TRACE"); + expect(seccompExecProbe("linux", "x64", "/definitely/not/here").reason).toContain( + "not an executable file", + ); + expect(seccompExecProbe("linux", "x64", "/definitely/not/here").supervisor).toBeUndefined(); + }); + + it("rules seccomp out on an architecture its filter was not written for", () => { + const probe = seccompExecProbe("linux", "arm64", "/bin/sh"); + expect(probe.usable).toBe(false); + expect(probe.arch).toBe(false); + // And says what it would take, because "a second syscall table" is a + // different size of job from "a redesign". + expect(probe.reason).toContain("x86-64"); + expect(probe.reason).toContain("arm64"); + }); + + it("refuses to answer for Linux from a host that is not Linux", () => { + // The `platform` override takes away every file the probe reads, so a + // verdict of `usable` would rest on checks that never ran. Only asserted + // off Linux, where it is the case that exists. + if (process.platform === "linux") { + expect(usernsExecProbe("linux").reason ?? "").not.toContain("not Linux"); + return; + } + expect(usernsExecProbe("linux").usable).toBe(false); + expect(seccompExecProbe("linux", "x64", "/bin/sh").usable).toBe(false); + }); +}); + +describe("cwdRefusal", () => { + it("refuses a cwd that is the mountpoint, or under it", () => { + // The witnessed deadlock, refused up front: `uv_spawn` blocks the thread + // that answers FUSE until the child execs, and the child's first act is the + // `chdir` that needs an answer from it. + expect(cwdRefusal("/mnt/x", "/mnt/x")).toContain("deadlocks"); + expect(cwdRefusal("/mnt/x/deep/er", "/mnt/x")).toContain("deadlocks"); + // And names the spelling that works, since the caller does want to be in there. + expect(cwdRefusal("/mnt/x", "/mnt/x")).toContain("$MOUNTX_ROOT"); + }); + + it("allows everything else, including a path that merely starts the same way", () => { + expect(cwdRefusal("/tmp", "/mnt/x")).toBeUndefined(); + expect(cwdRefusal("/mnt", "/mnt/x")).toBeUndefined(); + // `/mnt/xy` is not inside `/mnt/x`; a prefix match with no separator would + // say it was. + expect(cwdRefusal("/mnt/xy", "/mnt/x")).toBeUndefined(); + }); + + it("compares resolved paths, not the strings it was handed", () => { + expect(cwdRefusal("/mnt/x/../x/sub", "/mnt/x")).toContain("deadlocks"); + expect(cwdRefusal("/mnt/x/..", "/mnt/x")).toBeUndefined(); + }); +}); + +describe("exec", () => { + it("needs a command", async () => { + await expect(exec(createMemoryDriver(), [])).rejects.toThrow(/needs a command to run/); + }); + + it.skipIf(here.chosen !== undefined)( + "refuses with both mechanisms' reasons when neither can run", + async () => { + await expect(exec(createMemoryDriver(), ["true"])).rejects.toThrow( + /no mechanism can run a command with a driver on this host/, + ); + }, + ); + + it.skipIf(here.seccomp.usable)("lets a named mechanism fail in its own words", async () => { + // The assertion is the *absence* of the picker's sentence: naming a + // mechanism skips the probe, so whatever comes back is the mechanism's own + // and is more specific than anything `probeExec` could have said. Which + // sentence exactly is `src/exec/seccomp.ts`'s business, not this file's. + const failure = await exec(createMemoryDriver(), ["true"], { mechanism: "seccomp" }).then( + () => undefined, + (error: unknown) => error as Error, + ); + expect(failure).toBeInstanceOf(Error); + expect(failure?.message).not.toContain("no mechanism can run a command"); + }); + + it.skipIf(here.userns.usable)( + "lets a named userns mechanism fail in its own words too", + async () => { + await expect(exec(createMemoryDriver(), ["true"], { mechanism: "userns" })).rejects.toThrow( + /cannot run a command in a user namespace here/, + ); + }, + ); +}); diff --git a/test/exec/userns.test.ts b/test/exec/userns.test.ts new file mode 100644 index 0000000..e076451 --- /dev/null +++ b/test/exec/userns.test.ts @@ -0,0 +1,205 @@ +/** + * Tier 2, without root — a real namespace, a real FUSE mount, a real command. + * + * ```sh + * pnpm test:rootless + * ``` + * + * Everything here needs `/dev/fuse`, unprivileged user namespaces and + * `unshare(1)`, and skips itself when it does not have them, so `pnpm test` + * stays green on a host that has none of it. It deliberately does **not** + * re-test the filesystem: what the command sees is FUSE, and + * `test/fuse/conformance-mount.test.ts` already covers that whole column. What + * is left to prove is everything that *is* different — that the mount is + * private to the child, that `$MOUNTX_ROOT` is where the driver actually is, + * that a relay failure reaches the caller as an error rather than as the + * command's exit status, and that nothing survives the call. + * + * The threadpool gate is the same one `test/fuse/mount-rootless.test.ts` uses, + * and for a *weaker* reason: the read loop on `/dev/fuse` lives in the relay, + * a separate process, so this suite is not on both sides of its own mount the + * way that one is. What it does share is a `FuseSession` answering from this + * process while vitest runs other files' `fs` work alongside, so it keeps the + * same discipline rather than inventing a second rule. + */ + +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { createMemoryDriver } from "../../src/drivers/memory.ts"; +import { exec } from "../../src/exec/index.ts"; +import { usernsExecProbe } from "../../src/exec/probe.ts"; +import { execUserns } from "../../src/exec/userns.ts"; +import { createLoopback } from "../../src/harness.ts"; +import type { FsDriver } from "../../src/types.ts"; + +const probe = usernsExecProbe(); + +/** Has someone raised the threadpool for us? Same gate as the FUSE Tier-2 files. */ +const POOL = Number.parseInt(process.env.UV_THREADPOOL_SIZE ?? "", 10); +const roomToRun = Number.isFinite(POOL) && POOL >= 8; +const live = probe.usable && roomToRun; + +/** Real mounts are slow enough that vitest's 5s default is a coin flip. */ +const SLOW = 60_000; + +/** A driver with one obvious file in it, plus a directory. */ +async function demo(): Promise { + const driver = createMemoryDriver(); + const fs = createLoopback(driver); + await fs.mkdir("/docs", { recursive: true }); + await fs.writeFile("/hello.txt", "hello from a driver that is not on any disk\n"); + await fs.writeFile("/docs/a.txt", "alpha\n"); + return driver; +} + +describe.skipIf(!live)("execUserns", () => { + const directories: string[] = []; + + afterEach(async () => { + for (const directory of directories.splice(0)) { + await rm(directory, { recursive: true, force: true }).catch(() => {}); + } + }); + + async function scratch(): Promise { + const path = await mkdtemp(join(tmpdir(), "mountx-exec-test-")); + directories.push(path); + return path; + } + + it( + "serves the driver to the command at $MOUNTX_ROOT", + async () => { + const out = join(await scratch(), "out"); + // Everything the command touches on the *host* is `out`; everything it + // reads comes from the driver. `sh -c` so the `cd` happens after the + // exec — a `cwd` into the mount is the one thing that deadlocks. + const ran = await execUserns(await demo(), [ + "sh", + "-c", + `cd "$MOUNTX_ROOT" && cat hello.txt docs/a.txt > ${out} && ls >> ${out}`, + ]); + expect(ran.code).toBe(0); + expect(ran.signal).toBeNull(); + const text = await readFile(out, "utf8"); + expect(text).toContain("hello from a driver that is not on any disk"); + expect(text).toContain("alpha"); + expect(text).toContain("docs"); + }, + SLOW, + ); + + it( + "writes back through the driver, not to the host", + async () => { + const driver = await demo(); + const ran = await execUserns(driver, [ + "sh", + "-c", + 'printf again > "$MOUNTX_ROOT/written.txt"', + ]); + expect(ran.code).toBe(0); + // The driver kept it, and it never existed anywhere a host path could + // reach — the mountpoint itself is gone by now. + const written = await createLoopback(driver).readFile("/written.txt"); + expect(Buffer.from(written).toString("utf8")).toBe("again"); + }, + SLOW, + ); + + it( + "mounts where it was told, and leaves nothing behind", + async () => { + const mountpoint = join(await scratch(), "deep", "mnt"); + const out = join(await scratch(), "root"); + // A mountpoint that does not exist yet: created here, recursively, rather + // than failing three processes away inside `mount(8)`. + const ran = await execUserns(await demo(), ["sh", "-c", `printf %s "$MOUNTX_ROOT" > ${out}`]); + expect(ran.code).toBe(0); + expect(ran.mountpoint).toMatch(/^\//); + + const named = await execUserns( + await demo(), + ["sh", "-c", `printf %s "$MOUNTX_ROOT" > ${out}`], + { + mountpoint, + }, + ); + expect(named.mountpoint).toBe(mountpoint); + expect(await readFile(out, "utf8")).toBe(mountpoint); + // Back to an ordinary empty directory: the mount died with the namespace. + await writeFile(join(mountpoint, "still-a-directory"), ""); + }, + SLOW, + ); + + it( + "keeps the mount out of the host's mount table", + async () => { + // The property the whole mechanism exists for. Read from *this* process, + // which is outside the namespace and is where a leak would show up. + const before = await readFile("/proc/self/mounts", "utf8"); + const ran = await execUserns(await demo(), ["sh", "-c", 'test -f "$MOUNTX_ROOT/hello.txt"']); + expect(ran.code).toBe(0); + const after = await readFile("/proc/self/mounts", "utf8"); + expect(after).toBe(before); + expect(after).not.toContain(ran.mountpoint); + }, + SLOW, + ); + + it( + "hands back the command's own status rather than swallowing it", + async () => { + // A command that fails is not an error here, exactly as it is not for + // `child_process` — which is what makes the *thrown* errors mean + // "the mechanism could not be set up" and nothing else. + const ran = await execUserns(await demo(), ["sh", "-c", "exit 42"]); + expect(ran.code).toBe(42); + expect(ran.signal).toBeNull(); + }, + SLOW, + ); + + it( + "turns a relay failure into an error instead of a plausible exit status", + async () => { + // The relay exits 70 when it gives up, and a command exiting 70 is a + // thing that happens; the status file is what tells the two apart. + await expect( + execUserns(await demo(), ["definitely-not-a-binary-on-this-host"]), + ).rejects.toThrow(/could not run definitely-not-a-binary-on-this-host/); + }, + SLOW, + ); + + it( + "refuses a cwd inside the mount rather than deadlocking on it", + async () => { + const mountpoint = join(await scratch(), "mnt"); + await expect( + execUserns(await demo(), ["true"], { mountpoint, cwd: mountpoint }), + ).rejects.toThrow(/deadlocks rather than failing/); + }, + SLOW, + ); +}); + +describe.skipIf(!live)("exec", () => { + it( + "picks the user namespace on a host with a working /dev/fuse, and tags the result", + async () => { + const ran = await exec(await demo(), ["sh", "-c", 'cat "$MOUNTX_ROOT/hello.txt"']); + expect(ran.mechanism).toBe("userns"); + expect(ran.code).toBe(0); + // The tag is a discriminant over the mechanism's *own* result object, so + // narrowing on it reaches `mountpoint` with no cast. + if (ran.mechanism === "userns") { + expect(ran.mountpoint).toMatch(/^\//); + } + }, + SLOW, + ); +});