Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 9 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,25 @@ Changelog tracking starts with 0.2.0. Prior versions were not tracked.
### Added

- **`astrid:fs/host.fs-mkdir-all`** — unstubbed. Idempotent recursive directory creation via the existing VFS `mkdir` call (every VFS impl already routes through `std::fs::create_dir_all` under the hood). Capability gating, audit envelope (`astrid:fs/host.fs-mkdir-all`), and error mapping match `fs-mkdir`. Unblocks capsule code that wants the `std::fs::create_dir_all`-style idempotent variant instead of the strict `fs-mkdir`. (Closes one item of #753.)
- **Outbound TCP for capsules — `net.connect-tcp` host fn + `net_connect` capability.** Capsules can now open persistent TCP connections via the new `astrid:capsule/net.net-connect-tcp(host, port) -> stream-handle` host fn, gated by a per-capsule `net_connect = ["host:port", "host:*"]` allowlist in `Capsule.toml`. The returned handle flows through the existing `net-read` / `net-write` / `net-close-stream` plumbing, and the kernel reuses the same `is_safe_ip` airlock that gates `http-request` to reject loopback / private / link-local / multicast IPs after DNS resolution. Connect timeout bounded to 10s; per-capsule active-stream cap (`MAX_ACTIVE_STREAMS = 8`) shared with inbound `net-accept`. Unblocks WebSocket clients, MQTT, Discord/Telegram gateways, postgres/redis, and the immediate motivator: a Unicity-network capsule wrapping Sphere SDK (Fulcrum + Nostr WebSocket transports). Tracking issue: #745. RFC: [rfcs#27](https://github.com/unicity-astrid/rfcs/pull/27). WIT contract: [wit#5](https://github.com/unicity-astrid/wit/pull/5).
- **`NetStream` enum (Unix + Tcp)** in `engine::wasm::host_state` — replaces the bare `Arc<Mutex<UnixStream>>` value type in `active_streams`. The `net_read` / `net_write` dispatchers match on the variant; the inner framing (`read_frame` / `write_frame` generic helpers) is shared via `tokio::io::AsyncRead + AsyncWrite` trait bounds. Single-variant capsules see no behavior change.
- **`CapsuleSecurityGate::check_net_connect(capsule_id, host, port)`** — new trait method, default-deny. `ManifestSecurityGate` implements it by matching the requested `host:port` against the manifest's `net_connect` allowlist (case-insensitive host, exact-or-`*` port).
- **`astrid-build` is target-agnostic now.** Drops the hardcoded `--target wasm32-wasip2` flag on `cargo build`; instead lets the capsule's own `.cargo/config.toml` select the target. After compilation, probes `target/wasm32-unknown-unknown/release/` first, `target/wasm32-wasip2/release/` second (and the workspace root) for the produced `.wasm`. When the produced artifact is a core wasm module (no Component Model layer = 1 in the magic bytes) — which is what `wasm32-unknown-unknown` produces — `wit_component::ComponentEncoder::default().validate(true).module(&core).encode()` wraps it into a Component Model component. Required because the Astrid-canonical guest target is `wasm32-unknown-unknown` (zero `wasi:*` imports), and `cargo` does not produce a component directly for that target.
- **`chrono = { default-features = false, features = ["serde"] }`** workspace-wide. The `clock` feature is what pulls `wasm-bindgen` + `js-sys` on `wasm32-unknown-unknown`, which then inject `__wbindgen_placeholder__` imports that `wit-component`'s encoder refuses to round-trip. Records that need a clock value receive it from `astrid_sdk::time` (audited host fn); `DateTime<Utc>` is only used as a serializable field shape, never constructed via `Utc::now()` in capsule code.
- **`astrid-build`'s `ensure_component` overwrites the original `.wasm` artifact** instead of writing a sibling `.component.wasm`. Capsule `Capsule.toml [[component]] file = "<crate>.wasm"` directives keep resolving without per-target conditional logic — the toolchain hides which target produced the artifact.
- **`astrid-types` `clock` feature.** Default OFF — gates `IpcMessage::new()` and the `#[serde(default = "Utc::now")]` timestamp default behind a Cargo feature. Kernel-side consumers (`astrid-events`, the daemon) enable it via their dep declaration. Capsule-side consumers (via `astrid-sdk` on `wasm32-unknown-unknown`) leave it off; absent timestamps fall back to the Unix epoch via a `from_timestamp(0,0)` default. Capsule code reads timestamps from kernel-published messages, never constructs them.
- **`chrono = { default-features = false, features = ["serde"] }`** workspace-wide. The `clock` feature is what pulls `wasm-bindgen` + `js-sys` on `wasm32-unknown-unknown`, which then inject `__wbindgen_placeholder__` imports that `wit-component`'s encoder refuses to round-trip. Records that need a clock value receive it from `astrid_sdk::time` (audited host fn); `DateTime<Utc>` is only used as a serializable field shape, never constructed via `Utc::now()` in capsule code.
- **`uuid = { default-features = false, features = ["v4", "v5", "serde", "rng-getrandom"] }`** workspace-wide. The default features pull a `js`-based RNG on `wasm32-unknown-unknown` (via `wasm-bindgen`). `rng-getrandom` routes v4 generation through `getrandom`, which `astrid-sys` configures with a custom backend (`astrid:sys.random-bytes`) on capsule builds. Same dep wiring in `sdk-rust`'s `astrid-sdk`.
- **`astrid-build`'s `ensure_component` overwrites the original `.wasm` artifact** instead of writing a sibling `.component.wasm`. Capsule `Capsule.toml [[component]] file = "<crate>.wasm"` directives keep resolving without per-target conditional logic — the toolchain hides which target produced the artifact.

### Changed

- **Kernel exposes zero `wasi:*`.** `configure_kernel_linker` no longer registers `wasmtime_wasi::p2::add_to_linker_sync`. The Astrid-canonical guest target (`wasm32-unknown-unknown`) produces wasm with zero `wasi:*` imports; a capsule that somehow ships with a `wasi:*` import fails to instantiate at load time with a clear "interface not found" error — the intended posture, not a bug. Capsules that historically targeted `wasm32-wasip2` and relied on auto-injected `wasi:*` imports will fail to load until they migrate to `wasm32-unknown-unknown` via the upcoming SDK + capsule sweep (a separate PR cluster, blocked on this one landing first).
- **`crates/astrid-capsule/src/manifest.rs` split into a `manifest/` submodule.** The 1000-line single file became `manifest/mod.rs` + `manifest/capabilities.rs` + `manifest/topics.rs`. `CapabilitiesDef`, `PublishDef`, `SubscribeDef` (with their custom deserializers and TOML parsing tests) live in dedicated submodules; the top-level `manifest::*` public API is preserved via `pub use` re-exports — no consumer-side change required.
- **`astrid-storage::kv` split into a submodule directory.** `kv.rs` (now ~1200 lines after the CAS addition) became `kv/mod.rs` (validators, helpers, trait, re-exports) + `kv/memory.rs` (`MemoryKvStore`) + `kv/surreal.rs` (`SurrealKvStore`, behind the `kv` feature) + `kv/scoped.rs` (`ScopedKvStore`), each well under the 1000-line CI ceiling. Public API preserved verbatim via `pub use`.
- **`wit/astrid-capsule.wit` resynced from canonical `unicity-astrid/wit`** to pick up the new `net-connect-tcp` fn and the `ipc-message.principal` field (canonical PR #4 from May; was missing from the in-tree copy). Internal `IpcMessage → WitIpcMessage` conversion now forwards `principal`.

### Fixed

- **`TcpStream::read_bytes` / `peek` return `Err(ErrorCode::Closed)` on cancellation.** Previously both methods collapsed cancel to `Ok(Vec::new())`, which is indistinguishable from a clean EOF in byte-stream reads. Capsules with EOF finalizers (write trailers, send last-message IPC, flush logs) would execute those finalizers under a forced unload. Now matches `read_frame` / `write_bytes` / `shutdown` — Closed surfaces cancellation as its own signal. Empty Vec retains its "clean EOF" meaning.
- **Per-domain WIT review fixups (PR #752).** A multi-agent review surfaced fixes addressed in-branch before merge:
- `ipc::recv` mixed-principal batches are now truncated at the first publisher boundary so tail messages can't be silently mis-stamped with the head's principal context; new `truncate_to_homogeneous_principal` unit tests cover the boundary cases.
- `TcpStream::write` surfaces peer-disconnect IO kinds as `ErrorCode::ConnectionReset` instead of swallowing them as `Ok(())`; pure-function tests pin the `BrokenPipe / ConnectionReset / ConnectionAborted / UnexpectedEof → ConnectionReset` mapping and use `tokio::io::duplex` to reproduce the live close.
Expand All @@ -43,18 +50,6 @@ Changelog tracking starts with 0.2.0. Prior versions were not tracked.
- `MAX_ACTIVE_STREAMS` / `MAX_SUBSCRIPTIONS` / `MAX_BACKGROUND_PROCESSES` quota gates now read O(1) counter fields on `HostState` (`net_stream_count`, `subscription_count`, `process_count_total`, `process_count_by_principal`) instead of walking the entire `ResourceTable`. Each successful resource insert bumps the counter; the matching `drop` impl decrements. Per-principal sub-budgets for `spawn-background` use a `HashMap<PrincipalId, usize>` keyed on the creator. Single-threaded: wasmtime stores are owned by exactly one OS thread.
- CI workflows (`ci.yml`) now check out the `wit/` submodule recursively so `astrid-capsule`'s build script can stage the per-domain WIT packages. Previously every job that touched the build (`check`, `clippy`, `test`, `msrv`) panicked with `read wit/host: No such file or directory`.
- **Atomic `kv_cas` across capsules.** The host fn used to emulate compare-and-swap with a `get` followed by a `set`, which raced across capsules running concurrently in the same kernel. `KvStore` now exposes an atomic `compare_and_swap(namespace, key, expected, new)` primitive; `MemoryKvStore` serializes read+conditional-write under its existing `RwLock`, and `SurrealKvStore` runs the comparison inside a single MVCC transaction guarded by an internal mutex that closes the TOCTOU between surrealkv's `validate_write_conflicts` and its actual write phase. Two concurrent-spawn regression tests (memory + surreal) assert exactly one of 16–32 racing tasks succeeds the swap. The capsule `kv_cas` host fn delegates straight to the new primitive — no more emulation.
- **`astrid-storage::kv` split into a submodule directory.** `kv.rs` (now ~1200 lines after the CAS addition) became `kv/mod.rs` (validators, helpers, trait, re-exports) + `kv/memory.rs` (`MemoryKvStore`) + `kv/surreal.rs` (`SurrealKvStore`, behind the `kv` feature) + `kv/scoped.rs` (`ScopedKvStore`), each well under the 1000-line CI ceiling. Public API preserved verbatim via `pub use`.

### Added

- **Outbound TCP for capsules — `net.connect-tcp` host fn + `net_connect` capability.** Capsules can now open persistent TCP connections via the new `astrid:capsule/net.net-connect-tcp(host, port) -> stream-handle` host fn, gated by a per-capsule `net_connect = ["host:port", "host:*"]` allowlist in `Capsule.toml`. The returned handle flows through the existing `net-read` / `net-write` / `net-close-stream` plumbing, and the kernel reuses the same `is_safe_ip` airlock that gates `http-request` to reject loopback / private / link-local / multicast IPs after DNS resolution. Connect timeout bounded to 10s; per-capsule active-stream cap (`MAX_ACTIVE_STREAMS = 8`) shared with inbound `net-accept`. Unblocks WebSocket clients, MQTT, Discord/Telegram gateways, postgres/redis, and the immediate motivator: a Unicity-network capsule wrapping Sphere SDK (Fulcrum + Nostr WebSocket transports). Tracking issue: #745. RFC: [rfcs#27](https://github.com/unicity-astrid/rfcs/pull/27). WIT contract: [wit#5](https://github.com/unicity-astrid/wit/pull/5).
- **`NetStream` enum (Unix + Tcp)** in `engine::wasm::host_state` — replaces the bare `Arc<Mutex<UnixStream>>` value type in `active_streams`. The `net_read` / `net_write` dispatchers match on the variant; the inner framing (`read_frame` / `write_frame` generic helpers) is shared via `tokio::io::AsyncRead + AsyncWrite` trait bounds. Single-variant capsules see no behavior change.
- **`CapsuleSecurityGate::check_net_connect(capsule_id, host, port)`** — new trait method, default-deny. `ManifestSecurityGate` implements it by matching the requested `host:port` against the manifest's `net_connect` allowlist (case-insensitive host, exact-or-`*` port).

### Changed

- **`crates/astrid-capsule/src/manifest.rs` split into a `manifest/` submodule.** The 1000-line single file became `manifest/mod.rs` + `manifest/capabilities.rs` + `manifest/topics.rs`. `CapabilitiesDef`, `PublishDef`, `SubscribeDef` (with their custom deserializers and TOML parsing tests) live in dedicated submodules; the top-level `manifest::*` public API is preserved via `pub use` re-exports — no consumer-side change required.
- **`wit/astrid-capsule.wit` resynced from canonical `unicity-astrid/wit`** to pick up the new `net-connect-tcp` fn and the `ipc-message.principal` field (canonical PR #4 from May; was missing from the in-tree copy). Internal `IpcMessage → WitIpcMessage` conversion now forwards `principal`.

## [0.6.0] - 2026-05-19

Expand Down
25 changes: 20 additions & 5 deletions crates/astrid-capsule/src/engine/wasm/host/net/tcp_stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,15 @@ impl HostTcpStream for HostState {
Some(Ok(v)) => Ok(v),
Some(Err(e)) if e == "read would block" => Err(ErrorCode::WouldBlock),
Some(Err(e)) => Err(ErrorCode::Unknown(e)),
None => Ok(Vec::new()),
// Cancellation collapses to `Closed` rather than an empty
// Vec, mirroring `read_frame` / `write_bytes` / `shutdown`.
// An empty Vec is the conventional "clean EOF" signal in
// byte-stream reads; returning that on a forced unload
// would silently look like the peer closed gracefully and
// cause capsules that finalize on EOF (write trailers,
// send last-message IPC) to run those finalizers under a
// tear-down. Closed distinguishes the two.
None => Err(ErrorCode::Closed),
};
let bytes = result.as_ref().map(|v| v.len() as u64).unwrap_or(0);
audit_net(
Expand Down Expand Up @@ -184,10 +192,10 @@ impl HostTcpStream for HostState {
let sem = self.host_semaphore.clone();
let tok = self.cancel_token.clone();
let max = (max_bytes as usize).min(MAX_BYTES_PER_CALL);
match stream {
let result: Result<Vec<u8>, ErrorCode> = match stream {
NetStream::Tcp(slot) => {
let timeout = slot.read_timeout;
let result = util::bounded_block_on_cancellable(&rt, &sem, &tok, async move {
let opt = util::bounded_block_on_cancellable(&rt, &sem, &tok, async move {
let s = slot.stream.lock().await;
let mut buf = vec![0u8; max];
let fut = s.peek(&mut buf);
Expand All @@ -202,10 +210,17 @@ impl HostTcpStream for HostState {
buf.truncate(n);
Ok(buf)
});
result.unwrap_or(Ok(Vec::new()))
// Same reasoning as `read_bytes`: an empty Vec is "no
// data peeked yet, peer still connected," which is
// indistinguishable from a clean cancel. Surface Closed
// on cancellation so capsules can distinguish.
opt.unwrap_or(Err(ErrorCode::Closed))
},
NetStream::Unix(_) => Err(ErrorCode::NotTcp),
}
};
let bytes = result.as_ref().map(|v| v.len() as u64).unwrap_or(0);
audit_net(self, "astrid:net/host.tcp-stream.peek", bytes, &result);
result
}

fn shutdown(&mut self, self_: Resource<TcpStream>, how: ShutdownHow) -> Result<(), ErrorCode> {
Expand Down
Loading