Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
<PackageVersion Include="SlackNet.Extensions.DependencyInjection" Version="$(SlackNetVersion)" />
<PackageVersion Include="Cronos" Version="0.13.0" />
<PackageVersion Include="Netclaw.SkillClient" Version="0.4.0" />
<PackageVersion Include="ShellSyntaxTree" Version="0.1.5" />
<PackageVersion Include="ShellSyntaxTree" Version="0.2.0-alpha" />
<PackageVersion Include="Termina" Version="0.15.0" />
</ItemGroup>
<!-- Serialization -->
Expand Down
25 changes: 25 additions & 0 deletions TOOLING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,31 @@
- solution scaffold: `Netclaw.slnx` with `src/Akka.Agents` and
`src/Netclaw.App`

## Shell Execution Environment

Netclaw uses one canonical non-interactive shell per host. Linux and macOS use
`/bin/bash` with Bash grammar and POSIX paths. Windows uses PowerShell 7 via
`pwsh` with PowerShell grammar and Windows paths. `cmd.exe`, Windows PowerShell
(`powershell.exe`), and alternate POSIX shells are not compatibility fallbacks.

Windows operators must install PowerShell 7 and ensure `pwsh` is on the daemon
process `PATH`. Verify it from the same service account and environment that
runs Netclaw:

```powershell
pwsh --version
```

If the canonical executable is unavailable, `shell_execute` returns an
actionable error instead of changing grammar. Personal shell-capable turns also
receive the same platform, executable, preferred grammar, and path style in the
`execution_environment` subsection of `[working-context]`; agent-authored shell
commands must follow those declared values.

The PR validation workflow runs the full build and test suite on Linux, macOS,
and Windows. Native Windows coverage verifies PowerShell selection, execution,
parser behavior, and directory-scoped approval round trips.

## Planning and Spec Tooling

- `OpenSpec` CLI: installed and initialized in this repo
Expand Down
7 changes: 7 additions & 0 deletions docs/spec/SPEC-011-daemon-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,13 @@ works for:
The TUI is a presentation layer — it renders tool call/result output but does
not execute tools.

Shell execution uses the daemon's canonical platform environment: `/bin/bash`
with Bash grammar on Linux/macOS and PowerShell 7 (`pwsh`) with PowerShell
grammar on Windows. Parser selection, approval policy, process startup, and the
model-visible execution context consume that same immutable environment.
Windows deployments require `pwsh` on the daemon process `PATH`; absence is a
visible execution error, with no `cmd.exe` or `powershell.exe` fallback.

## Configuration Restart Coordination

### Trigger
Expand Down
2 changes: 1 addition & 1 deletion evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ log patterns** (skill loading, memory recall, checkpoint formation).
| Skill Auto-Loading | 4 | Keyword matching triggers correct skills |
| Memory Pipeline | 4 | Memory recall is active, identity-vs-memory routing is correct, explicit saves use memory tools, and automatic checkpointing still fires |
| Tool Discovery & Use | 9 | Progressive tool discovery and invocation, including timestamped webhook configuration |
| Grounding & Alignment | 4 | Uses tools to verify facts, admits uncertainty, and resolves announced attachment paths from the authoritative session root |
| Grounding & Alignment | 5 | Uses tools to verify facts, follows the declared shell grammar, admits uncertainty, and resolves announced attachment paths from the authoritative session root |
| Autonomy & Execution | 2 | Executes tasks rather than describing them |
| Deployment Mission | 1 | Applies the disk mission playbook, loads its required skill, and returns reviewed sales email |
| Subagents | 2 | Delegates through `spawn_agent`, completes ambiguous work, and gives specialized subagent guidance precedence over a conflicting deployment playbook |
Expand Down
12 changes: 12 additions & 0 deletions evals/run-evals.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1209,6 +1209,15 @@ assert_grounding_admit_unknown() {
stdout_not_contains 'is active'
}

assert_grounding_shell_grammar() {
# The eval container is Linux, so its execution_environment declares Bash.
# The model must ground command syntax in that context rather than emitting
# the PowerShell spelling from the prompt's contrastive hint.
stdout_contains '\[tool:call\] shell_execute' \
&& stdout_contains 'printf.*netclaw-grammar-eval' \
&& stdout_not_contains 'Write-Output'
}

assert_grounding_action_verification() {
stdout_contains '\[tool:call\] set_reminder'
}
Expand Down Expand Up @@ -1806,6 +1815,9 @@ run_all() {
run_case grounding_admit_unknown "no hallucinated status" \
"What's the status of the Petabridge Kubernetes cluster?"

run_case grounding_shell_grammar "uses declared Bash grammar instead of PowerShell" \
"Consult the execution_environment in your working context, then use shell_execute to print exactly netclaw-grammar-eval with that environment's native grammar. On Bash use printf; on PowerShell use Write-Output."

run_case grounding_action_verification "set_reminder called" \
"Schedule a reminder to check email in 10 minutes"

Expand Down
30 changes: 26 additions & 4 deletions feeds/skills/.system/files/netclaw-operations/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: netclaw-operations
description: "REQUIRED when the user asks about scheduling, reminders, cron jobs, timers, background jobs, diagnostics, troubleshooting, MCP tools, daemon health, identity updates, or Netclaw capabilities and self-maintenance."
metadata:
author: netclaw
version: "2.35.0"
version: "2.37.0"
---

# Netclaw Operations
Expand Down Expand Up @@ -53,6 +53,26 @@ receive a read-only project/recent-file snapshot. Successful and partial runs
return only file edits confirmed through their own tools; failed or cancelled
runs contribute no parent working-context changes.

## Shell Environment

For shell-capable sessions, `[working-context]` includes an
`execution_environment` block with the runtime platform, shell executable,
preferred grammar, and path style. Treat it as authoritative: use Bash syntax
only when `preferred_grammar` is `bash`, and PowerShell syntax only when it is
`powershell`. Do not mix operators, quoting, variables, or paths from another
grammar. Windows shell execution requires PowerShell 7 (`pwsh`) and does not
fall back to `cmd.exe` or Windows PowerShell. Do not wrap commands in either
unsupported shell explicitly; those grammars are rejected before execution.
Invoke `pwsh` command strings directly rather than through `Start-Process`;
indirect shell process launchers are unresolved and rejected.
Commands whose verb, arguments, or redirects depend on runtime expansion
cannot be saved as persistent approvals because their eventual executable or
path is unresolved.

Every executable pipeline clause is evaluated independently by hard-deny,
safe-verb, trust-zone, and approval policy. An approved or read-only pipeline
head never authorizes a different tail command.

## Scheduling & Background Jobs

Reminders: `set_reminder` with schedule type `once` / `interval` / `cron`. Always
Expand Down Expand Up @@ -194,9 +214,11 @@ implicit. Mutating verbs in the same directory still prompt.

**When the prompt offers fewer buttons.** Two cases:

- **Complex commands** (bash control-flow like `for/while/done`, unbalanced
quotes/brackets) get only `Once` and `Deny`. The matcher cannot extract a
clean verb chain to remember, so persistence is structurally impossible.
- **Complex or unresolved commands** (bash control-flow like
`for/while/done`, unbalanced quotes/brackets, dynamic verbs, dynamic path
operands, or dynamic redirects) get only `Once` and `Deny`. The matcher
cannot extract a complete static command and path set to remember, so
persistence is structurally impossible.
- **Shallow cwd** (e.g. `/etc/`, `/`) hides `Always here` only. Persisting a
too-shallow root would grant the verb across most of the filesystem;
`This chat` and `Always anywhere` remain available.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-18
76 changes: 76 additions & 0 deletions openspec/changes/canonical-bash-powershell-execution/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
## Context

Netclaw currently executes `/bin/bash -c` on Unix and `cmd.exe /c` on Windows. Security policy uses ShellSyntaxTree 0.1.5 for Bash but retains separate handwritten Windows semantics. Pipeline parsing produces multiple clauses, yet approval extraction and some policy consumers only evaluate the head. Working-context snapshots already provide a cache-stable, audience-aware seam for Git-derived runtime facts.

The change crosses execution, security, session context, prompt guidance, and native platform testing. Persisted approval entries must remain consumable with platform-correct comparison rules.

## Goals / Non-Goals

**Goals:**

- Make execution, parsing, approval, and agent guidance agree on Bash or PowerShell.
- Evaluate every executable pipeline clause before starting a process.
- Reuse the existing working-context snapshot pipeline without changing persisted actor message shapes.
- Preserve byte-prefix caching and fail loudly for missing or unsupported shells.
- Deliver four independently safe review stages: dependency foundation, Bash hardening, PowerShell/context enablement, and Windows approval completion.

**Non-Goals:**

- Supporting `cmd.exe`, fish, zsh, dash, or ash as first-class grammars.
- Inferring a user's interactive login shell.
- Adding a configurable shell fallback or silently translating commands.
- Changing tool schemas, audience grants, or approval decision persistence shapes.

## Decisions

### One immutable execution environment

Introduce a required immutable value describing OS family, executable, grammar, and path style. A single resolver constructs it during daemon composition and the same value selects the `ShellTool` process, ShellSyntaxTree parser, security semantics, and model-visible working context.

Alternative considered: let each subsystem call `OperatingSystem.IsWindows()`. Rejected because parallel detection recreates the current disagreement and makes tests platform-dependent.

### ShellSyntaxTree is the structural authority

Upgrade to 0.2.0-alpha and adapt Netclaw behind its own small parser/semantics interface. Parse once per policy evaluation and retain ordered executable clauses. Every hard-deny, safe-verb, approval, trust-zone, and display consumer examines the same clause representation. Dynamic or unsupported constructs produce a deny or approval requirement according to the existing caller's security posture; they never become automatically safe.

Alternative considered: extend the handwritten Windows tokenizer. Rejected because it duplicates upstream PowerShell parsing and cannot reliably cover aliases, nested command strings, or encoded commands.

### PowerShell 7 is the Windows shell

Windows uses `pwsh` with non-interactive command execution. Absence of `pwsh` is an explicit execution error and startup/runtime diagnostics expose the prerequisite. No `powershell.exe` or `cmd.exe` fallback is permitted.

### Execution context rides the volatile working-context tail

Add an execution-environment inspector to the existing `WorkingContextSnapshotProvider` and render its immutable facts as an `execution_environment` subsection. The full volatile context remains inserted into history before the user message. Earlier bytes therefore remain unchanged while each new turn appends a fresh nudge. The provider remains the single composition seam used by sessions and child runs.

Alternative considered: a second context provider or a `OnceAtStart` layer. Rejected because DI expects one working-context provider and current `OnceAtStart` content disappears after initial assembly.

### Guidance belongs to the embedded operating core

The full embedded `AGENTS.md` tells tool-capable agents to consult the execution environment and not mix grammars. Detailed examples live in the versioned `netclaw-operations` skill. The operator-authored deployment playbook remains untouched. The Public core changes only if discovery shows Public can receive shell execution.

### No persistence migration unless canonical compatibility tests fail

Existing approval records remain unchanged. Platform-aware parsing must emit candidates that the current comparer and approval store consume. Load/round-trip tests prove compatibility; any incompatible legacy record fails visibly rather than being silently reinterpreted.

## Risks / Trade-offs

- [ShellSyntaxTree prerelease API changes] → isolate it behind Netclaw-owned semantics and pin the exact package version.
- [Policy consumers diverge] → build candidates once and add cross-consumer pipeline-tail tests.
- [Windows hosts lack `pwsh`] → fail with an actionable message and validate on native Windows CI.
- [Repeated environment text adds tokens] → keep the subsection short; accept the small cost to reuse proven cache-stable tail behavior.
- [Prompt guidance changes model behavior] → add deterministic assembly tests and run behavioral evals.
- [PowerShell aliases obscure canonical verbs] → use the parser's canonical verb and dynamic markers; never auto-allow unresolved verbs.

## Migration Plan

1. Upgrade and adapt the parser while preserving runtime shell selection.
2. Harden Bash pipeline evaluation and ship the security fix.
3. Introduce the canonical environment, switch Windows to `pwsh`, and inject guidance/context with native Windows proof.
4. Complete Windows directory approvals and legacy round-trip validation.

Rollback is per stage. The PowerShell stage may be reverted without reverting the preceding Bash hardening. No persisted schema migration is expected.

## Open Questions

None. Supported grammars, Windows executable, context placement, fallback policy, and delivery order are fixed by this design.
35 changes: 35 additions & 0 deletions openspec/changes/canonical-bash-powershell-execution/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
## Why

Shell approval and hard-deny policy currently reason about Bash pipelines incompletely, allowing a safe or approved pipeline head to hide a different tail command. Windows execution also uses `cmd.exe` while the parser and agent guidance assume Bash, so execution, security policy, and model-generated syntax can disagree.

Source PRDs: PRD-001, PRD-002, PRD-006, and PRD-007. Tracking issues: #1693, #964, #965, and #899.

## What Changes

- Upgrade ShellSyntaxTree to its PowerShell-capable `0.2.0-alpha` API and introduce one canonical Bash/PowerShell parser selection.
- Evaluate every shell pipeline clause consistently for hard denies, safe verbs, trust-zone checks, approval matching, and approval display.
- **BREAKING**: execute Windows shell commands with PowerShell 7 (`pwsh`) instead of `cmd.exe`; fail loudly when the required shell is unavailable.
- Surface the runtime platform, shell executable, preferred grammar, and path style through the existing cache-stable working-context snapshot pipeline.
- Teach the embedded operating core and the `netclaw-operations` system skill to follow the declared execution environment rather than assuming Bash.
- Complete Windows directory-scoped approval behavior using the same canonical command/path representation consumed at runtime.
- Add native Windows/PowerShell and prompt-prefix regression coverage.

In scope for MVP: Bash on Unix-like hosts, PowerShell 7 on Windows, pipeline-wide policy evaluation, runtime context, approval persistence compatibility, and automated validation. Out of scope: first-class `cmd.exe`, fish, zsh, dash, or ash grammars and silent compatibility fallbacks.

## Capabilities

### New Capabilities

- `canonical-shell-execution`: Canonical platform-to-shell selection, grammar parsing, fail-closed behavior, and pipeline-wide security evaluation.

### Modified Capabilities

- `netclaw-tools`: Shell execution and approval policy use the canonical grammar and all command clauses.
- `netclaw-session`: Working-context snapshots carry the execution environment without invalidating prior prompt-prefix bytes.
- `netclaw-agent-memory`: The embedded operating core grounds shell generation in the runtime execution environment.

## Impact

Affected areas include `ShellTool`, shell security and approval matchers, daemon DI, working-context snapshot assembly, embedded system-prompt resources, the operations skill, ShellSyntaxTree package APIs, Windows approval persistence, and Linux/Windows CI.

Security impact is positive but high-risk: the change closes pipeline-tail policy bypasses and removes grammar ambiguity. Unsupported/dynamic syntax and missing required shells fail closed. Operationally, Windows installations must have PowerShell 7 available; diagnostics and tests must make that prerequisite explicit.
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
## ADDED Requirements

### Requirement: Canonical platform shell environment

The runtime SHALL resolve one immutable execution environment used by shell process execution, syntax parsing, security policy, approval matching, and agent context. Unix-like hosts SHALL use Bash grammar and `/bin/bash`; Windows hosts SHALL use PowerShell grammar and PowerShell 7 (`pwsh`). The environment SHALL include the operating-system family, executable, preferred grammar, and path style.

#### Scenario: Unix environment selects Bash

- **GIVEN** Netclaw runs on a supported Unix-like host
- **WHEN** the execution environment is resolved
- **THEN** its executable is `/bin/bash`
- **AND** its preferred grammar is Bash
- **AND** its path style is POSIX

#### Scenario: Windows environment selects PowerShell

- **GIVEN** Netclaw runs on Windows
- **WHEN** the execution environment is resolved
- **THEN** its executable is `pwsh`
- **AND** its preferred grammar is PowerShell
- **AND** its path style is Windows

#### Scenario: Required shell is unavailable

- **GIVEN** the canonical shell executable cannot be started
- **WHEN** a shell command is requested
- **THEN** execution fails with an actionable unavailable-shell error
- **AND** the runtime does not fall back to another shell or grammar

### Requirement: Pipeline-wide structural policy

Shell input SHALL be structurally parsed using the canonical grammar before execution. Every executable clause in every pipeline SHALL be evaluated by hard-deny, safe-verb, trust-zone, approval matching, and approval display policy. A safe or approved pipeline head SHALL NOT exempt a different tail clause.

#### Scenario: Denied pipeline tail blocks execution

- **GIVEN** a pipeline begins with a safe command
- **AND** a later clause matches a hard-deny rule
- **WHEN** shell policy evaluates the command
- **THEN** the complete command is denied before process start

#### Scenario: Unapproved pipeline tail requires approval

- **GIVEN** a pipeline head has a persisted approval
- **AND** a later executable clause does not
- **WHEN** approval policy evaluates the command
- **THEN** the later clause appears in the approval candidate set
- **AND** the persisted head approval does not authorize it

#### Scenario: Dynamic syntax does not become safe

- **GIVEN** the canonical parser marks a command or verb as dynamic or unresolved
- **WHEN** safe-verb or autonomous trust policy evaluates it
- **THEN** it is not treated as an automatically safe command
- **AND** execution follows the caller's existing deny-or-approval path

### Requirement: First-class grammar boundary

Netclaw SHALL support Bash and PowerShell as first-class shell grammars. Commands using unsupported grammar or mixing Bash and PowerShell syntax SHALL NOT be silently translated or executed through a compatibility shell.

#### Scenario: PowerShell nested command parsing

- **GIVEN** a PowerShell command contains a nested `pwsh -Command` or encoded command
- **WHEN** policy parses the command
- **THEN** nested executable clauses are included in structural policy evaluation

#### Scenario: Unsupported shell grammar

- **GIVEN** a command requires an unsupported first-class grammar
- **WHEN** Netclaw attempts to classify or execute it
- **THEN** the operation fails visibly
- **AND** Netclaw does not substitute `cmd.exe`, Windows PowerShell, or another POSIX shell
Loading
Loading