From 3ce0deb18deb12985b16251d359a1bdba8b24458 Mon Sep 17 00:00:00 2001 From: Goon Date: Fri, 12 Jun 2026 00:34:41 +0700 Subject: [PATCH] feat(cli): align runtime packages contracts --- CHANGELOG.md | 8 + README.md | 37 ++++- cmd/admin_credentials.go | 49 +++--- cmd/admin_credentials_agents.go | 110 +++++++++++++ cmd/admin_credentials_grants.go | 27 ++-- cmd/admin_credentials_helpers.go | 107 +++++++++++++ cmd/admin_credentials_users.go | 19 +-- cmd/command_response_helpers.go | 49 ++++++ cmd/credentials_contract_test.go | 144 +++++++++++++++++ cmd/packages.go | 54 +++++-- cmd/packages_contract_helpers.go | 117 ++++++++++++++ cmd/packages_contract_test.go | 151 ++++++++++++++++++ cmd/packages_updates.go | 9 +- docs/codebase-summary.md | 6 +- docs/project-roadmap.md | 23 ++- .../phase-01-contract-lock.md | 55 +++++++ .../phase-02-package-commands.md | 65 ++++++++ .../phase-03-cli-credentials-commands.md | 57 +++++++ .../phase-04-docs-verification-ship.md | 61 +++++++ .../plan.md | 86 ++++++++++ .../reports/red-team.md | 25 +++ .../reports/validate.md | 25 +++ 22 files changed, 1208 insertions(+), 76 deletions(-) create mode 100644 cmd/admin_credentials_agents.go create mode 100644 cmd/admin_credentials_helpers.go create mode 100644 cmd/command_response_helpers.go create mode 100644 cmd/credentials_contract_test.go create mode 100644 cmd/packages_contract_helpers.go create mode 100644 cmd/packages_contract_test.go create mode 100644 plans/260612-0015-runtime-packages-cli-parity/phase-01-contract-lock.md create mode 100644 plans/260612-0015-runtime-packages-cli-parity/phase-02-package-commands.md create mode 100644 plans/260612-0015-runtime-packages-cli-parity/phase-03-cli-credentials-commands.md create mode 100644 plans/260612-0015-runtime-packages-cli-parity/phase-04-docs-verification-ship.md create mode 100644 plans/260612-0015-runtime-packages-cli-parity/plan.md create mode 100644 plans/260612-0015-runtime-packages-cli-parity/reports/red-team.md create mode 100644 plans/260612-0015-runtime-packages-cli-parity/reports/validate.md diff --git a/CHANGELOG.md b/CHANGELOG.md index dcda86c..057de79 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,8 +56,16 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - `goclaw activity aggregate --group-by {action|actor_type|entity_type|actor_id} [--from --to --limit --actor-type --actor-id --action --entity-type --entity-id]` — group audit-log activity by dimension with bucket counts (`GET /v1/activity/aggregate`). Attached as subcommand of existing `activity` parent. - `goclaw logs aggregate [--group-by {level|source}] [--level --source --from]` — summarize the runtime log ring buffer (`GET /v1/logs/runtime/aggregate`, admin-only). Distinct from `logs tail`. Epoch-millis `last_seen` rendered as RFC3339, never scientific notation. +**Runtime & Packages parity** +- `goclaw credentials agent-credentials` — list/get/set/delete per-agent credential material for secure CLI credentials. +- `goclaw packages updates apply-all [packages...]` — accepts positional package specs in addition to `--packages`. + ### Fixed +- `goclaw packages list` now decodes current server grouped payloads `{system,pip,npm,github}` in table mode while preserving raw object payloads for JSON/YAML. +- `goclaw packages install` and `goclaw packages uninstall` now send the server-compatible `package` key; legacy `--runtime python|node` translates to `pip:`/`npm:` specs. +- `goclaw packages runtimes`, `packages deny-groups`, and `packages github-releases --repo --limit` now match current server envelopes and required query parameters. +- `goclaw credentials list`, `credentials presets`, `credentials agent-grants list`, and `credentials user-credentials list` now decode current server envelope payloads. - `goclaw traces list` now decodes the current server payload `{traces,total,limit,offset}`. JSON/YAML mode preserves that envelope; table mode renders rows from `traces` using `id`, `total_input_tokens`, `total_output_tokens`, and `total_cost`. - `goclaw traces get ` — TTY mode now renders a human-readable summary (header card + span tree) instead of dumping raw JSON. JSON-mode payload unchanged. Decode failures surface as wrapped errors instead of an empty `{}`. Trace ids are validated against `^[A-Za-z0-9._-]+$` and reserved tokens (`.`, `..`) are rejected before any HTTP call. Distinct exit codes per failure: not-found → 3, permission-denied → 2, malformed-id → 4, server-failure → 5. Latent retry-body bug in `internal/client/http.go` fixed: the final 5xx/429 response body is now preserved so the typed `APIError` reaches the caller (previously collapsed to exit 1). Closes #17. - `goclaw traces get ` now handles the current server detail payload `{trace,spans}` while preserving the server envelope in JSON/YAML mode. diff --git a/README.md b/README.md index 5704609..05ddee3 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,8 @@ echo "Analyze this log" | goclaw chat myagent | `storage` | Workspace file browser | | `approvals` | Execution approval management | | `delegations` | Delegation history | -| `credentials` | CLI credential store | +| `packages` | Runtime package inventory, installs, updates, runtimes, deny groups, GitHub releases | +| `credentials` | CLI credential store, grants, user credentials, agent credentials | | `tts` | Text-to-speech operations | | `media` | Media upload/download | | `activity` | Audit log | @@ -132,6 +133,40 @@ goclaw logs aggregate [--group-by ] [--level ] [--source ] [ All are one-shot HTTP — no watch loops or WS streams. `logs aggregate` is admin-only on the server; `activity aggregate --group-by actor_id` is also admin-only (server-enforced). +### Runtime & Packages + +```bash +# Runtime inventory grouped by system, pip, npm, and GitHub package sources +goclaw packages list + +# Install or uninstall with legacy runtime flags translated to server package specs +goclaw packages install pandas --runtime python +goclaw packages uninstall typescript --runtime node --yes + +# Runtime readiness, deny groups, GitHub release lookup, and update lifecycle +goclaw packages runtimes +goclaw packages deny-groups +goclaw packages github-releases --repo cli/cli --limit 10 +goclaw packages updates list +goclaw packages updates apply pip:pandas +goclaw packages updates apply-all pip:pandas npm:typescript +``` + +### CLI Credentials + +```bash +# Server-side secure CLI credential store +goclaw credentials list +goclaw credentials presets +goclaw credentials create --body '{"preset":"git"}' + +# Access grants and per-principal credential material +goclaw credentials agent-grants list +goclaw credentials user-credentials list +goclaw credentials agent-credentials list +goclaw credentials agent-credentials set --body '{"credential_type":"pat","env":{"GITHUB_TOKEN":"..."}}' +``` + ### Reading a Trace by ID ```bash diff --git a/cmd/admin_credentials.go b/cmd/admin_credentials.go index b6e14f7..6b12c1c 100644 --- a/cmd/admin_credentials.go +++ b/cmd/admin_credentials.go @@ -1,10 +1,6 @@ package cmd import ( - "encoding/json" - "fmt" - - "github.com/nextlevelbuilder/goclaw-cli/internal/output" "github.com/nextlevelbuilder/goclaw-cli/internal/tui" "github.com/spf13/cobra" ) @@ -33,14 +29,10 @@ var adminCredentialsListCmd = &cobra.Command{ return err } if cfg.OutputFormat != "table" { - printer.Print(unmarshalList(data)) + printer.Print(rawPayload(data)) return nil } - tbl := output.NewTable("ID", "NAME", "CREATED") - for _, cr := range unmarshalList(data) { - tbl.AddRow(str(cr, "id"), str(cr, "name"), str(cr, "created_at")) - } - printer.Print(tbl) + printer.Print(cliCredentialsTable(data)) return nil }, } @@ -53,8 +45,11 @@ var adminCredentialsCreateCmd = &cobra.Command{ if err != nil { return err } - name, _ := cmd.Flags().GetString("name") - data, err := c.Post("/v1/cli-credentials", map[string]any{"name": name}) + body, err := credentialCreateBody(cmd) + if err != nil { + return err + } + data, err := c.Post("/v1/cli-credentials", body) if err != nil { return err } @@ -68,13 +63,9 @@ var adminCredentialsUpdateCmd = &cobra.Command{ Short: "Update a CLI credential", Args: cobra.ExactArgs(1), RunE: func(cmd *cobra.Command, args []string) error { - bodyJSON, _ := cmd.Flags().GetString("body") - if bodyJSON == "" { - return fmt.Errorf("--body is required (JSON object)") - } - var body map[string]any - if err := json.Unmarshal([]byte(bodyJSON), &body); err != nil { - return fmt.Errorf("invalid --body JSON: %w", err) + body, err := jsonObjectFlag(cmd, "body", true) + if err != nil { + return err } c, err := newHTTP() if err != nil { @@ -140,7 +131,11 @@ var adminCredentialsPresetsCmd = &cobra.Command{ if err != nil { return err } - printer.Print(unmarshalList(data)) + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + printer.Print(credentialPresetsTable(data)) return nil }, } @@ -149,12 +144,9 @@ var adminCredentialsCheckBinaryCmd = &cobra.Command{ Use: "check-binary", Short: "Verify a CLI binary is accessible on the server", RunE: func(cmd *cobra.Command, args []string) error { - bodyJSON, _ := cmd.Flags().GetString("body") - var body map[string]any - if bodyJSON != "" { - if err := json.Unmarshal([]byte(bodyJSON), &body); err != nil { - return fmt.Errorf("invalid --body JSON: %w", err) - } + body, err := jsonObjectFlag(cmd, "body", false) + if err != nil { + return err } c, err := newHTTP() if err != nil { @@ -170,8 +162,9 @@ var adminCredentialsCheckBinaryCmd = &cobra.Command{ } func init() { - adminCredentialsCreateCmd.Flags().String("name", "", "Credential name (required)") - _ = adminCredentialsCreateCmd.MarkFlagRequired("name") + adminCredentialsCreateCmd.Flags().String("name", "", "Credential binary name") + adminCredentialsCreateCmd.Flags().String("preset", "", "Credential preset name") + adminCredentialsCreateCmd.Flags().String("body", "", "Create payload as JSON object") adminCredentialsUpdateCmd.Flags().String("body", "", "Update payload as JSON object (required)") adminCredentialsCheckBinaryCmd.Flags().String("body", "", "Check payload as JSON object") diff --git a/cmd/admin_credentials_agents.go b/cmd/admin_credentials_agents.go new file mode 100644 index 0000000..74cf395 --- /dev/null +++ b/cmd/admin_credentials_agents.go @@ -0,0 +1,110 @@ +package cmd + +import ( + "github.com/nextlevelbuilder/goclaw-cli/internal/tui" + "github.com/spf13/cobra" +) + +// admin_credentials_agents.go adds per-agent credential material management. +// Routes: GET/PUT/DELETE /v1/cli-credentials/{id}/agent-credentials[/{agentId}] + +var adminCredAgentCredentialsCmd = &cobra.Command{ + Use: "agent-credentials", + Short: "Manage per-agent credentials for a CLI credential", +} + +var adminCredAgentCredentialsListCmd = &cobra.Command{ + Use: "list ", + Short: "List agent credentials for a CLI credential", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + c, err := newHTTP() + if err != nil { + return err + } + data, err := c.Get("/v1/cli-credentials/" + args[0] + "/agent-credentials") + if err != nil { + return err + } + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + printer.Print(agentCredentialsTable(data)) + return nil + }, +} + +var adminCredAgentCredentialsGetCmd = &cobra.Command{ + Use: "get ", + Short: "Get an agent credential entry", + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + c, err := newHTTP() + if err != nil { + return err + } + data, err := c.Get("/v1/cli-credentials/" + args[0] + "/agent-credentials/" + args[1]) + if err != nil { + return err + } + printer.Print(unmarshalMap(data)) + return nil + }, +} + +var adminCredAgentCredentialsSetCmd = &cobra.Command{ + Use: "set ", + Short: "Create or update an agent credential entry", + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + body, err := jsonObjectFlag(cmd, "body", true) + if err != nil { + return err + } + c, err := newHTTP() + if err != nil { + return err + } + _, err = c.Put("/v1/cli-credentials/"+args[0]+"/agent-credentials/"+args[1], body) + if err != nil { + return err + } + printer.Success("Agent credential set") + return nil + }, +} + +var adminCredAgentCredentialsDeleteCmd = &cobra.Command{ + Use: "delete ", + Short: "Delete an agent credential entry (requires --yes)", + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + if !tui.Confirm("Delete agent credential?", cfg.Yes) { + return nil + } + c, err := newHTTP() + if err != nil { + return err + } + _, err = c.Delete("/v1/cli-credentials/" + args[0] + "/agent-credentials/" + args[1]) + if err != nil { + return err + } + printer.Success("Agent credential deleted") + return nil + }, +} + +func init() { + adminCredAgentCredentialsSetCmd.Flags().String("body", "", "Credential payload as JSON object (required)") + _ = adminCredAgentCredentialsSetCmd.MarkFlagRequired("body") + + adminCredAgentCredentialsCmd.AddCommand( + adminCredAgentCredentialsListCmd, + adminCredAgentCredentialsGetCmd, + adminCredAgentCredentialsSetCmd, + adminCredAgentCredentialsDeleteCmd, + ) + adminCredentialsCmd.AddCommand(adminCredAgentCredentialsCmd) +} diff --git a/cmd/admin_credentials_grants.go b/cmd/admin_credentials_grants.go index b847f68..bb0e187 100644 --- a/cmd/admin_credentials_grants.go +++ b/cmd/admin_credentials_grants.go @@ -1,7 +1,6 @@ package cmd import ( - "encoding/json" "fmt" "github.com/nextlevelbuilder/goclaw-cli/internal/tui" @@ -29,7 +28,11 @@ var adminCredGrantsListCmd = &cobra.Command{ if err != nil { return err } - printer.Print(unmarshalList(data)) + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + printer.Print(credentialGrantsTable(data)) return nil }, } @@ -39,13 +42,9 @@ var adminCredGrantsCreateCmd = &cobra.Command{ Short: "Create an agent grant for a CLI credential", Args: cobra.ExactArgs(1), RunE: func(cmd *cobra.Command, args []string) error { - bodyJSON, _ := cmd.Flags().GetString("body") - if bodyJSON == "" { - return fmt.Errorf("--body is required (JSON object)") - } - var body map[string]any - if err := json.Unmarshal([]byte(bodyJSON), &body); err != nil { - return fmt.Errorf("invalid --body JSON: %w", err) + body, err := jsonObjectFlag(cmd, "body", true) + if err != nil { + return err } c, err := newHTTP() if err != nil { @@ -84,13 +83,9 @@ var adminCredGrantsUpdateCmd = &cobra.Command{ Short: "Update an agent grant", Args: cobra.ExactArgs(2), RunE: func(cmd *cobra.Command, args []string) error { - bodyJSON, _ := cmd.Flags().GetString("body") - if bodyJSON == "" { - return fmt.Errorf("--body is required (JSON object)") - } - var body map[string]any - if err := json.Unmarshal([]byte(bodyJSON), &body); err != nil { - return fmt.Errorf("invalid --body JSON: %w", err) + body, err := jsonObjectFlag(cmd, "body", true) + if err != nil { + return err } c, err := newHTTP() if err != nil { diff --git a/cmd/admin_credentials_helpers.go b/cmd/admin_credentials_helpers.go new file mode 100644 index 0000000..58d12d8 --- /dev/null +++ b/cmd/admin_credentials_helpers.go @@ -0,0 +1,107 @@ +package cmd + +import ( + "encoding/json" + "fmt" + "sort" + "strings" + + "github.com/nextlevelbuilder/goclaw-cli/internal/output" + "github.com/spf13/cobra" +) + +func jsonObjectFlag(cmd *cobra.Command, flag string, required bool) (map[string]any, error) { + bodyJSON, _ := cmd.Flags().GetString(flag) + if bodyJSON == "" { + if required { + return nil, fmt.Errorf("--%s is required (JSON object)", flag) + } + return nil, nil + } + var body map[string]any + if err := json.Unmarshal([]byte(bodyJSON), &body); err != nil { + return nil, fmt.Errorf("invalid --%s JSON: %w", flag, err) + } + return body, nil +} + +func credentialCreateBody(cmd *cobra.Command) (map[string]any, error) { + if body, err := jsonObjectFlag(cmd, "body", false); err != nil || body != nil { + return body, err + } + if preset, _ := cmd.Flags().GetString("preset"); preset != "" { + return map[string]any{"preset": preset}, nil + } + if name, _ := cmd.Flags().GetString("name"); name != "" { + return map[string]any{"binary_name": name, "name": name}, nil + } + return nil, fmt.Errorf("--body, --preset, or --name is required") +} + +func cliCredentialsTable(data []byte) *output.TableData { + tbl := output.NewTable("ID", "BINARY", "DESCRIPTION", "ENABLED", "CREATED") + for _, cr := range listFromResponse(data, "items") { + tbl.AddRow(str(cr, "id"), credentialBinaryName(cr), str(cr, "description"), str(cr, "enabled"), str(cr, "created_at")) + } + return tbl +} + +func credentialBinaryName(cr map[string]any) string { + if binary := str(cr, "binary_name"); binary != "" { + return binary + } + return str(cr, "name") +} + +func credentialPresetsTable(data []byte) *output.TableData { + tbl := output.NewTable("NAME", "BINARY", "DESCRIPTION") + if list := unmarshalList(data); len(list) > 0 { + for _, preset := range list { + tbl.AddRow(str(preset, "name"), str(preset, "binary_name"), str(preset, "description")) + } + return tbl + } + presets, _ := unmarshalMap(data)["presets"].(map[string]any) + for _, name := range sortedKeys(presets) { + preset, _ := presets[name].(map[string]any) + tbl.AddRow(name, str(preset, "binary_name"), str(preset, "description")) + } + return tbl +} + +func credentialGrantsTable(data []byte) *output.TableData { + tbl := output.NewTable("ID", "AGENT", "SCOPE", "ENABLED") + for _, grant := range listFromResponse(data, "grants") { + tbl.AddRow(str(grant, "id"), str(grant, "agent_id"), str(grant, "scope"), str(grant, "enabled")) + } + return tbl +} + +func userCredentialsTable(data []byte) *output.TableData { + tbl := output.NewTable("USER", "ENV_KEYS", "UPDATED") + for _, item := range listFromResponse(data, "user_credentials") { + tbl.AddRow(str(item, "user_id"), joinValueList(item["env_keys"]), str(item, "updated_at")) + } + return tbl +} + +func agentCredentialsTable(data []byte) *output.TableData { + tbl := output.NewTable("AGENT", "TYPE", "ENV_KEYS", "UPDATED") + for _, item := range listFromResponse(data, "agent_credentials") { + tbl.AddRow(str(item, "agent_id"), str(item, "credential_type"), joinValueList(item["env_keys"]), str(item, "updated_at")) + } + return tbl +} + +func joinValueList(value any) string { + return strings.Join(stringListFromValue(value), ",") +} + +func sortedKeys(m map[string]any) []string { + keys := make([]string, 0, len(m)) + for key := range m { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} diff --git a/cmd/admin_credentials_users.go b/cmd/admin_credentials_users.go index eeccab7..7d9547c 100644 --- a/cmd/admin_credentials_users.go +++ b/cmd/admin_credentials_users.go @@ -1,9 +1,6 @@ package cmd import ( - "encoding/json" - "fmt" - "github.com/nextlevelbuilder/goclaw-cli/internal/tui" "github.com/spf13/cobra" ) @@ -29,7 +26,11 @@ var adminCredUserListCmd = &cobra.Command{ if err != nil { return err } - printer.Print(unmarshalList(data)) + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + printer.Print(userCredentialsTable(data)) return nil }, } @@ -57,13 +58,9 @@ var adminCredUserSetCmd = &cobra.Command{ Short: "Create or update a user credential entry", Args: cobra.ExactArgs(2), RunE: func(cmd *cobra.Command, args []string) error { - bodyJSON, _ := cmd.Flags().GetString("body") - if bodyJSON == "" { - return fmt.Errorf("--body is required (JSON object)") - } - var body map[string]any - if err := json.Unmarshal([]byte(bodyJSON), &body); err != nil { - return fmt.Errorf("invalid --body JSON: %w", err) + body, err := jsonObjectFlag(cmd, "body", true) + if err != nil { + return err } c, err := newHTTP() if err != nil { diff --git a/cmd/command_response_helpers.go b/cmd/command_response_helpers.go new file mode 100644 index 0000000..2311406 --- /dev/null +++ b/cmd/command_response_helpers.go @@ -0,0 +1,49 @@ +package cmd + +import "encoding/json" + +func rawPayload(data json.RawMessage) any { + if m := unmarshalMap(data); len(m) > 0 { + return m + } + return unmarshalList(data) +} + +func listFromResponse(data json.RawMessage, key string) []map[string]any { + if list := unmarshalList(data); len(list) > 0 { + return list + } + return listFromValue(unmarshalMap(data)[key]) +} + +func listFromValue(value any) []map[string]any { + switch v := value.(type) { + case []map[string]any: + return v + case []any: + out := make([]map[string]any, 0, len(v)) + for _, item := range v { + if m, ok := item.(map[string]any); ok { + out = append(out, m) + } + } + return out + default: + return nil + } +} + +func stringListFromValue(value any) []string { + switch v := value.(type) { + case []string: + return v + case []any: + out := make([]string, 0, len(v)) + for _, item := range v { + out = append(out, str(map[string]any{"value": item}, "value")) + } + return out + default: + return nil + } +} diff --git a/cmd/credentials_contract_test.go b/cmd/credentials_contract_test.go new file mode 100644 index 0000000..decd1c5 --- /dev/null +++ b/cmd/credentials_contract_test.go @@ -0,0 +1,144 @@ +package cmd + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +func TestCredentialsListAndPresetsReadServerEnvelopes(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/v1/cli-credentials": + okJSON(t, w, map[string]any{"items": []map[string]any{{"id": "cred-1", "binary_name": "git", "created_at": "2026-06-12T00:00:00Z"}}}) + case "/v1/cli-credentials/presets": + okJSON(t, w, map[string]any{"presets": map[string]any{"git": map[string]any{"binary_name": "git"}}}) + default: + w.WriteHeader(http.StatusNotFound) + } + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "table") + + out, err := runCmdCaptureStdout(t, "credentials", "list") + if err != nil { + t.Fatalf("credentials list: %v", err) + } + if !strings.Contains(out, "cred-1") || !strings.Contains(out, "git") { + t.Fatalf("credentials list output missing item:\n%s", out) + } + out, err = runCmdCaptureStdout(t, "credentials", "presets") + if err != nil { + t.Fatalf("credentials presets: %v", err) + } + if !strings.Contains(out, "git") { + t.Fatalf("credentials presets output missing preset:\n%s", out) + } +} + +func TestCredentialsCreateCanSendServerPayload(t *testing.T) { + var got map[string]any + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/v1/cli-credentials" || r.Method != http.MethodPost { + w.WriteHeader(http.StatusNotFound) + return + } + _ = json.NewDecoder(r.Body).Decode(&got) + okJSON(t, w, map[string]any{"id": "cred-1", "binary_name": "git"}) + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "json") + + err := runCmd(t, "credentials", "create", "--body", `{"preset":"git"}`) + if err != nil { + t.Fatalf("credentials create: %v", err) + } + if got["preset"] != "git" { + t.Fatalf("create body = %#v, want preset git", got) + } +} + +func TestCredentialNestedListsReadServerEnvelopes(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/v1/cli-credentials/cred-1/agent-grants": + okJSON(t, w, map[string]any{"grants": []map[string]any{{"id": "grant-1", "agent_id": "agent-1"}}}) + case "/v1/cli-credentials/cred-1/user-credentials": + okJSON(t, w, map[string]any{"user_credentials": []map[string]any{{"user_id": "user-1", "env_keys": []string{"GITHUB_TOKEN"}}}}) + case "/v1/cli-credentials/cred-1/agent-credentials": + okJSON(t, w, map[string]any{"agent_credentials": []map[string]any{{"agent_id": "agent-1", "credential_type": "pat"}}}) + default: + w.WriteHeader(http.StatusNotFound) + } + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "table") + + cases := []struct { + name string + args []string + want string + }{ + {"agent grants", []string{"credentials", "agent-grants", "list", "cred-1"}, "grant-1"}, + {"user credentials", []string{"credentials", "user-credentials", "list", "cred-1"}, "user-1"}, + {"agent credentials", []string{"credentials", "agent-credentials", "list", "cred-1"}, "agent-1"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + out, err := runCmdCaptureStdout(t, tc.args...) + if err != nil { + t.Fatalf("%s: %v", tc.name, err) + } + if !strings.Contains(out, tc.want) { + t.Fatalf("%s output missing %q:\n%s", tc.name, tc.want, out) + } + }) + } +} + +func TestCredentialAgentCredentialsRoutes(t *testing.T) { + var setBody map[string]any + var deleted bool + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case r.Method == http.MethodGet && r.URL.Path == "/v1/cli-credentials/cred-1/agent-credentials/agent-1": + okJSON(t, w, map[string]any{"agent_id": "agent-1", "credential_type": "pat"}) + case r.Method == http.MethodPut && r.URL.Path == "/v1/cli-credentials/cred-1/agent-credentials/agent-1": + _ = json.NewDecoder(r.Body).Decode(&setBody) + okJSON(t, w, map[string]any{"ok": true}) + case r.Method == http.MethodDelete && r.URL.Path == "/v1/cli-credentials/cred-1/agent-credentials/agent-1": + deleted = true + okJSON(t, w, map[string]any{"ok": true}) + default: + w.WriteHeader(http.StatusNotFound) + } + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "json") + + if err := runCmd(t, "credentials", "agent-credentials", "get", "cred-1", "agent-1"); err != nil { + t.Fatalf("agent credential get: %v", err) + } + if err := runCmd(t, "credentials", "agent-credentials", "set", "cred-1", "agent-1", "--body", `{"credential_type":"pat"}`); err != nil { + t.Fatalf("agent credential set: %v", err) + } + if err := runCmd(t, "credentials", "agent-credentials", "delete", "cred-1", "agent-1", "--yes"); err != nil { + t.Fatalf("agent credential delete: %v", err) + } + if setBody["credential_type"] != "pat" { + t.Fatalf("set body = %#v", setBody) + } + if !deleted { + t.Fatalf("delete route was not called") + } +} diff --git a/cmd/packages.go b/cmd/packages.go index eeda894..ac1a10a 100644 --- a/cmd/packages.go +++ b/cmd/packages.go @@ -22,13 +22,11 @@ var packagesListCmd = &cobra.Command{ return err } if cfg.OutputFormat != "table" { - printer.Print(unmarshalList(data)) + printer.Print(rawPayload(data)) return nil } - tbl := output.NewTable("NAME", "VERSION", "RUNTIME", "STATUS") - for _, p := range unmarshalList(data) { - tbl.AddRow(str(p, "name"), str(p, "version"), str(p, "runtime"), str(p, "status")) - } + tbl := output.NewTable("SOURCE", "NAME", "VERSION", "DETAIL") + addInstalledPackageRows(tbl, data) printer.Print(tbl) return nil }, @@ -44,7 +42,11 @@ var packagesInstallCmd = &cobra.Command{ return err } runtime, _ := cmd.Flags().GetString("runtime") - body := buildBody("name", args[0], "runtime", runtime) + spec, err := packageSpecFromRuntime(args[0], runtime) + if err != nil { + return err + } + body := buildBody("package", spec) data, err := c.Post("/v1/packages/install", body) if err != nil { return err @@ -68,7 +70,11 @@ var packagesUninstallCmd = &cobra.Command{ return err } runtime, _ := cmd.Flags().GetString("runtime") - body := buildBody("name", args[0], "runtime", runtime) + spec, err := packageSpecFromRuntime(args[0], runtime) + if err != nil { + return err + } + body := buildBody("package", spec) data, err := c.Post("/v1/packages/uninstall", body) if err != nil { return err @@ -91,7 +97,13 @@ var packagesRuntimesCmd = &cobra.Command{ if err != nil { return err } - printer.Print(unmarshalList(data)) + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + tbl := output.NewTable("NAME", "AVAILABLE", "VERSION", "READY") + addRuntimeRows(tbl, data) + printer.Print(tbl) return nil }, } @@ -108,7 +120,13 @@ var packagesDenyGroupsCmd = &cobra.Command{ if err != nil { return err } - printer.Print(unmarshalList(data)) + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + tbl := output.NewTable("NAME", "DESCRIPTION", "DEFAULT") + addDenyGroupRows(tbl, listFromResponse(data, "groups")) + printer.Print(tbl) return nil }, } @@ -117,15 +135,27 @@ var packagesGitHubReleasesCmd = &cobra.Command{ Use: "github-releases", Short: "List GitHub releases for tracked packages (GET /v1/packages/github-releases)", RunE: func(cmd *cobra.Command, args []string) error { + repo, _ := cmd.Flags().GetString("repo") + limit, _ := cmd.Flags().GetInt("limit") + path, err := githubReleasesPath(repo, limit) + if err != nil { + return err + } c, err := newHTTP() if err != nil { return err } - data, err := c.Get("/v1/packages/github-releases") + data, err := c.Get(path) if err != nil { return err } - printer.Print(unmarshalList(data)) + if cfg.OutputFormat != "table" { + printer.Print(rawPayload(data)) + return nil + } + tbl := output.NewTable("TAG", "NAME", "PRERELEASE", "ASSETS") + addGitHubReleaseRows(tbl, listFromResponse(data, "releases")) + printer.Print(tbl) return nil }, } @@ -133,6 +163,8 @@ var packagesGitHubReleasesCmd = &cobra.Command{ func init() { packagesInstallCmd.Flags().String("runtime", "", "Target runtime: python, node") packagesUninstallCmd.Flags().String("runtime", "", "Target runtime: python, node") + packagesGitHubReleasesCmd.Flags().String("repo", "", "GitHub repository in owner/name format") + packagesGitHubReleasesCmd.Flags().Int("limit", 10, "Maximum releases to fetch") packagesCmd.AddCommand(packagesListCmd, packagesInstallCmd, packagesUninstallCmd, packagesRuntimesCmd, packagesDenyGroupsCmd, packagesGitHubReleasesCmd) diff --git a/cmd/packages_contract_helpers.go b/cmd/packages_contract_helpers.go new file mode 100644 index 0000000..ee75ca2 --- /dev/null +++ b/cmd/packages_contract_helpers.go @@ -0,0 +1,117 @@ +package cmd + +import ( + "encoding/json" + "fmt" + "net/url" + "strconv" + "strings" + + "github.com/nextlevelbuilder/goclaw-cli/internal/output" +) + +func packageSpecFromRuntime(name, runtime string) (string, error) { + name = strings.TrimSpace(name) + runtime = strings.ToLower(strings.TrimSpace(runtime)) + if name == "" { + return "", fmt.Errorf("package name is required") + } + if runtime == "" || strings.Contains(name, ":") { + return name, nil + } + switch runtime { + case "python", "python3", "pip": + return "pip:" + name, nil + case "node", "nodejs", "npm": + return "npm:" + name, nil + default: + return "", fmt.Errorf("unsupported runtime %q; use python, node, or an explicit package spec", runtime) + } +} + +func githubReleasesPath(repo string, limit int) (string, error) { + repo = strings.TrimSpace(repo) + if repo == "" { + return "", fmt.Errorf("--repo is required") + } + q := url.Values{} + q.Set("repo", repo) + if limit > 0 { + q.Set("limit", strconv.Itoa(limit)) + } + return "/v1/packages/github-releases?" + q.Encode(), nil +} + +func addInstalledPackageRows(tbl *output.TableData, data json.RawMessage) { + if list := unmarshalList(data); len(list) > 0 { + for _, pkg := range list { + tbl.AddRow(str(pkg, "runtime"), packageName(pkg), str(pkg, "version"), str(pkg, "status")) + } + return + } + payload := unmarshalMap(data) + for _, source := range []string{"system", "pip", "npm", "github"} { + for _, pkg := range listFromValue(payload[source]) { + tbl.AddRow(source, packageName(pkg), str(pkg, "version"), packageDetail(pkg)) + } + } +} + +func packageName(pkg map[string]any) string { + if name := str(pkg, "name"); name != "" { + return name + } + return str(pkg, "binary") +} + +func packageDetail(pkg map[string]any) string { + parts := []string{} + if repo := str(pkg, "repo"); repo != "" { + parts = append(parts, "repo="+repo) + } + if tag := str(pkg, "tag"); tag != "" { + parts = append(parts, "tag="+tag) + } + if binaries := strings.Join(stringListFromValue(pkg["binaries"]), ","); binaries != "" { + parts = append(parts, "binaries="+binaries) + } + if status := str(pkg, "status"); status != "" { + parts = append(parts, "status="+status) + } + return strings.Join(parts, " ") +} + +func addRuntimeRows(tbl *output.TableData, data json.RawMessage) { + if list := unmarshalList(data); len(list) > 0 { + for _, rt := range list { + tbl.AddRow(str(rt, "name"), str(rt, "available"), str(rt, "version"), "") + } + return + } + payload := unmarshalMap(data) + ready := str(payload, "ready") + for _, rt := range listFromValue(payload["runtimes"]) { + tbl.AddRow(str(rt, "name"), str(rt, "available"), str(rt, "version"), ready) + } +} + +func addDenyGroupRows(tbl *output.TableData, groups []map[string]any) { + for _, group := range groups { + tbl.AddRow(str(group, "name"), str(group, "description"), str(group, "default")) + } +} + +func addGitHubReleaseRows(tbl *output.TableData, releases []map[string]any) { + for _, rel := range releases { + tbl.AddRow(str(rel, "tag"), str(rel, "name"), str(rel, "prerelease"), releaseAssets(rel)) + } +} + +func releaseAssets(rel map[string]any) string { + for _, key := range []string{"matching_assets", "assets"} { + if assets := strings.Join(stringListFromValue(rel[key]), ","); assets != "" { + return assets + } + } + return str(rel, "asset_count") +} diff --git a/cmd/packages_contract_test.go b/cmd/packages_contract_test.go new file mode 100644 index 0000000..761b1af --- /dev/null +++ b/cmd/packages_contract_test.go @@ -0,0 +1,151 @@ +package cmd + +import ( + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "os" + "strings" + "testing" +) + +func runCmdCaptureStdout(t *testing.T, args ...string) (string, error) { + t.Helper() + old := os.Stdout + r, w, err := os.Pipe() + if err != nil { + t.Fatalf("pipe stdout: %v", err) + } + os.Stdout = w + cmdErr := runCmd(t, args...) + _ = w.Close() + os.Stdout = old + out, readErr := io.ReadAll(r) + if readErr != nil { + t.Fatalf("read stdout: %v", readErr) + } + return string(out), cmdErr +} + +func TestPackagesListRendersGroupedServerPayload(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/v1/packages" { + w.WriteHeader(http.StatusNotFound) + return + } + okJSON(t, w, map[string]any{ + "system": []map[string]any{{"name": "curl", "version": "8.9.1-r1"}}, + "pip": []map[string]any{{"name": "pandas", "version": "2.0.0"}}, + "npm": []map[string]any{{"name": "typescript", "version": "5.1.0"}}, + "github": []map[string]any{{"name": "gh", "repo": "cli/cli", "tag": "v2.72.0", "binaries": []string{"gh"}}}, + }) + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "table") + + out, err := runCmdCaptureStdout(t, "packages", "list") + if err != nil { + t.Fatalf("packages list: %v", err) + } + for _, want := range []string{"system", "pip", "npm", "github", "curl", "pandas", "typescript", "gh"} { + if !strings.Contains(out, want) { + t.Fatalf("packages list output missing %q:\n%s", want, out) + } + } +} + +func TestPackagesInstallAndUninstallSendPackageKey(t *testing.T) { + var installPackage, uninstallPackage string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + var body map[string]any + _ = json.NewDecoder(r.Body).Decode(&body) + switch r.URL.Path { + case "/v1/packages/install": + installPackage, _ = body["package"].(string) + case "/v1/packages/uninstall": + uninstallPackage, _ = body["package"].(string) + default: + w.WriteHeader(http.StatusNotFound) + return + } + okJSON(t, w, map[string]any{"ok": true}) + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "json") + + if err := runCmd(t, "packages", "install", "pandas", "--runtime", "python"); err != nil { + t.Fatalf("packages install: %v", err) + } + if err := runCmd(t, "packages", "uninstall", "typescript", "--runtime", "node", "--yes"); err != nil { + t.Fatalf("packages uninstall: %v", err) + } + if installPackage != "pip:pandas" { + t.Fatalf("install package = %q, want pip:pandas", installPackage) + } + if uninstallPackage != "npm:typescript" { + t.Fatalf("uninstall package = %q, want npm:typescript", uninstallPackage) + } +} + +func TestPackagesRuntimesRendersStatusObject(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/v1/packages/runtimes" { + w.WriteHeader(http.StatusNotFound) + return + } + okJSON(t, w, map[string]any{ + "ready": false, + "runtimes": []map[string]any{{"name": "python3", "available": true, "version": "Python 3.12.0"}}, + }) + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "table") + + out, err := runCmdCaptureStdout(t, "packages", "runtimes") + if err != nil { + t.Fatalf("packages runtimes: %v", err) + } + for _, want := range []string{"python3", "true", "Python 3.12.0"} { + if !strings.Contains(out, want) { + t.Fatalf("runtimes output missing %q:\n%s", want, out) + } + } +} + +func TestPackagesGitHubReleasesRequiresRepoAndSendsQuery(t *testing.T) { + var gotRepo, gotLimit string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/v1/packages/github-releases" { + w.WriteHeader(http.StatusNotFound) + return + } + gotRepo = r.URL.Query().Get("repo") + gotLimit = r.URL.Query().Get("limit") + okJSON(t, w, map[string]any{"releases": []map[string]any{{"tag": "v2.72.0", "name": "GitHub CLI"}}}) + })) + defer srv.Close() + t.Setenv("GOCLAW_SERVER", srv.URL) + t.Setenv("GOCLAW_TOKEN", "test-token") + t.Setenv("GOCLAW_OUTPUT", "table") + + if _, err := runCmdCaptureStdout(t, "packages", "github-releases"); err == nil { + t.Fatalf("github-releases without --repo should fail before HTTP") + } + out, err := runCmdCaptureStdout(t, "packages", "github-releases", "--repo", "cli/cli", "--limit", "10") + if err != nil { + t.Fatalf("github-releases: %v", err) + } + if gotRepo != "cli/cli" || gotLimit != "10" { + t.Fatalf("query repo=%q limit=%q", gotRepo, gotLimit) + } + if !strings.Contains(out, "v2.72.0") { + t.Fatalf("github releases output missing tag:\n%s", out) + } +} diff --git a/cmd/packages_updates.go b/cmd/packages_updates.go index 4982d37..8cc3360 100644 --- a/cmd/packages_updates.go +++ b/cmd/packages_updates.go @@ -63,8 +63,9 @@ var packagesUpdateApplyCmd = &cobra.Command{ } var packagesUpdatesApplyAllCmd = &cobra.Command{ - Use: "apply-all", + Use: "apply-all [packages...]", Short: "Apply all cached package updates", + Args: cobra.ArbitraryArgs, RunE: func(cmd *cobra.Command, args []string) error { c, err := newHTTP() if err != nil { @@ -72,8 +73,10 @@ var packagesUpdatesApplyAllCmd = &cobra.Command{ } packagesRaw, _ := cmd.Flags().GetString("packages") body := map[string]any{} - if packagesRaw != "" { - body["packages"] = splitCSV(packagesRaw) + packages := splitCSV(packagesRaw) + packages = append(packages, args...) + if len(packages) > 0 { + body["packages"] = packages } data, err := c.Post("/v1/packages/updates/apply-all", body) if err != nil { diff --git a/docs/codebase-summary.md b/docs/codebase-summary.md index 18ac127..524e80a 100644 --- a/docs/codebase-summary.md +++ b/docs/codebase-summary.md @@ -1,7 +1,7 @@ # GoClaw CLI - Codebase Summary -**Generated from:** `repomix-output.xml` (2026-04-15), updated manually 2026-06-11 -**Phase Status:** P0-P4 Complete (AI-First Expansion); Super Admin API Parity Complete; Domain Coverage P5 + P6 (Backend-Unblocked) Implemented +**Generated from:** `repomix-output.xml` (2026-04-15), updated manually 2026-06-12 +**Phase Status:** P0-P4 Complete (AI-First Expansion); Super Admin API Parity Complete; Domain Coverage P5 + P6 (Backend-Unblocked) Implemented; Runtime & Packages CLI Parity Implemented **Total Files:** 80+ **Estimated Tokens:** 80,000+ **Total Size:** 220+ KB @@ -10,7 +10,7 @@ ## Overview -GoClaw CLI is a production-ready Go application providing comprehensive command-line management for GoClaw AI agent gateway servers. Built with Cobra framework, it supports 30+ command groups across modular command files with dual modes: interactive (human) and automation (CI/agent). Phases 0-4 (AI-first expansion) add AI ergonomics, admin/ops, migration, vault, and advanced agent/team/memory support. The 2026-05-18 super-admin parity work adds gateway upgrade, package updates, workstations, webhooks, MCP user credentials, secure env reveal, media/TTS/storage/channel fillers, and focused route-contract tests. The 2026-05-19 P3/P4 filler pass adds first-class profile commands, `GOCLAW_PROFILE`, `sessions compact`, WS health, trace filter polish, `codex-pool`, `api-keys rotate`, `config defaults`, chat session convenience wrappers, and `tools invoke --args`. The 2026-05-20 P5 filler pass adds team attachment download, skill-specific evolution suggestion apply, and fixes evolution update payload compatibility. The 2026-05-27 P6 backend-unblocked pass adds seven new surfaces wired to backend PRs `#37` and `#44`: `traces follow`, `providers reconnect`, `sessions branch`, `sessions follow`, `channels writers test`, `activity aggregate`, and `logs aggregate` — all one-shot HTTP commands (no new watch loops; reuse the existing `client.FollowStream` only for true streaming surfaces). The 2026-06-11 traces contract pass aligns `traces list/get/follow/export` with server `dev` envelopes and adds `traces timeline`. +GoClaw CLI is a production-ready Go application providing comprehensive command-line management for GoClaw AI agent gateway servers. Built with Cobra framework, it supports 30+ command groups across modular command files with dual modes: interactive (human) and automation (CI/agent). Phases 0-4 (AI-first expansion) add AI ergonomics, admin/ops, migration, vault, and advanced agent/team/memory support. The 2026-05-18 super-admin parity work adds gateway upgrade, package updates, workstations, webhooks, MCP user credentials, secure env reveal, media/TTS/storage/channel fillers, and focused route-contract tests. The 2026-05-19 P3/P4 filler pass adds first-class profile commands, `GOCLAW_PROFILE`, `sessions compact`, WS health, trace filter polish, `codex-pool`, `api-keys rotate`, `config defaults`, chat session convenience wrappers, and `tools invoke --args`. The 2026-05-20 P5 filler pass adds team attachment download, skill-specific evolution suggestion apply, and fixes evolution update payload compatibility. The 2026-05-27 P6 backend-unblocked pass adds seven new surfaces wired to backend PRs `#37` and `#44`: `traces follow`, `providers reconnect`, `sessions branch`, `sessions follow`, `channels writers test`, `activity aggregate`, and `logs aggregate` — all one-shot HTTP commands (no new watch loops; reuse the existing `client.FollowStream` only for true streaming surfaces). The 2026-06-11 traces contract pass aligns `traces list/get/follow/export` with server `dev` envelopes and adds `traces timeline`. The 2026-06-12 Runtime & Packages parity pass aligns `packages` and secure `credentials` command envelopes with server `dev`, adds `credentials agent-credentials`, and preserves raw object payloads for machine output. **Key Metrics:** - **70+ command files** in `cmd/` (modularized for maintainability) diff --git a/docs/project-roadmap.md b/docs/project-roadmap.md index 94a22fe..ed8d102 100644 --- a/docs/project-roadmap.md +++ b/docs/project-roadmap.md @@ -1,9 +1,26 @@ # GoClaw CLI - Project Roadmap -**Last Updated:** 2026-05-20 +**Last Updated:** 2026-06-12 **Phase Structure:** Legacy Phases 1-9 (bootstrap → CI/CD) + AI-First Expansion Phases 0-5 (2026-04-15) -**Current Status:** Legacy Phases 1-9 ✓ COMPLETE; P0-P4 ✓ COMPLETE; Super Admin API Parity ✓ COMPLETE; Domain Coverage P5 implemented pending release. -**Next Phase:** Ship Domain Coverage P5 PR to `dev` and verify beta release. +**Current Status:** Legacy Phases 1-9 ✓ COMPLETE; P0-P4 ✓ COMPLETE; Super Admin API Parity ✓ COMPLETE; Domain Coverage P5/P6 ✓ COMPLETE; Runtime & Packages CLI parity implemented pending beta ship. +**Next Phase:** Ship Runtime & Packages parity PR to `dev` and verify beta release. + +--- + +## 2026-06-12: Runtime & Packages CLI Parity IMPLEMENTED + +**Objective:** Align CLI Runtime & Packages command contracts with current GoClaw server `dev` routes. + +**Deliverables:** +- [x] Fixed `packages list` table rendering for grouped `{system,pip,npm,github}` payloads while preserving raw machine output. +- [x] Fixed `packages install` and `packages uninstall` to send server-compatible `package` specs, translating `--runtime python|node` to `pip:`/`npm:`. +- [x] Fixed `packages runtimes`, `packages deny-groups`, and `packages github-releases --repo --limit` for current server envelopes. +- [x] Extended `packages updates apply-all` to accept positional package specs in addition to `--packages`. +- [x] Fixed secure CLI credential list/presets/grants/user credential envelope parsing. +- [x] Added `credentials agent-credentials` list/get/set/delete for per-agent credential material. +- [x] Added focused contract tests for package and credential server payloads. + +**Validation:** `/usr/local/go/bin/go test ./...`; `/usr/local/go/bin/go build ./...`; `/usr/local/go/bin/go vet ./...`. --- diff --git a/plans/260612-0015-runtime-packages-cli-parity/phase-01-contract-lock.md b/plans/260612-0015-runtime-packages-cli-parity/phase-01-contract-lock.md new file mode 100644 index 0000000..6565dfa --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/phase-01-contract-lock.md @@ -0,0 +1,55 @@ +--- +phase: 1 +title: "Contract Lock" +status: pending +priority: P1 +effort: "2h" +dependencies: [] +--- + +# Phase 1: Contract Lock + +## Overview + +Lock exact server response/request shapes before changing CLI code. This phase prevents passing tests that still encode old flat-array assumptions. + +## Requirements + +- Functional: capture package and CLI credential contract facts into failing httptest fixtures. +- Non-functional: no production code changes before failing tests exist. + +## Architecture + +CLI remains thin Cobra over `internal/client.HTTPClient`. Tests assert path, method, query string, request body, and printed payload handling where practical. + +## Related Code Files + +- Modify: `cmd/phase5_test.go` +- Modify: `cmd/super_admin_parity_test.go` +- Create: `cmd/packages_contract_test.go` +- Create: `cmd/credentials_contract_test.go` + +## Implementation Steps + +1. Add server-shaped package fixtures: + - `GET /v1/packages` returns `{system,pip,npm,github}`. + - `POST /v1/packages/install` and uninstall require body key `package`. + - `GET /v1/packages/runtimes` returns `{runtimes,ready}`. + - `GET /v1/packages/github-releases?repo=cli/cli&limit=10` returns `{releases}`. +2. Add credentials fixtures: + - list returns `{items}`. + - presets returns `{presets}`. + - grants returns `{grants}`. + - user credentials returns `{user_credentials}`. + - agent credentials returns `{agent_credentials}`. +3. Verify focused tests fail for the current implementation before production edits. + +## Success Criteria + +- [ ] Focused new tests fail for the known contract mismatches. +- [ ] Test names describe stable server contracts, not plan IDs. +- [ ] No implementation files changed in this phase except tests. + +## Risk Assessment + +Risk: overfitting fixtures. Mitigation: fixtures mirror current server source and web hooks, not inferred docs. diff --git a/plans/260612-0015-runtime-packages-cli-parity/phase-02-package-commands.md b/plans/260612-0015-runtime-packages-cli-parity/phase-02-package-commands.md new file mode 100644 index 0000000..1e30e9e --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/phase-02-package-commands.md @@ -0,0 +1,65 @@ +--- +phase: 2 +title: "Package Commands" +status: pending +priority: P1 +effort: "4h" +dependencies: [1] +--- + +# Phase 2: Package Commands + +## Overview + +Fix `goclaw packages` command contracts while preserving existing automation ergonomics. + +## Requirements + +- Functional: align request bodies, query params, and response renderers for package management. +- Non-functional: preserve JSON/YAML full payloads and central error handling. + +## Architecture + +Keep `cmd/packages.go` as the command owner if it stays under the file-size guideline after edits; otherwise extract small render/query helpers into a package-specific helper file. Do not alter `internal/client`. + +## Related Code Files + +- Modify: `cmd/packages.go` +- Modify: `cmd/packages_updates.go` +- Modify: `cmd/packages_contract_test.go` + +## Implementation Steps + +1. Add helpers: + - `normalizePackageSpec(name, runtime)` maps `--runtime python` to `pip:name`, `--runtime node` to `npm:name`, and leaves explicit `source:name` unchanged. + - `queryPath(path, url.Values)` for simple query building if no existing helper fits. +2. `packages list`: + - non-table: `printer.Print(unmarshalMap(data))`. + - table: flatten `system`, `pip`, `npm`, and `github` groups with a `SOURCE` column. +3. `packages install/uninstall`: + - send `{"package": normalizedSpec}`. + - keep uninstall confirmation. +4. `packages runtimes`: + - non-table: print map. + - table: render `READY`, runtime `NAME`, `AVAILABLE`, `VERSION`. +5. `packages deny-groups`: + - accept both current `{groups:[...]}` and older array fixture for backward compatibility. +6. `packages github-releases`: + - add required `--repo owner/repo`, optional `--limit`. + - call `/v1/packages/github-releases?repo=...&limit=...`. + - render `{releases}` in table mode without dropping JSON fields in machine modes. +7. `packages updates apply-all`: + - keep `--packages` CSV. + - optionally accept positional specs too if low-risk; when both are present, merge. + - preserve non-zero on non-empty `failed[]` unless `--allow-partial`. + +## Success Criteria + +- [ ] New package contract tests pass. +- [ ] Existing package update partial-failure test still passes. +- [ ] Commands stay Cobra-pattern consistent. +- [ ] No new server assumptions beyond source-verified contracts. + +## Risk Assessment + +Risk: changing output table breaks human muscle memory. Mitigation: only table changes; JSON/YAML payloads remain server-shaped for automation. diff --git a/plans/260612-0015-runtime-packages-cli-parity/phase-03-cli-credentials-commands.md b/plans/260612-0015-runtime-packages-cli-parity/phase-03-cli-credentials-commands.md new file mode 100644 index 0000000..ba296ec --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/phase-03-cli-credentials-commands.md @@ -0,0 +1,57 @@ +--- +phase: 3 +title: "CLI Credentials Commands" +status: pending +priority: P1 +effort: "4h" +dependencies: [1] +--- + +# Phase 3: CLI Credentials Commands + +## Overview + +Align server-side CLI credential commands with the Runtime & Packages CLI Credentials tab. + +## Requirements + +- Functional: parse current envelopes and expose missing agent credential endpoints. +- Non-functional: avoid printing decrypted secret values unless an existing command explicitly requires `--show-secrets`. + +## Architecture + +Keep top-level `credentials` command assembled by existing `admin_credentials*.go` files. Add a focused `cmd/admin_credentials_agents.go` if needed to keep files small. + +## Related Code Files + +- Modify: `cmd/admin_credentials.go` +- Modify: `cmd/admin_credentials_grants.go` +- Modify: `cmd/admin_credentials_users.go` +- Create: `cmd/admin_credentials_agents.go` +- Modify: `cmd/credentials_contract_test.go` + +## Implementation Steps + +1. Fix envelope parsing: + - `credentials list` reads `{items}`. + - `credentials presets` reads `{presets}`. + - `credentials agent-grants list` reads `{grants}`. + - `credentials user-credentials list` reads `{user_credentials}`. +2. Add `credentials agent-credentials` subtree: + - `list ` -> `GET /v1/cli-credentials/{id}/agent-credentials` + - `get ` -> `GET /v1/cli-credentials/{id}/agent-credentials/{agentID}` + - `set --body JSON` -> `PUT /v1/cli-credentials/{id}/agent-credentials/{agentID}` + - `delete --yes` -> `DELETE /v1/cli-credentials/{id}/agent-credentials/{agentID}` +3. Preserve raw map output for credential get/test/check-binary. +4. Confirm no command prints secret env values except existing `agent-grants env-reveal` requiring both `--yes` and `--show-secrets`. + +## Success Criteria + +- [ ] New credential contract tests pass. +- [ ] Missing `agent-credentials` route family is exposed. +- [ ] No broad credential UX rewrite. +- [ ] Secret-reveal guard remains explicit. + +## Risk Assessment + +Risk: confusing agent grants vs agent credentials. Mitigation: keep separate subtrees matching server endpoint names exactly. diff --git a/plans/260612-0015-runtime-packages-cli-parity/phase-04-docs-verification-ship.md b/plans/260612-0015-runtime-packages-cli-parity/phase-04-docs-verification-ship.md new file mode 100644 index 0000000..3fabd29 --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/phase-04-docs-verification-ship.md @@ -0,0 +1,61 @@ +--- +phase: 4 +title: "Docs Verification Ship" +status: pending +priority: P1 +effort: "3h" +dependencies: [2, 3] +--- + +# Phase 4: Docs Verification Ship + +## Overview + +Verify the implementation, update user-facing docs, review the diff, and ship a beta PR to `dev`. + +## Requirements + +- Functional: docs tell users the actual command syntax. +- Non-functional: all tests/build/vet pass before commit and PR. + +## Architecture + +No runtime architecture changes. Shipping follows beta flow: branch from `dev`, commit, push, PR to `dev`, review PR with fix loop, merge only after CI success. + +## Related Code Files + +- Modify: `README.md` +- Modify: `CHANGELOG.md` +- Modify: `docs/project-roadmap.md` +- Modify: `docs/codebase-summary.md` +- Modify: `docs/system-architecture.md` if command inventory needs sync. +- Modify: plan status files. + +## Implementation Steps + +1. Run focused tests: + - `/usr/local/go/bin/go test ./cmd -run 'TestPackages|TestCredentials|TestAdminCred' -count=1` +2. Run full verification: + - `/usr/local/go/bin/go test ./...` + - `/usr/local/go/bin/go build ./...` + - `/usr/local/go/bin/go vet ./...` +3. Update README/CHANGELOG/docs with package and credential command changes. +4. Run code review on pending diff: + - spec compliance against this plan. + - adversarial review for secret exposure, destructive operations, and output contract drift. +5. Commit with a non-`docs`/`chore` conventional message. +6. Push `codex/runtime-packages-cli-parity`. +7. Create PR to `dev`, review with `review-pr --fix`, then merge after checks are green. + +## Success Criteria + +- [ ] Focused and full Go verification pass. +- [ ] Docs reflect command syntax and output contracts. +- [ ] Pending diff review has no Critical or Important findings. +- [ ] PR exists against `dev`. +- [ ] PR CI green before merge. +- [ ] PR merged into `dev`. + +## Risk Assessment + +Risk: local shell PATH lacks Go. Mitigation: use `/usr/local/go/bin/go` verified on this machine. diff --git a/plans/260612-0015-runtime-packages-cli-parity/plan.md b/plans/260612-0015-runtime-packages-cli-parity/plan.md new file mode 100644 index 0000000..c4924fe --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/plan.md @@ -0,0 +1,86 @@ +--- +title: "Runtime Packages CLI Parity" +description: "Align goclaw-cli Runtime & Packages commands with digitopvn/goclaw dev contracts, including grouped package payloads and CLI credential tab routes." +status: pending +priority: P1 +effort: 1d +branch: "codex/runtime-packages-cli-parity" +base: "dev" +tags: [runtime-packages, cli, contract, tdd] +blockedBy: [] +blocks: [] +created: "2026-06-11T17:15:32.053Z" +createdBy: "ck:plan --tdd" +source: skill +--- + +# Runtime Packages CLI Parity + +## Overview + +Bring `goclaw packages` and server-side `goclaw credentials` command surfaces back in sync with `/Volumes/GOON/www/digitop/goclaw` `dev`. The current CLI has the command names, but several commands still assume older response/request shapes. + +## Scope + +In scope: +- Align package list/install/uninstall/runtimes/deny-groups/github-releases with current server contracts. +- Keep existing `packages install --runtime python|node` UX by translating to `pip:` or `npm:` before sending `{"package": ...}`. +- Preserve raw server payloads for JSON/YAML output; only table mode flattens grouped payloads for humans. +- Add missing `credentials agent-credentials` subtree for `/v1/cli-credentials/{id}/agent-credentials`. +- Fix existing CLI credential envelope handling where server returns `{items}`, `{presets}`, `{grants}`, `{user_credentials}`. +- Add focused httptest contract tests before implementation changes. +- Update README/CHANGELOG/docs summaries after verification. + +Out of scope: +- Server changes in `/Volumes/GOON/www/digitop/goclaw`. +- New web UI work. +- Runtime package rollback/version-history features. +- Live gateway smoke requiring credentials; local httptest contract coverage is enough for this PR. + +## Contract Sources + +- Server packages handler: `/Volumes/GOON/www/digitop/goclaw/internal/http/packages.go` +- Server package updates handler: `/Volumes/GOON/www/digitop/goclaw/internal/http/packages_updates.go` +- Server installed package shape: `/Volumes/GOON/www/digitop/goclaw/internal/skills/package_lister.go` +- Server runtime shape: `/Volumes/GOON/www/digitop/goclaw/internal/skills/runtime_check.go` +- Web packages page: `/Volumes/GOON/www/digitop/goclaw/ui/web/src/pages/packages/packages-page.tsx` +- Server CLI credentials handler: `/Volumes/GOON/www/digitop/goclaw/internal/http/secure_cli.go` +- Web CLI credential hooks: `/Volumes/GOON/www/digitop/goclaw/ui/web/src/pages/cli-credentials/hooks/` + +## Phase Overview + +| Phase | Name | Status | +|-------|------|--------| +| 1 | [Contract Lock](./phase-01-contract-lock.md) | Pending | +| 2 | [Package Commands](./phase-02-package-commands.md) | Pending | +| 3 | [CLI Credentials Commands](./phase-03-cli-credentials-commands.md) | Pending | +| 4 | [Docs Verification Ship](./phase-04-docs-verification-ship.md) | Pending | + +## TDD Strategy + +1. Replace stale package/credential tests with server-shaped fixtures that fail on current CLI. +2. Implement the smallest command changes to satisfy those fixtures. +3. Run focused package tests, then full `go test ./...`, `go build ./...`, and `go vet ./...`. +4. Review diff against server source contracts before beta PR. + +## Cross-Plan Notes + +- Older package/credential plans are completed or stale broad-backlog artifacts. +- This plan does not block trace issue plans. +- Relevant stale in-progress plans (`P5`, `P6`) already have their implementation landed in current `dev`; do not mutate them except optional status cleanup after ship. + +## Acceptance Criteria + +- `goclaw packages list` prints grouped server payload correctly in table mode and preserves full map in JSON/YAML mode. +- `goclaw packages install/uninstall` send `package` body and support `github:`, `pip:`, `npm:`, `apk:`/bare system names. +- `goclaw packages runtimes` renders `ready` plus each runtime from `{runtimes,ready}`. +- `goclaw packages github-releases --repo owner/repo [--limit N]` calls the query contract and prints `{releases}` correctly. +- `goclaw packages updates apply/apply-all` still supports `github|pip|npm|apk:` specs and preserves partial-failure non-zero behavior. +- `goclaw credentials list/presets/agent-grants/user-credentials` parse current server envelopes. +- New `goclaw credentials agent-credentials list|get|set|delete` maps to current server routes. +- All destructive commands keep `--yes` or interactive confirmation behavior where existing convention requires it. +- Full verification passes or blocker is reported with exact command output. + +## Unresolved Questions + +None. diff --git a/plans/260612-0015-runtime-packages-cli-parity/reports/red-team.md b/plans/260612-0015-runtime-packages-cli-parity/reports/red-team.md new file mode 100644 index 0000000..6a7e345 --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/reports/red-team.md @@ -0,0 +1,25 @@ +# Plan Red-Team Report + +## Verdict + +PASS after accepted safeguards below. + +## Findings + +| Severity | Finding | Decision | +|---|---|---| +| High | CLI credential work can leak env values if list/get renders decrypted fields blindly. | Accept: tests/review must verify only existing explicit `env-reveal --yes --show-secrets` reveals secrets. | +| High | `packages install/uninstall` could silently keep sending stale `name/runtime` fields if tests only check status code. | Accept: body assertions must verify exact `package` key. | +| Medium | `packages github-releases` without `--repo` is unusable and can waste GitHub quota with invalid requests. | Accept: require `--repo` client-side before HTTP call. | +| Medium | Flattened table render can drop important fields and mislead automation if output auto-detection changes. | Accept: only flatten when `cfg.OutputFormat == "table"`; JSON/YAML prints raw map. | +| Medium | Agent grants and agent credentials names are easy to confuse. | Accept: expose server endpoint names exactly; no aliases in this PR. | + +## Required Plan Edits + +- Include explicit no-secret-leak review gate. +- Include exact request-body tests. +- Include `--repo` client validation for GitHub releases. + +## Unresolved Questions + +None. diff --git a/plans/260612-0015-runtime-packages-cli-parity/reports/validate.md b/plans/260612-0015-runtime-packages-cli-parity/reports/validate.md new file mode 100644 index 0000000..c3e06b6 --- /dev/null +++ b/plans/260612-0015-runtime-packages-cli-parity/reports/validate.md @@ -0,0 +1,25 @@ +# Plan Validation Report + +## Result + +PASS with constraints. + +## Critical Questions + +| Question | Answer | +|---|---| +| Expected output? | `goclaw packages` and `goclaw credentials` commands work against current GoClaw Runtime & Packages contracts; docs and tests updated; beta PR to `dev`. | +| Acceptance criteria? | Contract tests pass for grouped package payloads, `package` request bodies, runtime status map, GitHub release query, credential envelopes, and agent credential routes. | +| Scope boundary? | No server changes, no web UI changes, no live gateway smoke requiring credentials. | +| Non-negotiable constraints? | Go/Cobra patterns, central error handling, no secret leaks, destructive commands keep confirmation, branch from `dev`. | +| Touchpoints? | `cmd/packages*.go`, `cmd/admin_credentials*.go`, command tests, README/CHANGELOG/docs. | + +## Validation Notes + +- Keep machine output server-shaped. Do not reshape JSON/YAML for convenience. +- Translate old `--runtime` UX locally; do not send stale `runtime` field. +- Add route-family tests before code so stale fixtures cannot hide drift. + +## Unresolved Questions + +None.