diff --git a/CLAUDE.md b/CLAUDE.md index 0102620..1110c38 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -3,4 +3,5 @@ For additional context about technologies to be used, project structure, shell commands, and other important information, read the current plan +at specs/019-upstream-pr/plan.md diff --git a/brainstorm/00-overview.md b/brainstorm/00-overview.md index 5f21b74..aa50621 100644 --- a/brainstorm/00-overview.md +++ b/brainstorm/00-overview.md @@ -1,6 +1,6 @@ # Brainstorm Overview -Last updated: 2026-07-03 (021 brainstormed) +Last updated: 2026-07-11 (024 brainstormed) ## Active Sessions @@ -10,8 +10,11 @@ Last updated: 2026-07-03 (021 brainstormed) | 015 | 2026-06-30 | local-port-listener | active | - | - | | 016 | 2026-07-01 | reverse-forwarding | active | - | [#18](https://github.com/rhuss/openshell-sdk-go/issues/18) | | 018 | 2026-07-02 | edge-auth | spec-created | 016 | [#20](https://github.com/rhuss/openshell-sdk-go/issues/20) | -| 020 | 2026-07-03 | oidc-login | active | - | [#24](https://github.com/rhuss/openshell-sdk-go/issues/24) | +| 020 | 2026-07-03 | oidc-login | spec-created | 018 | [#24](https://github.com/rhuss/openshell-sdk-go/issues/24) | | 021 | 2026-07-03 | sdk-dashboard-tui | active | - | [#25](https://github.com/rhuss/openshell-sdk-go/issues/25) | +| 022 | 2026-07-03 | auth-guide | active | - | [#28](https://github.com/rhuss/openshell-sdk-go/issues/28) | +| 023 | 2026-07-03 | schema-docs | active | - | [#29](https://github.com/rhuss/openshell-sdk-go/issues/29) | +| 024 | 2026-07-11 | upstream-pr | specified | 019 | [NVIDIA/OpenShell#2044](https://github.com/NVIDIA/OpenShell/issues/2044) | ## Attic (implemented) @@ -87,6 +90,11 @@ Last updated: 2026-07-03 (021 brainstormed) - Other edge proxy convenience constructors (Google IAP, Zscaler) when concrete use cases arise (from #018, future brainstorm) - Gateway management operations (Add, Remove, SetActive) if Go programs need them (from #019, future brainstorm) - Multi-gateway client support (connecting to multiple gateways from one process) (from #019, future brainstorm) +- Should sdk/go/ have its own CODEOWNERS entry for Go-specific review? (from #024) +- Should Go SDK CI steps run only when sdk/go/ or proto/ files change (path filter)? (from #024) +- Does the Fern docs build need Go SDK pages validated separately? (from #024) +- Should the Go SDK have its own release tagging scheme (e.g., sdk/go/v0.1.0)? (from #024) +- How should `option go_package` be added to upstream proto files without breaking existing consumers? (from #024) ## Resolved Threads - Interface evolution: follow client-go pattern, accept interface growth, provide fake, use concrete `*Client` in production diff --git a/brainstorm/022-auth-guide.md b/brainstorm/022-auth-guide.md new file mode 100644 index 0000000..cb85f45 --- /dev/null +++ b/brainstorm/022-auth-guide.md @@ -0,0 +1,76 @@ +# Brainstorm: Authentication Guide + +**Date:** 2026-07-03 +**Status:** active +**Issue:** https://github.com/rhuss/openshell-sdk-go/issues/28 + +## Problem Framing + +The SDK documentation covers individual API endpoints (16 pages under +`docs/src/api/`) but has no unified entry point for authentication. +A developer asking "how do I authenticate?" has to piece together +information from `api/client.md`, `api/gateway.md`, `api/oidc.md`, +and `api/refresh.md`. The new OIDC login package (PR #26) makes this +gap more visible: there are now four auth modes, three OIDC grant +types, and a full token lifecycle (login, persist, read, refresh) that +no single page explains. + +Both new developers (choosing an auth method) and existing users +(adding OIDC to a working setup) need a guide that covers the full +picture. + +## Approaches Considered + +### A: Single Guide Page (chosen) + +One `docs/src/authentication.md` page under the "Guides" section with: +- Decision tree for auth method selection at the top +- Each auth mode documented with when-to-use and code snippets +- OIDC grant types (browser, keyboard, device code, client credentials) +- Full token lifecycle section (login writes to disk, gateway.NewClient + reads, RefreshableToken handles refresh, expiry triggers re-auth) +- Cross-links to API reference pages for detailed options + +- Pros: single entry point, complete story in one place, easy to find +- Cons: could get long if many auth methods are added later + +### B: Authentication Section (Multiple Pages) + +A new SUMMARY.md section "# Authentication" with sub-pages per method +and a lifecycle page. + +- Pros: clean separation, focused pages +- Cons: more navigation, harder to see the full picture + +### C: Guide + Cheat Sheet + +Full guide plus a one-pager with just the decision table and code +snippets for quick reference. + +- Pros: serves both learning and copy-paste use cases +- Cons: two files to maintain, risk of drift + +## Decision + +**Option A: Single guide page.** At four auth modes and three OIDC +grant types, a single page stays manageable. Can be split into B later +if the auth surface grows significantly (e.g., mTLS, custom providers). + +## Key Requirements + +- Decision tree at the top mapping scenarios to auth methods +- All four auth modes covered: none, plaintext, cloudflare_jwt, oidc +- All three OIDC grant types: browser/keyboard, device code, client credentials +- Token lifecycle: login to disk, gateway.NewClient reads, RefreshableToken + handles refresh, expiry and re-auth +- Code snippets for each auth path (complete, runnable examples) +- Cross-links to api/oidc.md, api/gateway.md, api/refresh.md, api/client.md +- SUMMARY.md entry under "Guides" section +- Works for both new developers and existing SDK users + +## Open Questions + +- Should the guide mention mTLS as "coming soon" or omit it entirely + until implemented? +- Should the decision tree be a markdown table or a prose flow + (if/then style)? diff --git a/brainstorm/023-schema-docs.md b/brainstorm/023-schema-docs.md new file mode 100644 index 0000000..4ea64ee --- /dev/null +++ b/brainstorm/023-schema-docs.md @@ -0,0 +1,87 @@ +# Brainstorm: Schema and Data Type Documentation + +**Date:** 2026-07-03 +**Status:** active +**Issue:** https://github.com/rhuss/openshell-sdk-go/issues/29 + +## Problem Framing + +The SDK documentation has 16 API reference pages organized by gRPC +sub-client (Sandboxes, Exec, Providers, etc.) but no dedicated +documentation for the data types themselves. Types like `SandboxSpec`, +`PolicyRule`, `ProviderProfile`, and `NetworkPolicyEndpoint` appear +inline in endpoint pages without field descriptions, relationships, +or usage context. The `openshell/v1/types/` package contains ~96 types +across 20 files, but only ~30 are referenced in docs. + +For developers working with the SDK, understanding the data model is +more important than knowing the CRUD methods. A developer creating a +sandbox needs to understand `SandboxSpec`, `SandboxTemplate`, +`SandboxStatus`, and their relationships. Currently they have to read +Go source code to learn this. + +## Approaches Considered + +### A: Domain-Concept Schema Pages (chosen) + +New `docs/src/schema/` directory with pages grouped by domain concept +(sandbox, policy, provider, etc.). Each page contains field tables, +type relationships, and usage examples. All 16 existing API pages get +inline cross-links from type names to the schema pages. + +- Pros: types are documented where developers think about them (by + domain, not by Go file), cross-links make the docs navigable, + examples show types in context +- Cons: more pages to maintain, requires updating all 16 API pages + for cross-links, manual sync with Go source when types change + +### B: Single Types Reference Page + +One large `docs/src/schema/types.md` page listing all types with +anchor links, grouped by domain. + +- Pros: single page to search, easy to maintain +- Cons: very long page (~96 types), poor navigation, doesn't scale + +### C: Auto-Generated from Go Source + +Generate type docs from Go doc comments using a tool like `gomarkdoc` +or a custom generator. + +- Pros: always in sync with source, zero maintenance +- Cons: output format is hard to customize, loses the narrative + structure (relationships, examples, cross-domain context), requires + build tooling + +## Decision + +**Option A: Domain-concept schema pages with inline cross-links.** +Manual pages give full control over presentation, relationships, and +examples. Auto-generation (Option C) can be explored later as a +complement but not a replacement, since the value is in the curated +relationships and examples, not just field listings. + +## Key Requirements + +- New `docs/src/schema/` directory +- Pages organized by domain concept, not Go package structure +- Proposed pages: sandbox, policy, network-policy, provider, profile, + exec, service, config-settings, auth, health, watch-events +- Each page includes: field tables with types and descriptions, type + relationships (containment, references), usage examples +- Inline cross-links from all 16 existing API pages (first mention of + each type becomes a link to the schema page anchor) +- New "Schema" section in SUMMARY.md between "API Reference" and "Guides" +- Types derived from Go source in `openshell/v1/types/` +- Only SDK types documented (proto types excluded per Constitution I) + +## Open Questions + +- Should the schema section appear before or after the API Reference + section in SUMMARY.md? Before gives types prominence, after follows + the natural reading flow (learn the API, then dive into types). +- Should enum-like constants (e.g., `SandboxPhase` values) be + documented on the schema pages or kept on the endpoint pages where + they're used? +- Should there be a schema overview page with a visual domain model + diagram showing how the major types relate? diff --git a/brainstorm/024-upstream-pr.md b/brainstorm/024-upstream-pr.md new file mode 100644 index 0000000..edea4f0 --- /dev/null +++ b/brainstorm/024-upstream-pr.md @@ -0,0 +1,117 @@ +# Brainstorm: Upstream PR Preparation + +**Date:** 2026-07-11 +**Status:** active +**Upstream Issue:** [NVIDIA/OpenShell#2044](https://github.com/NVIDIA/OpenShell/issues/2044) + +## Problem Framing + +The Go SDK has matured through 18 spec-driven iterations covering all 55 +RPCs, auth (token refresh, OIDC, edge proxy), gateway config, SSH/TCP +tunneling, fake client, and comprehensive tests. It currently lives at +`github.com/rhuss/openshell-sdk-go` as a standalone repo. The goal is to +contribute it upstream to NVIDIA/OpenShell as a draft PR under `sdk/go/`, +matching the pattern where the Python SDK already lives in `python/`. + +This requires several coordinated changes: module path migration, example +extraction, documentation integration with the Fern-based docs site, CI +automation for proto freshness, and scoping development artifacts so they +don't clutter the upstream repo. + +## Approaches Considered + +### A: Fork-first migration (chosen) + +Prepare everything in the `rhuss/OpenShell` fork, then open a single draft +PR against `NVIDIA/OpenShell`: + +1. Create `github.com/rhuss/openshell-examples`, move `examples/oshell/` + TUI there with its own `go.mod` +2. In the OpenShell fork, create branch `sdk/go-sdk`, add `sdk/go/` with + the full SDK (rewritten module path, all sub-clients, auth packages, + fake client, tests, committed `.pb.go` files) +3. Include `specs/` as design documentation with a PR note asking upstream + whether to keep or remove them +4. Write Fern MDX docs under `docs/sdks/go/` and wire into navigation +5. Add `go:proto` mise task + CI validation step +6. Open as draft PR referencing issue #2044 + +- Pros: single coherent PR, reviewers see the full picture, docs and CI + show production readiness +- Cons: large PR, import path rewrite touches every file + +### B: Minimal SDK first, docs and CI follow + +Two sequential PRs: first the code, then docs and CI after feedback. + +- Pros: smaller first PR +- Cons: incomplete picture, reviewers will ask about missing docs/CI + +### C: RFC-first, code later + +Open an issue proposing the SDK, wait for approval, then submit code. + +- Pros: gets buy-in first +- Cons: slower, issue #2044 already exists as the proposal, code speaks + louder + +## Decision + +Approach A: fork-first migration as a single draft PR. + +The SDK is mature enough to present as a complete package. The draft +status gives reviewers space without merge pressure. Issue #2044 already +serves as the RFC, so a separate proposal step would be redundant. + +## Key Requirements + +### Module path migration +- Change `github.com/rhuss/openshell-sdk-go` to + `github.com/NVIDIA/OpenShell/sdk/go` in go.mod and all import paths +- Consumer imports become + `github.com/NVIDIA/OpenShell/sdk/go/openshell/v1` + +### Example extraction +- Move `examples/oshell/` (4,649 LOC, 12 files) to + `github.com/rhuss/openshell-examples` as a separate repo +- Own `go.mod` referencing the SDK via the new module path + +### Documentation (Fern MDX) +- Write Go SDK docs as Fern MDX pages under `docs/sdks/go/` + (getting-started, architecture, error-handling, authentication) +- Wire into `docs/index.yml` navigation alongside Python SDK docs +- Remove the local mdbook docs (`docs/`) from the PR + +### Proto change detection +- Add `go:proto` mise task in `tasks/go.toml` that regenerates Go + bindings from `proto/` using protoc +- Add CI step in `branch-checks.yml` that runs the task and diffs, + failing if generated files are stale +- Mirrors the Python SDK pattern (`python:proto` mise task) +- Commit generated `.pb.go` files so consumers can `go get` without + needing protoc + +### Spec artifacts +- Include `specs/` directory in the PR as design documentation +- PR description mentions spec-driven development methodology and asks + upstream whether specs should be retained or removed +- Exclude `brainstorm/`, `.specify/`, `.claude/`, `CLAUDE.md`, `AGENTS.md` + +### PR format +- Draft PR against `NVIDIA/OpenShell` main branch +- References issue [#2044](https://github.com/NVIDIA/OpenShell/issues/2044) +- Full SDK shipped at once (all sub-clients, auth, OIDC, edge, tunnels, + fake client, tests) +- Mentions that the SDK was developed using spec-driven development + with the spec-kit toolchain + +## Open Questions + +- Should `sdk/go/` have its own CODEOWNERS entry for Go-specific review? +- Should the Go SDK CI steps run only when `sdk/go/` or `proto/` files + change (path filter), or on every PR? +- Does the Fern docs build need Go SDK pages validated separately? +- Should the Go SDK have its own release tagging scheme (e.g., + `sdk/go/v0.1.0`) or follow the main repo's releases? +- How should `option go_package` be added to the upstream proto files + without breaking existing consumers? diff --git a/specs/019-upstream-pr/REVIEWERS.md b/specs/019-upstream-pr/REVIEWERS.md new file mode 100644 index 0000000..29ce781 --- /dev/null +++ b/specs/019-upstream-pr/REVIEWERS.md @@ -0,0 +1,141 @@ +# Review Guide: Upstream PR Preparation + +**Generated**: 2026-07-11 | **Spec**: [spec.md](spec.md) + +## Why This Change + +The OpenShell Go SDK has been developed in a standalone repository +(`rhuss/openshell-sdk-go`) providing typed gRPC clients for the gateway +and edge APIs, in-memory fakes for testing, and OIDC authentication. +Upstream issue [#2044](https://github.com/NVIDIA/OpenShell/issues/2044) +requests contributing this SDK into the `NVIDIA/OpenShell` monorepo under +`sdk/go/`. Without this contribution, Go developers must discover and +depend on a personal fork rather than the official project. + +## What Changes + +The Go SDK source is relocated into the upstream repo at `sdk/go/` with +its module path rewritten from `github.com/rhuss/openshell-sdk-go` to +`github.com/NVIDIA/OpenShell/sdk/go`. The 4,649-LOC oshell TUI example +is extracted to a separate repository (`rhuss/openshell-examples`) to +keep the PR focused on library code. Four concise Fern MDX documentation +pages are added under `docs/sdks/go/`. A Go CI job and proto freshness +check are added to the existing branch-checks workflow. The PR is +delivered as a single squashed draft commit referencing issue #2044. + +## How It Works + +The implementation proceeds in three phases: + +1. **Module path migration**: Mechanical `sed` replacement of the module + path in `go.mod`, all `.go` files, and the mise proto generation + scripts. Proto bindings are regenerated under the new module. Build + and test verification confirms zero regressions. + +2. **Documentation and CI**: Fern MDX pages (getting-started, architecture, + error-handling, authentication) are created under `docs/sdks/go/` and + wired into the docs navigation. A `tasks/go.toml` mise task wraps + proto generation for the monorepo context. A Go job is added to + `branch-checks.yml` following the existing Rust/Python pattern + (checkout, mise install, lint, build, test, proto:check). + +3. **PR assembly**: Internal artifacts (brainstorms, spec-kit config, + Claude Code config) are excluded. Design specs are included under + `sdk/go/specs/` with an explicit question to upstream maintainers + about whether to retain them. All changes are squashed into a single + DCO-signed commit. + +## When It Applies + +**Applies when**: +- Contributing the Go SDK to the NVIDIA/OpenShell upstream repository +- Preparing a draft PR with module path rewrite, docs, and CI integration +- Establishing the `sdk/` directory pattern for future language SDKs + +**Does not apply when**: +- Ongoing SDK feature development (that happens in the development repo) +- Python SDK or other language SDK contributions (separate scope) +- Changes to upstream proto definitions (the SDK adapts to upstream, not the reverse) + +## Key Decisions + +1. **SDK placed under `sdk/go/`, not `go/` (top-level)**. The `sdk/` + namespace establishes a pattern for future language SDKs. Python + already lives at `python/` but a new `sdk/` prefix provides cleaner + organization. Alternative: flat `go/` directory. Rejected because + `go/` is ambiguous and the issue description proposed `sdk/go/`. + +2. **Module path rewrite via `sed`, not specialized tools**. The rewrite + is a simple string replacement with no `replace` directives or + vendor directory to complicate matters. `gomvpkg` and `gofmt -r` + were considered but add unnecessary tooling for a one-time operation. + +3. **Examples extracted to separate repo, not kept in-tree**. The + 4,649-LOC oshell TUI is application code, not library code. Including + it would bloat the PR and blur the SDK scope. The examples repo + depends on the upstream module path, validating end-to-end importability. + +4. **Single squashed commit for the PR**. Avoids partial-state commits + where intermediate steps (e.g., module path rewrite without updated + imports) would not compile. Alternative: per-story commits. Rejected + because reviewers examine the full diff anyway and intermediate + commits add noise without independent value. + +5. **Proto generation uses M_FLAGS, not proto file modification**. + The `go_package` option is supplied via protoc command-line flags + (`--go_opt=M...`) rather than editing upstream `.proto` files. + This respects upstream ownership of proto definitions. + +6. **Concise reference docs, not full tutorials**. Fern MDX pages are + 1-2 pages each with key concepts and short code snippets. Full + tutorial walkthroughs can be added post-merge. This reduces PR + review burden while demonstrating production readiness. + +## Areas Needing Attention + +- **Module path completeness**: The `sed` rewrite must catch every + occurrence in `.go` files AND the mise.toml proto generation scripts. + A missed reference will cause subtle import failures. + +- **Proto M_FLAGS correctness**: The mapping between proto file names + and Go package paths in the mise task must match the upstream proto + directory structure. If upstream proto files are reorganized, these + flags need updating. + +- **Fern compatibility**: The Go SDK docs are the first SDK documentation + in the upstream docs site. The "SDKs" navigation section is new and + must integrate without breaking existing page routing. + +- **CI container toolchain**: The Go job assumes the CI container + (`ghcr.io/nvidia/openshell/ci:latest`) can install Go 1.25 and + protoc via mise. If the container lacks prerequisites for mise-managed + Go installation, the CI job will fail. + +- **Examples repo `replace` directive**: The examples repo temporarily + uses a `replace` directive since the upstream SDK module isn't + published until the PR merges. This directive must be removed once + the SDK is available at the upstream path. + +## Open Questions + +- Should the `specs/` directory be retained in the upstream repo? + The PR description explicitly asks upstream maintainers to decide. +- Will the CI container support Go 1.25 installation via mise, or + does a container image update need to be coordinated? + +## Review Checklist + +- [ ] Key decisions are justified +- [ ] Breaking changes are documented with migration guidance +- [ ] Scope matches the stated boundaries +- [ ] Success criteria are achievable +- [ ] No unstated assumptions +- [ ] Zero references to old module path (`github.com/rhuss/openshell-sdk-go`) in `sdk/go/` +- [ ] Fern MDX code examples use actual SDK function signatures (FR-016) +- [ ] SPDX license headers present on all `.go` files +- [ ] PR description references issue #2044 and includes spec retention question +- [ ] Internal artifacts (brainstorm, .specify, .claude) are excluded from PR + +--- + + diff --git a/specs/019-upstream-pr/checklists/requirements.md b/specs/019-upstream-pr/checklists/requirements.md new file mode 100644 index 0000000..1608433 --- /dev/null +++ b/specs/019-upstream-pr/checklists/requirements.md @@ -0,0 +1,37 @@ +# Specification Quality Checklist: Upstream PR Preparation + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: 2026-07-11 +**Feature**: [spec.md](../spec.md) + +## Content Quality + +- [ ] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [ ] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [ ] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [ ] No implementation details leak into specification + +## Notes + +- This spec is inherently technical (it describes preparing a Go SDK PR). + File paths, tool commands, and module paths are part of the problem + domain, not implementation details. Items 1, 3, 4, and 7 are unchecked + to acknowledge this intentional deviation from the generic template. diff --git a/specs/019-upstream-pr/data-model.md b/specs/019-upstream-pr/data-model.md new file mode 100644 index 0000000..cbc7508 --- /dev/null +++ b/specs/019-upstream-pr/data-model.md @@ -0,0 +1,73 @@ +# Data Model: Upstream PR Preparation + +**Date**: 2026-07-11 | **Feature**: 019-upstream-pr + +This feature does not introduce new runtime entities. The "data model" +describes the file tree mapping between the development repo and the +upstream PR target. + +## Entity: File Tree Mapping + +### Source (development repo: `rhuss/openshell-sdk-go`) + +| Source Path | Destination in PR | Transform | +|-------------|------------------|-----------| +| `openshell/` | `sdk/go/openshell/` | Module path rewrite in all `.go` files | +| `proto/*.proto` | `sdk/go/proto/` | Copy as-is | +| `proto/*v1/` | `sdk/go/proto/` | Regenerate with new module path | +| `go.mod` | `sdk/go/go.mod` | Rewrite module declaration | +| `go.sum` | `sdk/go/go.sum` | Regenerate via `go mod tidy` | +| `Makefile` | `sdk/go/Makefile` | Copy as-is | +| `mise.toml` | `sdk/go/mise.toml` | Update MODULE variable in proto tasks | +| `specs/` | `sdk/go/specs/` | Copy as-is (retention TBD by upstream) | +| `examples/oshell/` | (excluded) | Moved to `rhuss/openshell-examples` | +| `brainstorm/` | (excluded) | Internal only | +| `.specify/` | (excluded) | Internal only | +| `.claude/` | (excluded) | Internal only | +| `CLAUDE.md` | (excluded) | Internal only | +| `AGENTS.md` | (excluded) | Internal only | +| `docs/` | (excluded) | mdbook docs replaced by Fern MDX | + +### New Files (created in upstream fork) + +| Path | Purpose | +|------|---------| +| `docs/sdks/go/getting-started.mdx` | SDK installation and first API call | +| `docs/sdks/go/architecture.mdx` | Module structure and gRPC transport | +| `docs/sdks/go/error-handling.mdx` | Error types and gRPC status codes | +| `docs/sdks/go/authentication.mdx` | OIDC and gateway authentication | +| `tasks/go.toml` | Mise task for `go:proto` in upstream CI | + +### Modified Files (existing in upstream fork) + +| Path | Change | +|------|--------| +| `docs/index.yml` | Add "SDKs" section with Go subfolder | +| `.github/workflows/branch-checks.yml` | Add Go CI job | + +## Entity: Module Path + +| Field | Before | After | +|-------|--------|-------| +| go.mod module | `github.com/rhuss/openshell-sdk-go` | `github.com/NVIDIA/OpenShell/sdk/go` | +| Import prefix | `github.com/rhuss/openshell-sdk-go/openshell/v1` | `github.com/NVIDIA/OpenShell/sdk/go/openshell/v1` | +| Proto M_FLAGS | `$MODULE/proto/openshellv1` | Same pattern, new module | + +## Entity: Examples Repository + +| Field | Value | +|-------|-------| +| Name | `rhuss/openshell-examples` | +| Visibility | Public | +| go.mod module | `github.com/rhuss/openshell-examples` | +| SDK dependency | `github.com/NVIDIA/OpenShell/sdk/go` | +| Initial content | `examples/oshell/` (connection.go, demo.go, 4,649 LOC) | + +## Validation Rules + +- Every `.go` file under `sdk/go/` must contain zero references to + `github.com/rhuss/openshell-sdk-go` after rewrite +- `sdk/go/go.mod` must declare module `github.com/NVIDIA/OpenShell/sdk/go` +- Proto M_FLAGS in `sdk/go/mise.toml` must reference the new module path +- No files from the exclusion list appear in the PR diff +- All unit tests pass under the new module identity diff --git a/specs/019-upstream-pr/plan.md b/specs/019-upstream-pr/plan.md new file mode 100644 index 0000000..455f271 --- /dev/null +++ b/specs/019-upstream-pr/plan.md @@ -0,0 +1,143 @@ +# Implementation Plan: Upstream PR Preparation + +**Branch**: `019-upstream-pr` | **Date**: 2026-07-11 | **Spec**: [spec.md](spec.md) +**Input**: Feature specification from `/specs/019-upstream-pr/spec.md` + +## Summary + +Prepare and submit a draft PR contributing the Go SDK to `NVIDIA/OpenShell` +under `sdk/go/`. This involves rewriting the Go module path, extracting +examples into a separate repo, creating Fern MDX documentation, adding a +Go proto CI job to branch-checks.yml, and opening the PR as a single +squashed commit referencing upstream issue #2044. + +## Technical Context + +**Language/Version**: Go 1.25.0 +**Primary Dependencies**: gRPC 1.81.1, protobuf 1.36.11, testify 1.11.1, websocket 1.8.15, oauth2 0.36.0 +**Storage**: N/A +**Testing**: go test + testify (assert/require), `//go:build integration` tag for integration tests +**Target Platform**: Cross-platform Go library (consumers run Go 1.25+) +**Project Type**: Library (Go SDK) + CI automation + Fern documentation +**Performance Goals**: N/A (one-time PR preparation, not a runtime feature) +**Constraints**: Single squashed commit, no upstream proto file modifications, `go_package` supplied via protoc M_FLAGS +**Scale/Scope**: ~192 Go files, 3 proto files, 4 Fern MDX pages, 1 CI job, 1 mise task file + +## Constitution Check + +*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* + +| Principle | Status | Notes | +|-----------|--------|-------| +| I. Proto Isolation | PASS | Proto stays in `sdk/go/proto/`, never exposed publicly | +| II. Idiomatic Go | PASS | Module path follows Go conventions (`github.com/NVIDIA/OpenShell/sdk/go`) | +| III. Test-First | N/A | No new SDK code; existing tests validate post-rewrite correctness | +| IV. Upstream Tracking | PASS | Proto sync task + CI freshness check maintain tracking | +| V. Minimal Dependencies | PASS | No new dependencies added | +| VI. Secrets Never Leak | CHECK | Must verify no credentials leak into PR diff (`.env`, tokens, config) | +| VII. Deep Copy at Boundaries | N/A | No new boundary code | +| VIII. Doc Examples Compile | PASS | FR-016 requires Fern code examples match actual SDK API | +| IX. Agent-Friendly Docs | N/A | Applies to godoc, not Fern MDX | +| X. Proto-SDK Naming | N/A | No new types | +| XI. Fake-Real Parity | N/A | No fake changes | +| XII. Graceful Shutdown | N/A | No shutdown code | +| XIII. Docs Accompany Features | PASS | Story 3 delivers Fern MDX documentation | + +**Gate result**: CHECK (Principle VI secret scan pending; all other principles satisfied) + +## Project Structure + +### Documentation (this feature) + +```text +specs/019-upstream-pr/ +├── plan.md # This file +├── research.md # Phase 0: upstream structure analysis +├── data-model.md # Phase 1: file tree layout for PR +├── quickstart.md # Phase 1: step-by-step execution guide +└── tasks.md # Phase 2: task breakdown (created by /speckit.tasks) +``` + +### Source Code (repository root) + +The PR creates the following structure in the `rhuss/OpenShell` fork: + +```text +sdk/ +└── go/ + ├── go.mod # Module: github.com/NVIDIA/OpenShell/sdk/go + ├── go.sum + ├── Makefile + ├── mise.toml # Go-specific tool versions + tasks (build, test, lint) + ├── openshell/ + │ └── v1/ + │ ├── edge/ # Edge API client + │ ├── fake/ # In-memory fakes for testing + │ ├── gateway/ # Gateway gRPC client + │ ├── internal/ # Internal helpers + │ ├── oidc/ # OIDC authentication + │ └── types/ # Domain types + ├── proto/ + │ ├── openshell.proto # Synced from upstream proto/ + │ ├── datamodel.proto + │ ├── sandbox.proto + │ ├── openshellv1/ # Generated .pb.go files + │ ├── datamodelv1/ + │ └── sandboxv1/ + └── specs/ # Design documentation (retention TBD by upstream) + ├── 001-project-setup/ + ├── 002-proto-generation/ + ├── 003-core-sdk/ + └── ... + +docs/ +└── sdks/ + └── go/ + ├── getting-started.mdx + ├── architecture.mdx + ├── error-handling.mdx + └── authentication.mdx + +tasks/ +└── go.toml # Mise task: go:proto (references sdk/go/ paths) +``` + +Files modified in the fork (not new): +- `docs/index.yml` - Add SDK navigation section +- `.github/workflows/branch-checks.yml` - Add Go CI job + +**Structure Decision**: The SDK lives under `sdk/go/` as a self-contained Go +module. This mirrors the existing `python/` top-level directory pattern but +places the Go SDK under `sdk/` to establish the namespace for future +language-specific SDKs. Fern docs go under `docs/sdks/go/` following the +existing `docs/
/` folder pattern. The mise task file at +`tasks/go.toml` follows the existing `tasks/.toml` pattern. + +## Complexity Tracking + +No constitution violations requiring justification. + +## Implementation Strategy + +### Phase 1: Module Path Rewrite + Example Extraction (P1 stories) + +1. Create a working branch in the `rhuss/OpenShell` fork based on upstream main +2. Copy SDK source into `sdk/go/`, rewriting module path from + `github.com/rhuss/openshell-sdk-go` to `github.com/NVIDIA/OpenShell/sdk/go` +3. Update all internal imports, go.mod, and proto generation scripts +4. Verify `go build ./...` and `go test ./...` pass under new module +5. Create `github.com/rhuss/openshell-examples` repo, move `examples/oshell/` +6. Verify examples compile against upstream module path + +### Phase 2: Documentation + CI (P2 stories) + +7. Create 4 concise Fern MDX pages under `docs/sdks/go/` +8. Wire into `docs/index.yml` navigation +9. Create `tasks/go.toml` with `go:proto` task (adapted from current mise.toml) +10. Add Go CI job to `.github/workflows/branch-checks.yml` + +### Phase 3: PR Assembly (P3 stories) + +11. Exclude internal artifacts (brainstorm, .specify, .claude, CLAUDE.md, AGENTS.md, docs/) +12. Squash into single commit with comprehensive message +13. Open draft PR referencing issue #2044 with spec retention question diff --git a/specs/019-upstream-pr/quickstart.md b/specs/019-upstream-pr/quickstart.md new file mode 100644 index 0000000..ec01b33 --- /dev/null +++ b/specs/019-upstream-pr/quickstart.md @@ -0,0 +1,87 @@ +# Quickstart: Upstream PR Preparation + +**Date**: 2026-07-11 | **Feature**: 019-upstream-pr + +## Prerequisites + +- Go 1.25+ installed +- `protoc` 29.6+ and Go protoc plugins (managed via mise) +- `gh` CLI authenticated with GitHub +- `rhuss/OpenShell` fork synced with `NVIDIA/OpenShell` main +- Access to create repos under `github.com/rhuss/` + +## Execution Order + +### Step 1: Create PR branch in fork + +```bash +cd /path/to/rhuss/OpenShell +git fetch upstream +git checkout -b go-sdk upstream/main +``` + +### Step 2: Copy SDK source with module path rewrite + +```bash +# Copy source directories +cp -r /path/to/openshell-sdk-go/openshell sdk/go/openshell +cp -r /path/to/openshell-sdk-go/proto sdk/go/proto +cp /path/to/openshell-sdk-go/{go.mod,go.sum,Makefile,mise.toml} sdk/go/ +cp -r /path/to/openshell-sdk-go/specs sdk/go/specs + +# Rewrite module path +OLD="github.com/rhuss/openshell-sdk-go" +NEW="github.com/NVIDIA/OpenShell/sdk/go" +find sdk/go -name '*.go' -exec sed -i.bak "s|$OLD|$NEW|g" {} + +sed -i.bak "s|$OLD|$NEW|g" sdk/go/go.mod sdk/go/mise.toml +find sdk/go -name '*.bak' -delete + +# Regenerate proto bindings with new module path, then tidy +cd sdk/go && mise run proto:gen && go mod tidy && cd ../.. +``` + +### Step 3: Verify build and tests + +```bash +cd sdk/go +go build ./... +go test ./... +``` + +### Step 4: Extract examples + +```bash +gh repo create rhuss/openshell-examples --public +# Clone, add examples, push (details in tasks) +``` + +### Step 5: Create Fern docs + +Create 4 MDX files under `docs/sdks/go/` and add navigation entry to +`docs/index.yml`. + +### Step 6: Add CI job + +Add Go job to `.github/workflows/branch-checks.yml` and create +`tasks/go.toml` with proto generation task. + +### Step 7: Squash and open PR + +```bash +git add -A +git commit -s -m "feat(sdk): add Go SDK for OpenShell" # Full message per research.md R7 +gh pr create --draft --base main --repo NVIDIA/OpenShell \ + --title "feat(sdk): add Go SDK for OpenShell" \ + --body-file pr-description.md +``` + +## Verification Checklist + +- [ ] `go build ./...` passes in `sdk/go/` +- [ ] `go test ./...` passes in `sdk/go/` +- [ ] Zero references to `github.com/rhuss/openshell-sdk-go` in `sdk/go/` +- [ ] `rhuss/openshell-examples` compiles against upstream module +- [ ] `docs/index.yml` includes SDK section +- [ ] PR excludes brainstorm/, .specify/, .claude/, CLAUDE.md, AGENTS.md +- [ ] PR references issue #2044 +- [ ] Commit has `Signed-off-by` trailer diff --git a/specs/019-upstream-pr/research.md b/specs/019-upstream-pr/research.md new file mode 100644 index 0000000..07fca88 --- /dev/null +++ b/specs/019-upstream-pr/research.md @@ -0,0 +1,175 @@ +# Research: Upstream PR Preparation + +**Date**: 2026-07-11 | **Feature**: 019-upstream-pr + +## R1: Upstream Repository Structure + +**Decision**: Place SDK under `sdk/go/` as a self-contained Go module. + +**Rationale**: The upstream repo has no `sdk/` directory yet. Python code +lives at `python/` (top-level). Creating `sdk/go/` establishes a +namespace for future language SDKs and separates the Go SDK cleanly from +the Rust crates (`crates/`), Python (`python/`), and other top-level +directories. The Go SDK is the first under `sdk/`, setting the pattern. + +**Alternatives considered**: +- `go/` (top-level): Mirrors `python/` but Go is a reserved keyword in + some contexts and `go/` is ambiguous. Rejected. +- `sdks/go/`: Plural `sdks` is unconventional. Rejected. +- `python/` pattern (flat `go/`): Inconsistent with the issue description + which proposes `sdk/go/`. Rejected. + +## R2: Module Path Rewrite Mechanics + +**Decision**: Use `sed` for mechanical find-and-replace of the module path +in go.mod and all `.go` files, followed by `go mod tidy`. + +**Rationale**: The rewrite is a simple string replacement: +`github.com/rhuss/openshell-sdk-go` to `github.com/NVIDIA/OpenShell/sdk/go`. +No `replace` directives exist in go.mod. No vendor directory exists. +The `go.sum` regenerates from `go mod tidy`. `sed` is sufficient and +avoids adding tooling dependencies. + +**Alternatives considered**: +- `gomvpkg`: Overkill for a module-level rename with no package splits. +- `gofmt -r`: Cannot rewrite import paths. +- Manual editing: Error-prone with 192 Go files. + +**Implementation detail**: Proto generation scripts (`mise.toml`) also +contain the module path in M_FLAGS. These must be updated simultaneously. + +## R3: Fern Documentation Integration + +**Decision**: Create a new `SDKs` navigation section in `docs/index.yml` +with a `go/` subfolder containing 4 MDX pages. + +**Rationale**: The upstream docs have no SDK section yet (no Python SDK +docs either despite `python/` existing). The Go SDK docs will be the +first SDK documentation. The navigation structure uses `folder` entries +pointing to directories with `index.yml` files, or inline `page` entries. +A new top-level section "SDKs" with a Go subfolder follows the existing +pattern (e.g., `providers/`, `reference/`). + +**Fern version**: 5.40.0 (supports MDX, tabs, callouts, code blocks). + +**Docs layout**: +``` +docs/sdks/go/ +├── getting-started.mdx # SDK installation, basic connection, first API call +├── architecture.mdx # gRPC transport, module structure, proto layer +├── error-handling.mdx # Error types, gRPC status codes, retry patterns +└── authentication.mdx # OIDC, token refresh, gateway auth +``` + +Navigation entry in `docs/index.yml`: +```yaml +- section: "SDKs" + slug: sdks + contents: + - folder: sdks/go + title: "Go SDK" +``` + +## R4: CI Job Integration + +**Decision**: Add a `go` job to `.github/workflows/branch-checks.yml` +following the existing Rust/Python job pattern. + +**Rationale**: The branch-checks workflow already runs Rust and Python +checks in the same CI container (`ghcr.io/nvidia/openshell/ci:latest`). +Adding a Go job follows the same pattern: checkout, install tools via +mise, run lint/build/test/proto-check. The CI container needs Go and +protoc, both installable via mise. + +**Job structure** (mirrors Rust/Python jobs): +```yaml +go: + name: Go + needs: pr_metadata + if: needs.pr_metadata.outputs.should_run == 'true' + runs-on: linux-amd64-cpu8 + container: + image: ghcr.io/nvidia/openshell/ci:latest + steps: + - checkout + - install tools (mise install --locked) + - lint (cd sdk/go && mise run lint) + - build (cd sdk/go && mise run build) + - test (cd sdk/go && mise run test) + - proto check (cd sdk/go && mise run proto:check) +``` + +**Proto task adaptation**: The current `proto:gen` and `proto:check` tasks +in `mise.toml` use `MODULE="github.com/rhuss/openshell-sdk-go"`. After +rewrite, this becomes `MODULE="github.com/NVIDIA/OpenShell/sdk/go"`. +The `tasks/go.toml` file provides two tasks for the monorepo CI: +- `go:proto` wraps `cd sdk/go && mise run proto:gen` (regeneration) +- `go:proto:check` wraps `cd sdk/go && mise run proto:check` (freshness validation) +The CI job calls `go:proto:check`; contributors run `go:proto` to regenerate. + +## R5: Examples Repository + +**Decision**: Create `github.com/rhuss/openshell-examples` as a public +repository with the oshell TUI extracted. + +**Rationale**: The examples repo must be public for `go get` to resolve +it. It gets its own `go.mod` depending on +`github.com/NVIDIA/OpenShell/sdk/go`. Since the upstream SDK won't be +published until the PR is merged, the examples `go.mod` will initially +use a `replace` directive pointing to the local SDK during development, +switched to the real upstream path once published. + +**Bootstrap**: +```bash +gh repo create rhuss/openshell-examples --public --description "Examples for the OpenShell Go SDK" +``` + +## R6: File Exclusion Strategy + +**Decision**: Use a curated file list for the PR rather than copying +everything and excluding. + +**Rationale**: The SDK repo contains many internal-only artifacts. +Instead of rsync with exclusions (fragile), build the PR branch by +selectively copying the needed directories: +- `openshell/` (SDK source) +- `proto/` (proto files + generated code) +- `specs/` (design documentation, retention TBD) +- `go.mod`, `go.sum`, `Makefile`, `mise.toml` (build configuration) + +Explicitly excluded (never copied): +- `brainstorm/` (internal ideation) +- `.specify/` (spec-kit tooling) +- `.claude/` (Claude Code config) +- `CLAUDE.md`, `AGENTS.md` (agent instructions) +- `docs/` (mdbook, replaced by Fern docs in upstream) +- `examples/` (extracted to separate repo) +- `.github/` (development repo CI, not upstream CI) + +## R7: Commit Message Structure + +**Decision**: Single squashed commit with structured message following +upstream's DCO convention. + +**Rationale**: Upstream requires DCO sign-off (`Signed-off-by:` trailer). +The commit message should describe the full SDK contribution scope. + +**Template**: +``` +feat(sdk): add Go SDK for OpenShell + +Add a Go SDK that provides typed clients for the OpenShell gateway +and edge APIs. The SDK wraps the gRPC transport layer and exposes +idiomatic Go types with functional option configuration. + +Includes: +- Gateway and Edge API clients with full CRUD operations +- In-memory fakes for testing (matching real client validation) +- OIDC authentication with automatic token refresh +- Proto generation automation (mise task + CI check) +- Fern documentation (getting-started, architecture, auth, errors) +- Design specifications from spec-driven development + +Resolves: #2044 +Signed-off-by: Roland Huß +``` diff --git a/specs/019-upstream-pr/spec.md b/specs/019-upstream-pr/spec.md new file mode 100644 index 0000000..a8418ce --- /dev/null +++ b/specs/019-upstream-pr/spec.md @@ -0,0 +1,305 @@ +# Feature Specification: Upstream PR Preparation + +**Feature Branch**: `019-upstream-pr` +**Created**: 2026-07-11 +**Status**: Draft +**Input**: Brainstorm 024 - Prepare draft PR against NVIDIA/OpenShell contributing the Go SDK under sdk/go/ +**Upstream Issue**: [NVIDIA/OpenShell#2044](https://github.com/NVIDIA/OpenShell/issues/2044) + +## User Scenarios & Testing + +### User Story 1 - Module Path Migration (Priority: P1) + +A contributor prepares the Go SDK source code for inclusion in the +NVIDIA/OpenShell monorepo by rewriting the Go module path from the +development repo to the upstream path, ensuring all internal imports +compile and tests pass under the new module identity. + +**Why this priority**: Without correct module paths, no consumer can +import the SDK and no CI can compile it. This is the foundation for +every other story. + +**Independent Test**: After rewriting, `go build ./...` and `go test ./...` +succeed under the new module path with zero import errors. + +**Acceptance Scenarios**: + +1. **Given** the SDK source at `github.com/rhuss/openshell-sdk-go`, + **When** the module path is rewritten to `github.com/NVIDIA/OpenShell/sdk/go`, + **Then** `go.mod` declares the new module, all internal import statements + reference the new path, and `go build ./...` succeeds with no errors. + +2. **Given** the rewritten module, + **When** `go test ./...` is run, + **Then** all existing unit tests pass with the same results as before + the rewrite. + +3. **Given** the rewritten module, + **When** a consumer creates a new Go project and runs + `go get github.com/NVIDIA/OpenShell/sdk/go/openshell/v1`, + **Then** the SDK packages resolve and compile correctly. + +--- + +### User Story 2 - Example Extraction (Priority: P1) + +A contributor extracts the oshell TUI example from the SDK repo into a +separate repository so the upstream PR contains only SDK library code, +not application-level examples. + +**Why this priority**: The 4,649-LOC TUI example adds significant bulk +to the PR and is not part of the SDK library. Extracting it keeps the +upstream contribution focused. + +**Independent Test**: The examples repo builds and runs independently, +importing the SDK via the new module path. + +**Acceptance Scenarios**: + +1. **Given** the `examples/oshell/` directory in the SDK repo, + **When** it is moved to `github.com/rhuss/openshell-examples`, + **Then** the examples repo has its own `go.mod` with a dependency on + `github.com/NVIDIA/OpenShell/sdk/go` and compiles successfully. + +2. **Given** the examples have been extracted, + **When** the SDK repo is examined, + **Then** the `examples/` directory no longer exists. + +--- + +### User Story 3 - Fern Documentation Integration (Priority: P2) + +A contributor writes Go SDK documentation as Fern MDX pages that +integrate into the existing OpenShell docs site, appearing alongside +the Python SDK documentation. + +**Why this priority**: Documentation demonstrates that the SDK is +production-ready and helps reviewers understand the API surface. +Ranked P2 because the SDK is functional without docs. + +**Independent Test**: The Fern docs build succeeds with the new Go SDK +pages and they appear in the navigation. + +**Acceptance Scenarios**: + +1. **Given** the OpenShell Fern docs structure, + **When** Go SDK MDX pages are added under `docs/sdks/go/`, + **Then** the Fern build (`fern check`) succeeds and the pages appear + in the rendered navigation under an "SDKs > Go" section. + +2. **Given** the Go SDK docs pages, + **When** a reader navigates to the docs site, + **Then** they find getting-started, architecture, error-handling, and + authentication guides with code examples that match the actual SDK API. + +3. **Given** the existing Python SDK documentation, + **When** Go SDK docs are added, + **Then** the Python SDK docs remain unchanged and accessible. + +--- + +### User Story 4 - Proto Generation Automation (Priority: P2) + +A contributor adds a mise task and CI validation step that regenerates +Go protobuf bindings from the repo's `proto/` directory and fails the +build if the committed `.pb.go` files are stale. + +**Why this priority**: Ensures the Go SDK stays in sync with proto +changes. Ranked P2 because manual regeneration works as a fallback. + +**Independent Test**: Modify a proto file, run CI, and observe that it +detects the stale generated files and fails. + +**Acceptance Scenarios**: + +1. **Given** the proto files in `proto/` and the mise task `go:proto`, + **When** a contributor runs `mise run go:proto`, + **Then** Go protobuf bindings are regenerated in `sdk/go/proto/` from + the current proto definitions. + +2. **Given** a PR that modifies a proto file without regenerating Go bindings, + **When** CI runs the proto freshness check, + **Then** the check fails with a clear message indicating which files + are stale and how to regenerate them. + +3. **Given** a PR where proto files and generated Go bindings are in sync, + **When** CI runs the proto freshness check, + **Then** the check passes. + +--- + +### User Story 5 - Spec Artifact Scoping (Priority: P3) + +A contributor includes the `specs/` directory in the upstream PR as +design documentation while excluding internal development artifacts +(brainstorms, spec-kit config, Claude Code config). + +**Why this priority**: The specs provide design rationale but are not +required for the SDK to function. The PR description asks upstream +maintainers whether to keep or remove them. + +**Independent Test**: The PR diff includes `sdk/go/specs/` but does not +include `brainstorm/`, `.specify/`, `.claude/`, `CLAUDE.md`, or `AGENTS.md`. + +**Acceptance Scenarios**: + +1. **Given** the SDK repo with specs and brainstorms, + **When** the PR is prepared, + **Then** `sdk/go/specs/` is included in the PR and `brainstorm/`, + `.specify/`, `.claude/`, `CLAUDE.md`, `AGENTS.md` are excluded. + +2. **Given** the draft PR description, + **When** a reviewer reads it, + **Then** they see a note explaining that the specs were created using + spec-driven development and asking whether they should be retained + or removed from the repository. + +--- + +### User Story 6 - Draft PR Creation (Priority: P3) + +A contributor opens a draft PR against NVIDIA/OpenShell that presents +the complete Go SDK, referencing the existing upstream issue. + +**Why this priority**: The PR is the delivery vehicle but depends on +all prior stories being complete. + +**Independent Test**: The draft PR exists on GitHub, references issue +#2044, and contains the expected file tree under `sdk/go/`. + +**Acceptance Scenarios**: + +1. **Given** all SDK code, docs, and CI changes are ready in the fork, + **When** a draft PR is opened against `NVIDIA/OpenShell`, + **Then** the PR title references the Go SDK, the description references + issue [#2044](https://github.com/NVIDIA/OpenShell/issues/2044), and + the PR is marked as draft. + +2. **Given** the draft PR, + **When** a reviewer examines the file tree, + **Then** they see `sdk/go/` with Go source, `sdk/go/proto/` with + committed `.pb.go` files, `sdk/go/specs/` with design documentation, + `docs/sdks/go/` with Fern MDX pages, and `tasks/go.toml` with the + mise task. + +--- + +### Edge Cases + +- What happens when a proto file adds a new service or message type? + The CI freshness check detects the drift and fails, prompting + regeneration. +- What if the upstream repo already has an `sdk/` directory by the time + the PR is ready? The PR adapts to the existing directory structure. +- What if the Fern docs build uses a version of Fern that is + incompatible with the added MDX pages? The contributor runs + `fern check` locally before submitting. +- What if `option go_package` is missing from upstream proto files? + The proto generation task supplies the package option via protoc + flags rather than modifying the proto source files. + +## Clarifications + +### Session 2026-07-11 + +- Q: Should creating the actual GitHub repo for examples be in scope, or only prepare the extraction? → A: Create the GitHub repo now and push the extracted examples. +- Q: How deep should the Fern MDX documentation pages be? → A: Concise reference pages (1-2 pages each) with key concepts and short code snippets. +- Q: Should the draft PR use a single squashed commit or separate commits per story? → A: Single squashed commit with a comprehensive commit message. + +## Requirements + +### Functional Requirements + +- **FR-001**: The Go module path MUST be rewritten from + `github.com/rhuss/openshell-sdk-go` to + `github.com/NVIDIA/OpenShell/sdk/go` in `go.mod` and all `.go` files. +- **FR-002**: All unit tests MUST pass after the module path rewrite + with no behavior changes. +- **FR-003**: The `examples/oshell/` directory MUST be moved to + `github.com/rhuss/openshell-examples` as a separate repository with + its own `go.mod`. The GitHub repo MUST be created and the extracted + code pushed as part of this feature (not deferred). +- **FR-004**: The examples repository MUST depend on the SDK via the new + upstream module path. +- **FR-005**: Fern MDX documentation pages MUST be created under + `docs/sdks/go/` covering getting-started, architecture, error-handling, + and authentication topics. Each page SHOULD be concise (1-2 pages) with + key concepts and short code snippets, not full tutorial walkthroughs. +- **FR-006**: The MDX pages MUST be wired into `docs/index.yml` so they + appear in the Fern docs site navigation. +- **FR-007**: A `go:proto` mise task MUST be created in `tasks/go.toml` + that regenerates Go protobuf bindings from the repo's `proto/` directory. +- **FR-008**: A CI step MUST be added to `branch-checks.yml` that runs + the proto generation task and fails if committed `.pb.go` files diverge + from what the task produces. +- **FR-009**: Generated `.pb.go` files MUST be committed to the repository + so consumers can `go get` the SDK without needing protoc installed. +- **FR-010**: The `specs/` directory MUST be included in the PR under + `sdk/go/specs/`. +- **FR-011**: The following from the SDK development repo MUST NOT be + copied to the upstream fork: `brainstorm/`, `.specify/`, `.claude/`, + `CLAUDE.md`, `AGENTS.md`, `docs/` (the mdbook documentation site). + Note: this does not affect the new Fern MDX pages created under + `docs/sdks/go/` per FR-005, which are authored directly in the fork. +- **FR-012**: The PR MUST be opened as a draft against the + `NVIDIA/OpenShell` main branch with a single squashed commit + containing all changes. +- **FR-013**: The PR description MUST reference issue + [#2044](https://github.com/NVIDIA/OpenShell/issues/2044). +- **FR-014**: The PR description MUST mention the spec-driven development + methodology and ask upstream maintainers whether to retain or remove + the `specs/` directory. +- **FR-015**: The proto generation MUST work without modifying upstream + proto files (supply `go_package` via protoc command-line flags). +- **FR-016**: Code examples in Fern MDX documentation pages MUST use + actual SDK function signatures, type names, and argument counts so + they compile against the current SDK API. + +### Key Entities + +- **SDK source tree**: The Go packages under `sdk/go/openshell/` and + `sdk/go/proto/` that comprise the library. +- **Examples repository**: A standalone GitHub repo at + `github.com/rhuss/openshell-examples` holding the oshell TUI and + future examples. +- **Fern MDX pages**: Documentation files under `docs/sdks/go/` that + integrate into the OpenShell docs site. +- **Mise task**: The `go:proto` task definition in `tasks/go.toml` that + drives proto-to-Go code generation. +- **Draft PR**: The GitHub pull request against NVIDIA/OpenShell + delivering all changes. + +## Success Criteria + +### Measurable Outcomes + +- **SC-001**: `go build ./...` and `go test ./...` succeed under the new + module path `github.com/NVIDIA/OpenShell/sdk/go` with zero failures. +- **SC-002**: The examples repository at `github.com/rhuss/openshell-examples` + compiles and runs the oshell TUI against a gateway using the upstream + SDK module path. +- **SC-003**: `fern check` passes with the Go SDK documentation pages + included and they appear in the rendered navigation. +- **SC-004**: The proto freshness CI step detects intentionally stale + `.pb.go` files and fails within the normal CI pipeline. +- **SC-005**: The draft PR file tree contains exactly the expected + directories (`sdk/go/`, `docs/sdks/go/`, `tasks/go.toml` changes) and + excludes all internal development artifacts. +- **SC-006**: A reviewer can read the PR description, understand the SDK + scope, find the referenced upstream issue, and see the question about + spec retention without additional context. + +## Assumptions + +- The `rhuss/OpenShell` fork is up to date with `NVIDIA/OpenShell` main + branch at the time the PR branch is created. +- The upstream repo does not already have an `sdk/` directory. If it + does, the directory structure adapts accordingly. +- The Fern docs framework version used by upstream supports the MDX + features needed for Go SDK pages (code blocks, tabs, callouts). +- `protoc` and the Go gRPC plugin are available in the CI environment + or can be installed via mise. +- The existing SPDX license headers (Apache-2.0) are compatible with + the upstream repo's licensing requirements. +- The module path rewrite is a mechanical find-and-replace operation + that does not change any SDK behavior or API surface. diff --git a/specs/019-upstream-pr/tasks.md b/specs/019-upstream-pr/tasks.md new file mode 100644 index 0000000..1e7bee2 --- /dev/null +++ b/specs/019-upstream-pr/tasks.md @@ -0,0 +1,269 @@ +# Tasks: Upstream PR Preparation + +**Input**: Design documents from `/specs/019-upstream-pr/` +**Prerequisites**: plan.md, spec.md, research.md, data-model.md, quickstart.md + +**Tests**: No test tasks generated (existing tests validate post-rewrite correctness; no new SDK code is written). + +**Organization**: Tasks grouped by user story. US1 (Module Path Migration) is the MVP and must complete before other stories. US3/US4/US5 can execute in parallel after US1 completes. US6 depends on all other stories. + +## Format: `[ID] [P?] [Story] Description` + +- **[P]**: Can run in parallel (different files, no dependencies) +- **[Story]**: Which user story this task belongs to (US1-US6) +- Paths are relative to the `rhuss/OpenShell` fork unless noted otherwise + +--- + +## Global Constraints + +These constraints apply to ALL tasks and are inherited implicitly: + +- **Module path**: Target module is `github.com/NVIDIA/OpenShell/sdk/go`. Old module is `github.com/rhuss/openshell-sdk-go`. +- **SPDX headers**: Every `.go` file must have `SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.` and `SPDX-License-Identifier: Apache-2.0`. +- **DCO sign-off**: The final commit must include `Signed-off-by:` trailer. +- **No proto file modifications**: `go_package` is supplied via protoc M_FLAGS, not by editing `.proto` files. +- **Work location**: All file operations happen in the `rhuss/OpenShell` fork, not in the SDK development repo. + +--- + +## Phase 1: Setup + +**Purpose**: Prepare the working branch in the upstream fork + +- [ ] T001 Sync rhuss/OpenShell fork with NVIDIA/OpenShell main branch +- [ ] T002 Create `go-sdk` branch in rhuss/OpenShell fork based on upstream main +- [ ] T003 Create `sdk/go/` directory structure in the fork working branch + +**Checkpoint**: Clean working branch exists in fork, ready for SDK content + +--- + +## Phase 2: User Story 1 - Module Path Migration (Priority: P1) MVP + +**Goal**: Rewrite Go module path from development repo to upstream path so all imports compile under the new module identity. + +**Independent Test**: `go build ./...` and `go test ./...` succeed in `sdk/go/` with zero references to the old module path. + +### Implementation for User Story 1 + +- [ ] T004 [US1] Copy SDK source to fork: `openshell/` to `sdk/go/openshell/`, `proto/` to `sdk/go/proto/`, and root build files (`go.mod`, `go.sum`, `Makefile`, `mise.toml`) to `sdk/go/`. Exclude `examples/`, `brainstorm/`, `.specify/`, `.claude/`, `docs/`, `CLAUDE.md`, `AGENTS.md`. +- [ ] T005 [US1] Rewrite module path from `github.com/rhuss/openshell-sdk-go` to `github.com/NVIDIA/OpenShell/sdk/go` in `sdk/go/go.mod` and all `.go` files under `sdk/go/` +- [ ] T006 [US1] Update MODULE variable in `sdk/go/mise.toml` proto generation tasks (`proto:gen`, `proto:check`) to use new module path +- [ ] T007 [US1] Regenerate proto bindings in `sdk/go/` using updated mise proto:gen task +- [ ] T008 [US1] Run `go mod tidy` in `sdk/go/` to regenerate `go.sum` +- [ ] T009 [US1] Verify `go build ./...` passes in `sdk/go/` +- [ ] T010 [US1] Verify `go test ./...` passes in `sdk/go/` +- [ ] T011 [US1] Verify zero remaining references to `github.com/rhuss/openshell-sdk-go` in `sdk/go/` using grep + +**Checkpoint**: SDK compiles and all tests pass under new module path. MVP complete. + +**Interfaces for downstream phases**: After US1, the following paths exist and are stable: +- `sdk/go/go.mod` (module `github.com/NVIDIA/OpenShell/sdk/go`) +- `sdk/go/openshell/v1/` (all SDK packages with rewritten imports) +- `sdk/go/proto/` (regenerated `.pb.go` files) +- `sdk/go/mise.toml` (proto tasks with updated MODULE variable) + +--- + +## Phase 3: User Story 2 - Example Extraction (Priority: P1) + +**Goal**: Extract the oshell TUI example into a standalone public repository so the upstream PR contains only library code. + +**Independent Test**: The examples repo compiles and its `go.mod` depends on the upstream SDK module path. + +### Implementation for User Story 2 + +- [ ] T012 [US2] Create public GitHub repository `rhuss/openshell-examples` with description "Examples for the OpenShell Go SDK" +- [ ] T013 [US2] Initialize examples repo with `go.mod` declaring module `github.com/rhuss/openshell-examples` and dependency on `github.com/NVIDIA/OpenShell/sdk/go` +- [ ] T014 [US2] Copy `examples/oshell/` source files (connection.go, demo.go, README.md) from SDK dev repo to examples repo root +- [ ] T015 [US2] Rewrite import paths in examples repo from `github.com/rhuss/openshell-sdk-go` to `github.com/NVIDIA/OpenShell/sdk/go` +- [ ] T016 [US2] Add temporary `replace` directive in examples `go.mod` pointing to local SDK copy for pre-merge development (use `go.work` or remove the directive before pushing) +- [ ] T017 [US2] Verify examples repo compiles with `go build ./...` +- [ ] T018 [US2] Remove the temporary `replace` directive from examples `go.mod` and push examples repo to GitHub. Note: the SDK dependency will not resolve via `go get` until the upstream PR is merged. The pushed repo serves as a reference; a follow-up commit updates imports after merge. + +**Checkpoint**: Examples repo exists at github.com/rhuss/openshell-examples and compiles independently. + +--- + +## Phase 4: User Story 3 - Fern Documentation Integration (Priority: P2) + +**Goal**: Create concise Go SDK documentation as Fern MDX pages integrated into the OpenShell docs site navigation. + +**Independent Test**: `fern check` passes (or `mise run docs`) and Go SDK pages appear in navigation. + +### Implementation for User Story 3 + +- [ ] T019 [P] [US3] Create `docs/sdks/go/getting-started.mdx` covering SDK installation, basic gateway connection, and first API call with code snippets using actual SDK function signatures +- [ ] T020 [P] [US3] Create `docs/sdks/go/architecture.mdx` covering module structure, gRPC transport layer, proto isolation pattern, and package hierarchy +- [ ] T021 [P] [US3] Create `docs/sdks/go/error-handling.mdx` covering SDK error types, gRPC status code mapping, and retry patterns with code snippets +- [ ] T022 [P] [US3] Create `docs/sdks/go/authentication.mdx` covering OIDC flow, token refresh, gateway authentication options, and configuration examples +- [ ] T023 [US3] Add "SDKs" navigation section with Go subfolder to `docs/index.yml` following existing section pattern (see research.md R3 for YAML structure) +- [ ] T024 [US3] Verify Fern docs build passes with Go SDK pages included (run `mise run docs` or `fern check` in fork) + +**Checkpoint**: Go SDK docs pages render in Fern navigation under "SDKs > Go SDK". + +**Interfaces for US6**: `docs/sdks/go/*.mdx` (4 files) and `docs/index.yml` (modified with SDKs section). + +--- + +## Phase 5: User Story 4 - Proto Generation Automation (Priority: P2) + +**Goal**: Add a mise task and CI validation step that regenerates Go protobuf bindings and fails if committed files are stale. + +**Independent Test**: Modify a proto file, run the CI check, and observe it detects stale generated files. + +### Implementation for User Story 4 + +- [ ] T025 [P] [US4] Create `tasks/go.toml` with `go:proto` task that runs proto generation from `sdk/go/` directory (delegate to `sdk/go/mise.toml` proto:gen task, adapting paths for monorepo context) +- [ ] T026 [P] [US4] Add `go` job to `.github/workflows/branch-checks.yml` following Rust/Python job pattern: checkout, mise install, lint, build, test, proto:check (see research.md R4 for job structure) +- [ ] T027 [US4] Verify proto freshness check detects intentionally stale `.pb.go` files by modifying a proto file and running the check + +**Checkpoint**: Go CI job runs lint, build, test, and proto check in branch-checks workflow. + +**Interfaces for US6**: `tasks/go.toml` (new file) and `.github/workflows/branch-checks.yml` (modified with Go job). + +--- + +## Phase 6: User Story 5 - Spec Artifact Scoping (Priority: P3) + +**Goal**: Include design specs in the PR while excluding internal development artifacts. + +**Independent Test**: `sdk/go/specs/` exists in the PR diff; `brainstorm/`, `.specify/`, `.claude/`, `CLAUDE.md`, `AGENTS.md` do not. + +### Implementation for User Story 5 + +- [ ] T028 [US5] Copy `specs/` directory to `sdk/go/specs/` in fork, including all numbered spec directories with their plan.md, spec.md, and research.md files +- [ ] T029 [US5] Verify no internal artifacts exist in the fork working branch: grep for `.specify/`, `.claude/`, `CLAUDE.md`, `AGENTS.md`, `brainstorm/` in the file tree + +**Checkpoint**: Spec artifacts are included under `sdk/go/specs/`, all internal artifacts excluded. + +--- + +## Phase 7: User Story 6 - Draft PR Creation (Priority: P3) + +**Goal**: Open a draft PR against NVIDIA/OpenShell presenting the complete Go SDK contribution. + +**Independent Test**: Draft PR exists on GitHub, references issue #2044, contains expected file tree under `sdk/go/`. + +### Implementation for User Story 6 + +- [ ] T030 [US6] Write PR description in a local file: include SDK overview, feature list, file tree summary, reference to issue [#2044](https://github.com/NVIDIA/OpenShell/issues/2044), spec-driven development methodology note, and question about `specs/` directory retention (see research.md R7 for commit message template) +- [ ] T031 [US6] Squash all commits on the `go-sdk` branch into a single commit with DCO sign-off (`Signed-off-by: Roland Huß `) and comprehensive commit message +- [ ] T032 [US6] Open draft PR against `NVIDIA/OpenShell` main branch using `gh pr create --draft` +- [ ] T033 [US6] Verify PR file tree contains `sdk/go/` (source + proto + specs), `docs/sdks/go/` (MDX pages), `tasks/go.toml`, and modified `docs/index.yml` and `.github/workflows/branch-checks.yml` + +**Checkpoint**: Draft PR is live on GitHub with correct references, file tree, and description. + +--- + +## Phase 8: Polish & Cross-Cutting Concerns + +**Purpose**: Final validation across all stories + +- [ ] T034 Run quickstart.md verification checklist end-to-end +- [ ] T035 Final review of PR description for completeness and clarity +- [ ] T036 Verify SPDX license headers present on all new `.go` files in fork + +--- + +## Dependencies & Execution Order + +### Phase Dependencies + +- **Setup (Phase 1)**: No dependencies, start immediately +- **US1 (Phase 2)**: Depends on Setup. BLOCKS all other user stories. +- **US2 (Phase 3)**: Depends on US1 (needs new module path for examples repo dependency) +- **US3 (Phase 4)**: Depends on US1 (needs SDK API for code examples). Can parallel with US2, US4, US5. +- **US4 (Phase 5)**: Depends on US1 (needs proto tasks with new module path). Can parallel with US2, US3, US5. +- **US5 (Phase 6)**: Depends on US1 (specs copied after SDK source). Can parallel with US2, US3, US4. +- **US6 (Phase 7)**: Depends on US1, US2, US3, US4, US5 (all must complete) +- **Polish (Phase 8)**: Depends on US6 + +### User Story Dependencies + +```text +Setup ──► US1 (MVP) ──┬──► US2 ──────────────┐ + ├──► US3 (parallel) ──►│ + ├──► US4 (parallel) ──►├──► US6 ──► Polish + └──► US5 (parallel) ──►│ +``` + +### Within Each User Story + +- Copy/create before rewrite +- Rewrite before regenerate/tidy +- Regenerate before verify +- All verification tasks run last in their phase + +### Parallel Opportunities + +After US1 completes, three stories can execute in parallel: +- **US3** (Fern docs): Touches `docs/sdks/go/` and `docs/index.yml` +- **US4** (Proto CI): Touches `tasks/go.toml` and `.github/workflows/branch-checks.yml` +- **US5** (Spec scoping): Touches `sdk/go/specs/` + +No file conflicts between these three stories. + +Within US3, all four MDX pages (T019-T022) can be written in parallel since they are independent files. + +Within US4, the task file (T025) and CI workflow (T026) can be written in parallel. + +--- + +## Parallel Example: Post-US1 Fan-Out + +```bash +# After US1 checkpoint passes, launch three stories in parallel: + +# Story 3 (docs): +Task: "Create docs/sdks/go/getting-started.mdx" +Task: "Create docs/sdks/go/architecture.mdx" +Task: "Create docs/sdks/go/error-handling.mdx" +Task: "Create docs/sdks/go/authentication.mdx" + +# Story 4 (CI): +Task: "Create tasks/go.toml with go:proto task" +Task: "Add Go job to .github/workflows/branch-checks.yml" + +# Story 5 (specs): +Task: "Copy specs/ to sdk/go/specs/" +``` + +--- + +## Implementation Strategy + +### MVP First (User Story 1 Only) + +1. Complete Phase 1: Setup (T001-T003) +2. Complete Phase 2: US1 Module Path Migration (T004-T011) +3. **STOP and VALIDATE**: `go build ./...` and `go test ./...` pass, zero old module references +4. SDK is compilable under upstream path + +### Incremental Delivery + +1. Setup + US1 = SDK compiles under upstream path (MVP) +2. Add US2 = Examples extracted to separate repo +3. Add US3 + US4 + US5 in parallel = Docs, CI, specs ready +4. US6 = Draft PR opened on GitHub +5. Polish = Final validation + +### Parallel Team Strategy + +With multiple developers after US1: +- Developer A: US2 (Example Extraction) + US5 (Spec Scoping) +- Developer B: US3 (Fern Documentation) +- Developer C: US4 (Proto CI Automation) +- Everyone: US6 (PR assembly after all stories merge) + +--- + +## Notes + +- [P] tasks = different files, no dependencies on incomplete tasks +- [Story] label maps task to specific user story for traceability +- US1 is both the first user story AND the foundational blocker for all others +- US3, US4, US5 have zero file conflicts and can run fully in parallel +- All work happens in the rhuss/OpenShell fork, not the SDK development repo +- The SDK development repo (`openshell-sdk-go`) is read-only during this process (source of truth for copy)