Skip to content

Move bundle runtime, config, and cache from CLI to SDK (#59)#64

Open
shwetank-dev wants to merge 14 commits intomainfrom
issue-59-restructure-sdk
Open

Move bundle runtime, config, and cache from CLI to SDK (#59)#64
shwetank-dev wants to merge 14 commits intomainfrom
issue-59-restructure-sdk

Conversation

@shwetank-dev
Copy link
Copy Markdown
Collaborator

Summary

  • Stage 1: Clean up SDK exports — remove type aliases, use schema types directly. Consumers import response types from @nimblebrain/mpak-schemas.
  • Stage 2: Add ConfigManager class to SDK — manages ~/.mpak/config.json with per-package user config, registry URL, and file permissions.
  • Stage 3: Add BundleCache class and MpakSDK facade — local cache management (loadBundle, checkForUpdateAsync, extractZip), plus a top-level facade wiring config, client, and cache together.
  • Stage 4: Add prepareServer and parsePackageSpec to MpakSDK — resolves a registry bundle into a ready-to-spawn ServerCommand (command, args, env, cwd). Handles node/python/binary server types, user config substitution, and manifest validation via Zod schema. Narrows public exports to facade + types + errors only.

Stages 5 (update CLI to import from SDK) and 6 (adopt Biome) are planned as follow-ups.

Key design decisions

  • MpakSDK facade is the primary entry point — internal classes (MpakClient, ConfigManager, BundleCache) are accessed via mpak.client, mpak.config, mpak.cache
  • prepareServer returns a ServerCommand (command/args/env/cwd) rather than spawning — consumers control process lifecycle
  • parsePackageSpec is a public static method that validates scoped @scope/name[@version] format
  • --local bundle handling stays in CLI — it's a CLI-only developer workflow, not an SDK concern
  • Missing required user config throws — SDK validates, CLI catches and handles interactive prompting

Test plan

  • 171 unit tests pass (pnpm --filter @nimblebrain/mpak-sdk test)
  • 31 integration tests pass against live registry (pnpm --filter @nimblebrain/mpak-sdk test:integration)
  • Typecheck passes (pnpm --filter @nimblebrain/mpak-sdk typecheck)
  • Verify README accurately documents all public methods
  • Review manifest Zod schema against MCPB v0.3 spec

🤖 Generated with Claude Code

shwetank-dev and others added 8 commits March 21, 2026 13:19
…59 stage 1)

Deduplicate and prune the SDK's public API surface from ~25 type exports
down to 1 type export (MpakClientConfig) plus 6 value exports.

types.ts:
- Delete 22 alias/passthrough re-exports that just renamed schema types
  (BundleDetailResponse → BundleDetail, SkillDownloadResponse → SkillDownloadInfo, etc.)
- Delete Author interface (duplicate of SkillAuthor/PackageAuthor in schemas)
- Remove dead `surface` field from SkillSearchParams (registry ignores it)
- Keep BundleSearchParams and SkillSearchParams as SDK-local interfaces
  because the schema versions use z.default() which makes fields required
  in the inferred output type, incompatible with the SDK's all-optional
  client-side API

client.ts:
- Import response types (BundleDetail, VersionsResponse, VersionDetail,
  DownloadInfo, SkillDetail, SkillDownloadInfo, PlatformInfo) directly
  from @nimblebrain/mpak-schemas instead of local aliases
- Remove surface query param from searchSkills (dead code)
- Fix crypto import to use node: protocol

index.ts:
- Strip to 6 exports: MpakClient, MpakClientConfig, 4 error classes
- Consumers needing response types import from @nimblebrain/mpak-schemas

client.test.ts:
- Remove surface param and assertion from skill search test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move the config management logic from the CLI into the SDK so that
BundleCache (stage 3) and BundleRuntime (stage 4) can access registry
URL and per-package user_config values without a circular dependency
on the CLI.

New file: src/config-manager.ts
- Zod schema (MpakConfigSchema) replaces the hand-rolled validation from
  the CLI's original implementation — .strict() rejects unknown fields
- ConfigManager constructor accepts an optional configDir parameter
  (defaults to ~/.mpak) for testability and custom deployments
- Public methods grouped separately from private methods
- Full JSDoc on every public method, type, and error class
- MpakConfig and PackageConfig types inferred from the Zod schema

New file: tests/config-manager.test.ts (29 tests)
- All tests use OS temp directories — never touches ~/.mpak
- Covers: construction, load/save round-trip, caching, registry URL
  resolution (saved > env var > default), per-package config CRUD,
  empty entry cleanup, file permissions (0o600), and all validation
  error paths (invalid JSON, missing fields, wrong types, unknown fields)

index.ts:
- Export ConfigManager, ConfigCorruptedError, CONFIG_VERSION (values)
- Export MpakConfig, PackageConfig (types)

package.json:
- Add zod as an explicit dependency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BundleCache manages the local bundle cache (~/.mpak/cache/), handling
download, extraction, metadata tracking, and update checks. Moved from
CLI to SDK so both CLI and future consumers share the same cache logic.

Key design decisions:
- Optional constructor injection: MpakClient is passed via constructor
  options, required only for registry operations (loadBundle,
  checkForUpdateAsync). Local-only operations (metadata, listing) work
  without a client.
- Explicit logging over silent swallowing: checkForUpdateAsync logs
  "up to date", TTL skip with remaining time, and error messages
  instead of silently catching all errors.
- 1-hour TTL on update checks prevents redundant registry calls.

MpakSDK facade wires ConfigManager → MpakClient → BundleCache in a
single constructor. ConfigManager is the sole source of truth for
registry URL — if passed via MpakSDKOptions, it's persisted to config
before the client reads it.

Also refactored ConfigManager to accept an options object instead of
positional args, matching BundleCache's constructor pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ds (#59 stage 3)

Fix: loadBundle no longer stamps lastCheckedAt on fresh downloads.
lastCheckedAt is meant to gate the TTL for checkForUpdateAsync — a
first-time pull is not an "update check", so stamping it incorrectly
caused checkForUpdateAsync to skip its first real check after a pull.
The "registry resolved same version" path still stamps it correctly.

Tests: reorganize facade tests by component and add integration smoke
tests against the live registry using @nimblebraininc/echo.

- mpak.test.ts (15): pure facade wiring — construction, option
  propagation, cross-component integration, MpakClient standalone
- cache.test.ts (+6): "via MpakSDK facade" block — loadBundle
  through wired client, cache hit on second call, checkForUpdateAsync,
  registry URL propagation, local-only ops, listCachedBundles
- config-manager.test.ts (+3): "via MpakSDK facade" block — config
  persistence across instances, registry URL persistence, multi-package
- mpak.integration.test.ts (10 new): end-to-end smoke against live
  registry — download, cache metadata, list, cache hit, force pull,
  update check, TTL skip, config alongside cache, fresh instance reuse

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Relative paths passed as mpakHome now get resolved to absolute via
path.resolve(), so all derived paths (config file, cache dirs) are
always absolute. Also simplifies _mpakHome to a public readonly field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Define a Zod schema for the MCPB v0.3 manifest format (previously an
unvalidated local interface in the CLI's run command). Add a public
readManifest(packageName) method to BundleCache that reads, parses, and
validates manifest.json from a package's cache directory, returning null
when the package isn't cached or the manifest is missing/corrupt.

Narrow SDK public exports to the MpakSDK facade, types, and errors —
internal classes (MpakClient, ConfigManager, BundleCache) are accessed
through the facade rather than imported directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add prepareServer() to the MpakSDK facade — the main public API for
resolving a registry bundle into a ready-to-spawn server command. It
handles the full lifecycle: parse package spec, load/cache the bundle,
read the manifest, validate required user config, and resolve the
command/args/env based on server type (node/python/binary).

Add parsePackageSpec() as a public static method that parses and
validates scoped package specs (@scope/name or @scope/name@version),
throwing on invalid format.

New types: PrepareServerOptions (version pinning, force, env overrides,
workspaceDir) and ServerCommand (command, args, env, cwd returned).

Narrow public exports to facade + types + errors only — internal
classes accessed through the facade, not imported directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add integration smoke tests for readManifest and prepareServer against
the live registry using @nimblebraininc/echo. Verifies manifest parsing,
server command resolution, workspaceDir option, and inline version
parsing with real downloaded bundles.

Rewrite README to reflect the current public API surface: MpakSDK
facade as primary entry point, prepareServer workflow, user config,
parsePackageSpec, and full API reference tables for all public methods
on MpakSDK, MpakClient, BundleCache, and ConfigManager. Remove
references to non-existent methods (resolveSkillRef, downloadSkillContent).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@shwetank-dev
Copy link
Copy Markdown
Collaborator Author

@mgoldsborough Claude on why no run in the SDK

  1. Flexibility — different consumers want different spawn behavior.
    The CLI wants stdio: 'inherit' (transparent passthrough). A
    programmatic consumer might want stdio: 'pipe' to read/write MCP
    JSON-RPC messages. Someone else might want to run it in Docker. If
    the SDK spawns, it has to either pick one or expose a growing set of
    options.
  2. Process lifecycle is the caller's concern — signal forwarding,
    exit handling, restart logic, timeouts. The CLI does
    process.on('SIGINT', () => child.kill('SIGINT')) and
    child.on('exit', (code) => process.exit(code)). Other consumers
    would handle this differently. The SDK shouldn't own any of that.
  3. The hard part is resolution, not spawning — downloading, caching,
    manifest parsing, config validation, command resolution across
    node/python/binary types — that's what takes work. The actual
    spawn(command, args, { env, cwd }) call is one line. The SDK does
    the complex part, the consumer does the trivial part.

@mgoldsborough
Copy link
Copy Markdown
Contributor

mgoldsborough commented Mar 25, 2026

Solid foundation - a few things before merge:

The facade design is right. prepareServer → ServerCommand is exactly the interface both consumers need. Three things to address:

1. Export the components from index.ts

Right now only MpakSDK is exported. The CLI needs ConfigManager standalone (for config set), MpakClient standalone (for search), BundleCache standalone (for outdated). Export all three classes plus their public types (CachedBundle, PackageConfig, MpakConfig, ConfigCorruptedError). The facade is the recommended path, not the only path.

Ideal Public API Surface

// ─── Facade (80% use case) ───
export { MpakSDK } from './MpakSDK.js';                                                                                           
export type { MpakSDKOptions, PrepareServerOptions, ServerCommand } from './MpakSDK.js';                                          
                                                                                                                                    
// ─── Components (advanced / CLI use) ───                                                                                        
export { MpakClient } from './client.js';                                                                                         
export { BundleCache } from './cache.js';                                                                                         
export { ConfigManager } from './config-manager.js';
                                                                                                                                    
// ─── Utilities ───                                                                                                              
export { parsePackageSpec } from './utils.js';
                                                                                                                                    
// ─── Types ───                                                
export type { MpakClientConfig, BundleSearchParams, SkillSearchParams } from './types.js';
export type { McpbManifest, UserConfigField, CachedBundle } from './cache.js';
export type { MpakConfig, PackageConfig } from './config-manager.js';

// ─── Errors (all typed, all catchable) ───
export {
  MpakError,
  MpakNotFoundError,
  MpakIntegrityError,
  MpakNetworkError,
  MpakConfigError,       // NEW: missing credentials, carries structured field list
  MpakManifestError,     // NEW: corrupt/missing manifest                                                                         
  ConfigCorruptedError,  // EXISTS but not exported                                                                               
} from './errors.js';                                        

2. Typed error for missing credentials

prepareServer throws a bare Error with a formatted string when required config is missing. The CLI needs to catch this, figure out which fields are missing, prompt for them, and retry. An agent consumer needs the same structured data for vault lookups. Add a MpakConfigError that carries packageName and a missingFields array (key, title, sensitive). Same treatment for the other bare Error throws (manifest missing, unsupported server type) — use MpakError subclasses so consumers can instanceof instead of string-matching.

3. parsePackageSpec as a standalone export

It's duplicated in run.ts, pull.ts, and as a static method on MpakSDK. It's a pure string function — export it from index.ts directly. Every CLI command needs it and shouldn't have to import the facade class to get it.

None of this changes the architecture. This is about surfacing what's already there and making the error contract reliable for programmatic consumers.

@mgoldsborough
Copy link
Copy Markdown
Contributor

BundleCache.downloadAndExtract and MpakClient.fetchWithTimeout both call global fetch directly. Every test that touches downloads or registry calls will need to mock the global, which bleeds between tests. Accept an optional fetch function in the constructor options now. Retrofitting it later is a breaking change to the public API.

@mgoldsborough
Copy link
Copy Markdown
Contributor

Other items to consider:

  • Move ensureConfigDir() from constructor to first write — mpak search shouldn't mkdir ~/.mpak/
  • Local .mcpb support in prepareServer (the ~90 lines in run.ts are pure logic with no TTY deps — they belong in the SDK)
  • CachedBundle interface isn't exported from cache.ts

@mgoldsborough
Copy link
Copy Markdown
Contributor

The SDK has zero test files. This is the shared foundation for both CLI and agent — it needs coverage before merge. The CLI already has a thorough config-manager.test.ts (~450 lines) that should move with the code. prepareServer needs at minimum: cache-hit, cache-miss, and missing-credentials paths covered. BundleCache.loadBundle needs: fresh download, cached version match, and force-redownload.

@shwetank-dev
Copy link
Copy Markdown
Collaborator Author

Re: injectable fetch

Good callout on testability. I looked into this and since Vitest isolates each test file in its own worker context, vi.stubGlobal scoped with afterEach cleanup gives solid isolation without cross-file bleed — so I'm comfortable with the testing story as-is.

On the API design side — injectable fetch is a nice pattern for things like proxy support or custom retry wrappers, but there's no concrete consumer need for it today. Since adding an optional fetch param to an options object is non-breaking, I can add it later when there's a real use case.

Happy to revisit if you feel strongly though.

…PR review)

Address PR #64 review feedback to make SDK classes usable independently:

ConfigManager → MpakConfigManager:
- Rename to MpakConfigManager with MpakConfigManagerOptions interface
- Make loadConfig() private — consumers use named getters/setters only
- Lazy directory creation: constructor no longer creates ~/.mpak,
  deferred to first write via saveConfig()
- Remove getPackageConfigValue() — use getPackageConfig() instead
- Remove env var fallback from getRegistryUrl() (2-tier: config > default)
- Make setRegistryUrl() private — only callable via constructor option
- Rename listPackagesWithConfig() → getPackageNames()
- Validate config against Zod schema before writing to disk
- Throw invariant error if saveConfig() called before config loaded

Typed errors:
- Move ConfigCorruptedError to errors.ts as MpakConfigCorruptedError
  (extends MpakError with code 'CONFIG_CORRUPTED')
- Add MpakConfigError for missing required user config fields
  (carries packageName and structured missingFields array)
- Export both new errors plus MpakConfigManager from index.ts

Tests updated to reflect all API changes (170 pass).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
shwetank-dev and others added 5 commits March 26, 2026 14:00
…59 PR review)

- Rename BundleCache → MpakBundleCache, update constructor to require MpakClient
- Move extractZip, isSemverEqual, constants to helpers.ts (plain functions)
- Remove logger pattern: checkForUpdate returns string|null instead of logging
- Use mpakClient.downloadContent (with integrity check) instead of raw fetch
- Download temp files to os.tmpdir(), cleanup in finally block
- Add removeCachedBundle method
- Inline resolveFromRegistry (single-use wrapper)
- Make writeCacheMetadata private, drop write-side schema validation
- Move McpbManifest schema + CachedBundleInfo to @nimblebrain/mpak-schemas
- Rewrite cache.test.ts for new API, add helpers.test.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…PR review)

- Rename MpakSDK → Mpak, MpakSDKOptions → MpakOptions
- Rename fields: config → configManager, cache → bundleCache
- Replace all bare throw new Error() with typed errors across SDK
- Fix env merge ordering so caller env overrides MPAK_WORKSPACE default
- Extract parsePackageSpec to utils.ts as standalone export
- Add exhaustive switch check in resolveCommand
- Update index.ts exports and tests for new names
- Rename file MpakSDK.ts → mpakSDK.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all manifest-related Zod schemas (ServerType, UserConfigField,
McpConfig, ManifestAuthor, ManifestServer, Capability, McpbManifest)
out of package.ts into a new manifest.ts as the single source of truth.
package.ts re-exports these for backward compatibility.

The upstream MCPB spec v0.3 and v0.4 are backward-compatible (v0.4 only
adds "uv" as a server runtime type), so a single permissive schema
handles both versions without a discriminated union.

- Add "uv" to ServerTypeSchema enum
- Add "uv" case to SDK resolveCommand (defaults to `uv run <entry_point>`)
- Export manifest.ts from schemas barrel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mgoldsborough mgoldsborough self-requested a review March 28, 2026 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants