Skip to content

feat(functions push): --manifest path for non-SDK function types#150

Open
Imen (imen-kedir) wants to merge 1 commit intobraintrustdata:mainfrom
imen-kedir:imen/manifest-push-149
Open

feat(functions push): --manifest path for non-SDK function types#150
Imen (imen-kedir) wants to merge 1 commit intobraintrustdata:mainfrom
imen-kedir:imen/manifest-push-149

Conversation

@imen-kedir
Copy link
Copy Markdown

@imen-kedir Imen (imen-kedir) commented Apr 24, 2026

Closes #149.

Summary

  • Adds bt functions push --manifest <path> that posts a raw JSON function definition directly to /insert-functions, bypassing the SDK runner.
  • Use case: function types the SDK doesn't expose builders for — facets, classifiers, and inline-quickjs preprocessors (the shape Topics-pipeline preprocessors need to be executable; SDK pushes always emit data.type: "bundle" with runtime: "node", which the Topics aggregator rejects: Brainstore preprocessor aggregate only supports built-in 'thread' and inline quickjs preprocessors).

Manifest shape

Three accepted shapes — pick whichever fits how you generate the JSON:

{ "functions": [ /* entry */, /* entry */ ] }   # exact /insert-functions wire body
[ /* entry */, /* entry */ ]                     # entry array
{ /* single entry */ }                           # single object

Each entry minimally carries name and slug (everything else, including function_type and function_data, is forwarded verbatim — the server validates the wire shape):

{
  "name": "my-lens-issues",
  "slug": "my-lens-issues-v1",
  "function_type": "facet",
  "function_data": {
    "type": "facet",
    "prompt": "...",
    "no_match_pattern": "^NONE",
    "preprocessor": { "id": "<preprocessor-function-id>", "type": "function" }
  }
}

Inline-quickjs preprocessor:

{
  "name": "my-lens-preprocessor",
  "slug": "my-lens-preprocessor-v1",
  "function_type": "preprocessor",
  "function_data": {
    "type": "code",
    "data": {
      "type": "inline",
      "code": "function handler({ span_attributes, input }) { /* ... */ }",
      "runtime_context": { "runtime": "quickjs", "version": "ES2023" }
    }
  }
}

What's reused from the runner path

The manifest path leans on the existing runner machinery so behavior matches:

  • Project resolution: parse_project_selector + add_selector_requirement classify per-entry selectors; resolve_named_projects (honors --create-missing-projects) + resolve_default_project_id resolve them in bulk.
  • Validation: validate_direct_project_ids checks accessible direct IDs; validate_duplicate_slugs catches cross-specifier slug collisions.
  • Pre-resolution dedup: an early pass canonicalizes Fallback to its default project name so a manifest with duplicate slugs and -p new-project is rejected before the project gets created on the server.
  • Auth gates: the auth context check, app-url validation, and explicit-org validation that the runner path runs upfront also gate the manifest path now.
  • Confirmation: same build_push_confirm_prompt as the runner path (shows <file> to <org>/<projects>); fires before any project-creation API call.

Behavior details

  • Mutually exclusive with the input-source flags (--file, positional paths). Runner-specific flags (--runner, --language, --requirements, --tsconfig, --external-packages) are silently ignored on this path so env-backed defaults don't block the command.
  • Failure paths emit a manifest-aware PushSummary with total_files: 1 and files[0].source_file = <manifest-path> so --json consumers see the right context.
  • Two-step parse distinguishes ManifestInvalidJson (raw JSON syntax) from ManifestSchemaInvalid (field/shape errors).
  • Spinner is suppressed in --json mode to keep stderr clean for machine readers.
  • --if-exists defaults are applied to manifest entries that don't specify their own.

Test plan

  • Unit tests for RawFunctionEntry's minimal-shape requirements (name + slug only; everything else optional).
  • Clap parser tests: --manifest parses, BT_FUNCTIONS_PUSH_MANIFEST env var works, --manifest conflicts with file paths.
  • CLI fixtures: --manifest appears in --help, BT_FUNCTIONS_PUSH_MANIFEST listed as env, --manifest <p> file.ts errors at parse-time.
  • Mock-API integration coverage:
    • functions_push_manifest_works_against_mock_api — happy path; asserts function_data passes through and --if-exists flows.
    • functions_push_manifest_inline_quickjs_preprocessor_array — original use case from bt functions push: support raw JSON manifest for facet and inline-quickjs preprocessor types #149; preprocessor + facet round-trip.
    • functions_push_manifest_accepts_wrapped_shape{"functions": [...]} wire body.
    • functions_push_manifest_ignores_runner_env_vars — exports BT_FUNCTIONS_PUSH_RUNNER/LANGUAGE/TSCONFIG and asserts the push still succeeds.
    • functions_push_manifest_creates_missing_default_project-p points at a non-existent project; verifies --create-missing-projects honored.
    • functions_push_manifest_duplicate_slugs_dont_create_fresh_project — covers two alias shapes (both fallback, fallback + explicit name); asserts no project AND no functions left behind.
    • functions_push_manifest_json_failure_reports_manifest_path — malformed JSON; asserts files[0].source_file populated and error_reason: manifest_invalid_json.
  • Existing cargo test --bin bt (449) and cargo test --test functions (34) all pass.
  • rustfmt --check clean.

🤖 Generated with Claude Code

The runner-based push path can only ship function types the Braintrust
SDK exposes builders for: tools, scorers, prompts, parameters. Topics-
pipeline types (facet, classifier) and inline-quickjs preprocessors
have no SDK builder, so AI-agent workflows that author lenses cannot
push them through `bt`.

`--manifest <path>` accepts a JSON file with three shapes — the
`/insert-functions` wire body `{"functions": [...]}`, a bare entry
array, or a single entry object. The body bypasses the SDK runner
and is posted directly to `/insert-functions` (the same endpoint
`bt topics config enable` uses to create classifier functions).

Reuses the runner path's preflight machinery: `parse_project_selector`
+ `add_selector_requirement` for project classification,
`validate_direct_project_ids` for direct-id verification,
`resolve_named_projects` (honors `--create-missing-projects`) +
`resolve_default_project_id` for project resolution, and
`validate_duplicate_slugs` for cross-specifier collision detection.
A pre-resolution dedup pass canonicalizes Fallback to its default
project name so duplicate slugs never trigger fresh-project creation
on the server.

Failure paths emit a manifest-aware `PushSummary` with `total_files: 1`
and `source_file: <manifest-path>` so JSON consumers see the right
context. Two-step parse distinguishes `ManifestInvalidJson` from
`ManifestSchemaInvalid`. Spinner is suppressed in `--json` mode to
keep stderr clean.

Mutually exclusive only with `--file` and the positional file paths
(the genuinely ambiguous overlap). Runner-specific options
(`--runner`, `--language`, etc.) are silently ignored on this path
so env-backed defaults don't block the command.

Closes braintrustdata#149.
@imen-kedir Imen (imen-kedir) marked this pull request as ready for review April 25, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bt functions push: support raw JSON manifest for facet and inline-quickjs preprocessor types

1 participant