feat(functions push): --manifest path for non-SDK function types#150
Open
Imen (imen-kedir) wants to merge 1 commit intobraintrustdata:mainfrom
Open
feat(functions push): --manifest path for non-SDK function types#150Imen (imen-kedir) wants to merge 1 commit intobraintrustdata:mainfrom
Imen (imen-kedir) wants to merge 1 commit intobraintrustdata:mainfrom
Conversation
The runner-based push path can only ship function types the Braintrust
SDK exposes builders for: tools, scorers, prompts, parameters. Topics-
pipeline types (facet, classifier) and inline-quickjs preprocessors
have no SDK builder, so AI-agent workflows that author lenses cannot
push them through `bt`.
`--manifest <path>` accepts a JSON file with three shapes — the
`/insert-functions` wire body `{"functions": [...]}`, a bare entry
array, or a single entry object. The body bypasses the SDK runner
and is posted directly to `/insert-functions` (the same endpoint
`bt topics config enable` uses to create classifier functions).
Reuses the runner path's preflight machinery: `parse_project_selector`
+ `add_selector_requirement` for project classification,
`validate_direct_project_ids` for direct-id verification,
`resolve_named_projects` (honors `--create-missing-projects`) +
`resolve_default_project_id` for project resolution, and
`validate_duplicate_slugs` for cross-specifier collision detection.
A pre-resolution dedup pass canonicalizes Fallback to its default
project name so duplicate slugs never trigger fresh-project creation
on the server.
Failure paths emit a manifest-aware `PushSummary` with `total_files: 1`
and `source_file: <manifest-path>` so JSON consumers see the right
context. Two-step parse distinguishes `ManifestInvalidJson` from
`ManifestSchemaInvalid`. Spinner is suppressed in `--json` mode to
keep stderr clean.
Mutually exclusive only with `--file` and the positional file paths
(the genuinely ambiguous overlap). Runner-specific options
(`--runner`, `--language`, etc.) are silently ignored on this path
so env-backed defaults don't block the command.
Closes braintrustdata#149.
158f844 to
2263927
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #149.
Summary
bt functions push --manifest <path>that posts a raw JSON function definition directly to/insert-functions, bypassing the SDK runner.data.type: "bundle"withruntime: "node", which the Topics aggregator rejects:Brainstore preprocessor aggregate only supports built-in 'thread' and inline quickjs preprocessors).Manifest shape
Three accepted shapes — pick whichever fits how you generate the JSON:
{ "functions": [ /* entry */, /* entry */ ] } # exact /insert-functions wire body [ /* entry */, /* entry */ ] # entry array { /* single entry */ } # single objectEach entry minimally carries
nameandslug(everything else, includingfunction_typeandfunction_data, is forwarded verbatim — the server validates the wire shape):{ "name": "my-lens-issues", "slug": "my-lens-issues-v1", "function_type": "facet", "function_data": { "type": "facet", "prompt": "...", "no_match_pattern": "^NONE", "preprocessor": { "id": "<preprocessor-function-id>", "type": "function" } } }Inline-quickjs preprocessor:
{ "name": "my-lens-preprocessor", "slug": "my-lens-preprocessor-v1", "function_type": "preprocessor", "function_data": { "type": "code", "data": { "type": "inline", "code": "function handler({ span_attributes, input }) { /* ... */ }", "runtime_context": { "runtime": "quickjs", "version": "ES2023" } } } }What's reused from the runner path
The manifest path leans on the existing runner machinery so behavior matches:
parse_project_selector+add_selector_requirementclassify per-entry selectors;resolve_named_projects(honors--create-missing-projects) +resolve_default_project_idresolve them in bulk.validate_direct_project_idschecks accessible direct IDs;validate_duplicate_slugscatches cross-specifier slug collisions.Fallbackto its default project name so a manifest with duplicate slugs and-p new-projectis rejected before the project gets created on the server.build_push_confirm_promptas the runner path (shows<file> to <org>/<projects>); fires before any project-creation API call.Behavior details
--file, positional paths). Runner-specific flags (--runner,--language,--requirements,--tsconfig,--external-packages) are silently ignored on this path so env-backed defaults don't block the command.PushSummarywithtotal_files: 1andfiles[0].source_file = <manifest-path>so--jsonconsumers see the right context.ManifestInvalidJson(raw JSON syntax) fromManifestSchemaInvalid(field/shape errors).--jsonmode to keep stderr clean for machine readers.--if-existsdefaults are applied to manifest entries that don't specify their own.Test plan
RawFunctionEntry's minimal-shape requirements (name+slugonly; everything else optional).--manifestparses,BT_FUNCTIONS_PUSH_MANIFESTenv var works,--manifestconflicts with file paths.--manifestappears in--help,BT_FUNCTIONS_PUSH_MANIFESTlisted as env,--manifest <p> file.tserrors at parse-time.functions_push_manifest_works_against_mock_api— happy path; asserts function_data passes through and--if-existsflows.functions_push_manifest_inline_quickjs_preprocessor_array— original use case from bt functions push: support raw JSON manifest for facet and inline-quickjs preprocessor types #149; preprocessor + facet round-trip.functions_push_manifest_accepts_wrapped_shape—{"functions": [...]}wire body.functions_push_manifest_ignores_runner_env_vars— exportsBT_FUNCTIONS_PUSH_RUNNER/LANGUAGE/TSCONFIGand asserts the push still succeeds.functions_push_manifest_creates_missing_default_project—-ppoints at a non-existent project; verifies--create-missing-projectshonored.functions_push_manifest_duplicate_slugs_dont_create_fresh_project— covers two alias shapes (both fallback, fallback + explicit name); asserts no project AND no functions left behind.functions_push_manifest_json_failure_reports_manifest_path— malformed JSON; assertsfiles[0].source_filepopulated anderror_reason: manifest_invalid_json.cargo test --bin bt(449) andcargo test --test functions(34) all pass.rustfmt --checkclean.🤖 Generated with Claude Code