feat: add init and upload in kernel-builder cli by drbh · Pull Request #378 · huggingface/kernels

drbh · 2026-03-18T17:44:45Z

This PR is a WIP to add the init command to the kernel-builder rust cli. This branch mainly ports the init command from the python cli, with some improvements to the interface and updates to the template

This PR adds an the init and upload command to the rust kernel-builder cli similar to implementation in the current python cli. It also adds a build command that is a facade for the nix run so devs can interact with a single tool for the full creation lifecycle of a kernel.

Mainly it aims to have better defaults to make the commands easier to use

the init command only requires the name of the kernels (folder) and the owner is pulled via a whoami request if the user is already logged into hf
the init command creates a new dir if a name is specified, if called within a dir, the cli will use the dir name as the kernel name
the upload command does not require a repo id if the id can be read from the build
the upload command does not require a path to the build folder and will default to looking for the build dir if no explicit path is specified

example usage

init from inside of a existing dir

mkdir some-kernel
cd some-kernel
kernel-builder init
# Downloading template from kernels-community/template...
# Initialized `drbh/some-kernelt` at /home/drbh/Projects/kernels/kernel-builder/some-kernel

init from outside of a dir

kernel-builder init some-kernel
# Downloading template from kernels-community/template...
# Initialized `drbh/some-kernelt` at /home/drbh/Projects/kernels/kernel-builder/some-kernel

build

kernel-builder build
# ...
# some-kernel-torch-ext> no Makefile or custom installCheckPhase, doing nothing
# some-kernel-torch-ext> Checking of ABI compatibility
# some-kernel-torch-ext> 🐍 Checking for compatibility with manylinux_2_28 and Python ABI version 3.9
# some-kernel-torch-ext> ✅ No compatibility issues found
# some-kernel-torch-ext> Checking loading kernel with get_kernel
# some-kernel-torch-ext> Check whether the kernel can be loaded with get-kernel: some_kernel
# some-kernel-torch-ext> Running phase: removeBytecodeHook
# some-kernel-torch-ext> Removing Python bytecode

upload

kernel-builder upload
# Found 7 build variant(s) in /home/drbh/Projects/kernels/kernel-builder/some-kernel/build
# Using branch `v1` (new)
# Uploading 36 operations...
# Kernel upload successful. Find the kernel at: https://hf.co/drbh/some-kernel

sayakpaul · 2026-03-20T05:09:35Z

Some high-level thoughts first:

the init command only requires the name of the kernels (folder) and the owner is pulled via a whoami request if the user is already logged into hf

Should we somehow inform the users about it and take confirmation before proceeding?

# Kernel upload successful. Find the kernel at: https://hf.co/drbh/some-kernel

Should this be Find the kernel at: https://hf.co/drbh/some-kernel/tree/{version}? This way, users can directly use that link to inspect files, etc.

sayakpaul

Very minor comments. No RUST expertise yet.

sayakpaul · 2026-03-20T05:30:45Z

template/CARD.md

@@ -0,0 +1,38 @@
+---


Do we want to maintain two copies? We have one in https://github.com/huggingface/kernels/blob/main/kernels/src/kernels/cli/card_template.md

danieldk · 2026-03-20T12:41:42Z

kernel-builder/src/build.rs

+
+    /// Nix flake target to run.
+    #[arg(long, default_value = "build-and-copy")]
+    pub target: BuildTarget,


I think top-level

kernel-builder build kernel-builder build-and-copy kernel-builder build-and-upload

are nicer, easier to discover. I think we can use #[command(flatten)] to share arguments between different build commands, something like:

// ... Build { #[command(flatten)] pub common_build_args: CommonBuildArgs, // ... }

danieldk · 2026-03-20T12:42:58Z

kernel-builder/src/build.rs

+
+    /// Additional arguments passed through to `nix run`.
+    #[arg(last = true)]
+    pub nix_args: Vec<String>,


Not sure if we want this?

danieldk · 2026-03-20T12:47:07Z

kernel-builder/src/init.rs

+                let bare_name = path.file_name().and_then(OsStr::to_str).ok_or_else(|| {
+                    eyre::eyre!("Cannot determine directory name from `{path_str}`")
+                })?;
+                let owner = hf::whoami_username()?;


danieldk · 2026-03-20T12:51:08Z

kernel-builder/src/init.rs

+                for (from, to) in replacements {
+                    text = text.replace(from, to);
+                }
+                fs::write(&destination, text).wrap_err_with(|| {


I possible (not sure if it supports everything needed), I think it would be nice to use the fileset API that is also used for generating pyproject files. The nice thing is that it is atomic, if something fails while still preparing the output, nothing is written.

I think we would have to extend it with the notion of files that should not be overwritten (if a project already exists and you only want to add new files). But I think making it impossible to have an operation half-completed is a nice feature.

danieldk · 2026-03-20T12:53:42Z

kernel-builder/src/init.rs

+        .wrap_err_with(|| format!("Cannot parse TOML in `{}`", build_toml_path.display()))?;
+
+    // Update [general].backends
+    if let Some(general) = document.get_mut("general").and_then(|v| v.as_table_mut()) {


I forget if we discussed this before, but it would be worthwhile putting jinja templates in the template, so that we can render everything editing TOML (but maybe parse + write to format).

This has the benefit that if we ever change the build.toml format, we don't have to change it both in the template and here in the code.

E.g., suppose that backends moves somewhere else, we have to change it in the template and here.

danieldk · 2026-03-20T12:54:37Z

kernel-builder/src/init.rs

+    let mut impl_block = String::new();
+    for (i, (condition, device, ref_prefix)) in conditions.iter().enumerate() {
+        let directive = if i == 0 { "#if" } else { "#elif" };
+        impl_block.push_str(directive);
+        impl_block.push(' ');
+        impl_block.push_str(condition);
+        impl_block.push_str("\n  ops.impl(\"");
+        impl_block.push_str(func_name);
+        impl_block.push_str("\", ");
+        impl_block.push_str(device);
+        impl_block.push_str(", ");
+        impl_block.push_str(ref_prefix);
+        impl_block.push_str(func_name);
+        impl_block.push_str(");\n");
+    }


Could be a jinja template with if blocks.

danieldk · 2026-03-20T12:55:14Z

kernel-builder/src/init.rs

+    let marker = "ops.def(\"";
+    let start = content.find(marker)? + marker.len();
+    let rest = &content[start..];
+    let end = rest.find('(')?;
+    Some(&rest[..end])
+}
+
+/// Replace `#if defined(CPU_KERNEL)...#endif` block with new content
+fn replace_ifdef_block(content: &str, replacement: &str) -> String {
+    const START_MARKER: &str = "#if defined(CPU_KERNEL)";
+    const END_MARKER: &str = "#endif";
+
+    let Some(start) = content.find(START_MARKER) else {
+        return content.to_owned();
+    };
+
+    let search_region = &content[start..];
+    let Some(end_offset) = search_region.find(END_MARKER) else {
+        return content.to_owned();
+    };
+
+    let end = start + end_offset + END_MARKER.len();
+
+    let mut result = String::with_capacity(content.len());
+    result.push_str(&content[..start]);
+    result.push_str(replacement);
+    result.push_str(&content[end..]);
+    result
+}


Would be easier to maintain with a jinja template and some if blocks I think.

danieldk · 2026-03-20T12:56:05Z

kernel-builder/src/main.rs

+    /// Upload kernel build artifacts to the Hugging Face Hub.
+    Upload(UploadArgs),
+
+    #[command(hide = true)]


If I understand correctly, this removes the subcommand from help. Sounds bad? Leftover from testing?

drbh temporarily deployed to testpypi March 19, 2026 15:24 — with GitHub Actions Inactive

drbh changed the title ~~feat: support init in builder cli~~ feat: add init and upload in kernel-builder cli Mar 19, 2026

drbh added 2 commits March 19, 2026 11:25

feat: support init in builder cli

fb0a3b5

feat: add upload support

368adad

drbh force-pushed the kernel-builder-init-command branch from 534af0b to 368adad Compare March 19, 2026 15:25

drbh temporarily deployed to testpypi March 19, 2026 15:25 — with GitHub Actions Inactive

fix: use cwd when init called within a dir

c35424a

drbh temporarily deployed to testpypi March 19, 2026 15:38 — with GitHub Actions Inactive

feat: add build that wraps nix commands

a314749

drbh temporarily deployed to testpypi March 19, 2026 15:53 — with GitHub Actions Inactive

drbh added 3 commits March 19, 2026 12:09

fix: clippy clean up

47a3b36

fix: update minijinja apis

ab13016

fix: lock rust dep hashes

bf97126

drbh temporarily deployed to testpypi March 19, 2026 16:32 — with GitHub Actions Inactive

drbh marked this pull request as ready for review March 19, 2026 16:42

fix: improve getting started

2f14b74

drbh temporarily deployed to testpypi March 19, 2026 16:54 — with GitHub Actions Inactive

sayakpaul reviewed Mar 20, 2026

View reviewed changes

danieldk reviewed Mar 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add init and upload in kernel-builder cli#378

feat: add init and upload in kernel-builder cli#378
drbh wants to merge 8 commits intomainfrom
kernel-builder-init-command

drbh commented Mar 18, 2026 •

edited

Loading

Uh oh!

sayakpaul commented Mar 20, 2026 •

edited

Loading

Uh oh!

sayakpaul left a comment

Uh oh!

sayakpaul Mar 20, 2026

Uh oh!

danieldk Mar 20, 2026 •

edited

Loading

Uh oh!

danieldk Mar 20, 2026

Uh oh!

danieldk Mar 20, 2026

Uh oh!

danieldk Mar 20, 2026 •

edited

Loading

Uh oh!

danieldk Mar 20, 2026

Uh oh!

danieldk Mar 20, 2026

Uh oh!

danieldk Mar 20, 2026 •

edited

Loading

Uh oh!

danieldk Mar 20, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

drbh commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sayakpaul commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

sayakpaul Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danieldk Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

drbh commented Mar 18, 2026 •

edited

Loading

sayakpaul commented Mar 20, 2026 •

edited

Loading

danieldk Mar 20, 2026 •

edited

Loading

danieldk Mar 20, 2026 •

edited

Loading

danieldk Mar 20, 2026 •

edited

Loading

danieldk Mar 20, 2026 •

edited

Loading