Skip to content

[pull] master from ggml-org:master #1082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pull merged 4 commits into from
Apr 16, 2026
Merged

[pull] master from ggml-org:master #1082

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ae2d348
metal: Implement ROLL op (#21946)
kushagharahi Apr 16, 2026
3f7c29d
ggml: add graph_reused (#21764)
am17an Apr 16, 2026
03b3d07
Convert: Fix NemotronH Config Parsing (#21664)
anavp-nvidia Apr 16, 2026
b572d1e
codeowners: add team member comments (#21714)
0cc4m Apr 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion CODEOWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# collaborators can optionally add themselves here to indicate their availability for reviewing related PRs
# multiplie collaborators per item can be specified
# multiple collaborators per item can be specified
#
# ggml-org/ci : CISC, danbev, ggerganov, netrunnereve, ngxson, taronaeo
# ggml-org/ggml-cann : hipudding
# ggml-org/ggml-cuda : JohannesGaessler, am17an, IMbackK, ORippler
# ggml-org/ggml-hexagon : lhez, max-krasnyansky
# ggml-org/ggml-metal : ggerganov
# ggml-org/ggml-opencl : lhez, max-krasnyansky
# ggml-org/ggml-rpc : rgerganov
# ggml-org/ggml-sycl : arthw
# ggml-org/ggml-vulkan : 0cc4m, jeffbolznv
# ggml-org/ggml-webgpu : reeselevine
# ggml-org/ggml-zdnn : taronaeo
# ggml-org/llama-common : ggerganov, aldehir, angt, danbev, ngxson, pwilkin
# ggml-org/llama-mtmd : ngxson
# ggml-org/llama-server : ggerganov, ngxson, allozaur, angt, ServeurpersoCom
# ggml-org/llama-webui : allozaur

/.devops/*.Dockerfile @ngxson
/.github/actions/ @ggml-org/ci
Expand Down
59 changes: 58 additions & 1 deletion convert_hf_to_gguf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10893,7 +10893,64 @@ def set_gguf_parameters(self):
self.gguf_writer.add_moe_latent_size(latent_size)

def set_vocab(self):
super().set_vocab()
# The NemotronH config uses pattern characters (e.g. '-') that may not
# be supported by the installed transformers version. AutoTokenizer
# internally calls AutoConfig which triggers this parsing failure.
# Using trust_remote_code=True to load the model's own config class.
tokens: list[str] = []
toktypes: list[int] = []

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(self.dir_model, trust_remote_code=True)

# Pad vocab size (from Mamba2Model/GraniteHybridModel)
self.hparams["pad_vocab_size_multiple"] = 8 # Setting this here since GraniteHybridModel.set_vocab() isn't being invoked now.
# From Mamba2Model.set_vocab():
vocab_size = self.hparams["vocab_size"]
pad_vocab = self.hparams.get("pad_vocab_size_multiple", 16)
# ref: https://stackoverflow.com/a/17511341/22827863
vocab_size = -(vocab_size // -pad_vocab) * pad_vocab
self.hparams["vocab_size"] = vocab_size

assert max(tokenizer.vocab.values()) < vocab_size

Check failure on line 10915 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10915:20: unresolved-attribute: Attribute `vocab` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10915 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10915:20: unresolved-attribute: Attribute `vocab` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

tokpre = self.get_vocab_base_pre(tokenizer)

reverse_vocab = {id_: encoded_tok for encoded_tok, id_ in tokenizer.vocab.items()}

Check failure on line 10919 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10919:67: unresolved-attribute: Attribute `vocab` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10919 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10919:67: unresolved-attribute: Attribute `vocab` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default
added_vocab = tokenizer.get_added_vocab()

Check failure on line 10920 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10920:23: unresolved-attribute: Attribute `get_added_vocab` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10920 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10920:23: unresolved-attribute: Attribute `get_added_vocab` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

added_tokens_decoder = tokenizer.added_tokens_decoder

Check failure on line 10922 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10922:32: unresolved-attribute: Attribute `added_tokens_decoder` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10922 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10922:32: unresolved-attribute: Attribute `added_tokens_decoder` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

for i in range(vocab_size):
if i not in reverse_vocab:
tokens.append(f"[PAD{i}]")
toktypes.append(gguf.TokenType.UNUSED)
else:
token: str = reverse_vocab[i]
if token in added_vocab:
if not added_tokens_decoder[i].normalized:
previous_token = token
token = tokenizer.decode(tokenizer.encode(token, add_special_tokens=False))

Check failure on line 10933 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10933:50: unresolved-attribute: Attribute `encode` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10933 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10933:33: unresolved-attribute: Attribute `decode` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10933 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (invalid-assignment) 

convert_hf_to_gguf.py:10933:33: invalid-assignment: Object of type `Unknown | str | list[str]` is not assignable to `str`
  convert_hf_to_gguf.py:10933:25: Declared type `str`
  info: rule `invalid-assignment` is enabled by default

Check failure on line 10933 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10933:50: unresolved-attribute: Attribute `encode` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10933 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (unresolved-attribute) 

convert_hf_to_gguf.py:10933:33: unresolved-attribute: Attribute `decode` is not defined on `None` in union `Unknown | TokenizersBackend | None | SentencePieceBackend`
  info: rule `unresolved-attribute` is enabled by default

Check failure on line 10933 in convert_hf_to_gguf.py

View workflow job for this annotation

GitHub Actions / python type-check

ty (invalid-assignment) 

convert_hf_to_gguf.py:10933:33: invalid-assignment: Object of type `Unknown | str | list[str]` is not assignable to `str`
  convert_hf_to_gguf.py:10933:25: Declared type `str`
  info: rule `invalid-assignment` is enabled by default
if previous_token != token:
logger.info(f"{repr(previous_token)} is encoded and decoded back to {repr(token)} using AutoTokenizer")

if added_tokens_decoder[i].special or self.does_token_look_special(token):
toktypes.append(gguf.TokenType.CONTROL)
else:
token = token.replace(b"\xe2\x96\x81".decode("utf-8"), " ") # pre-normalize user-defined spaces
toktypes.append(gguf.TokenType.USER_DEFINED)
else:
toktypes.append(gguf.TokenType.NORMAL)
tokens.append(token)

# From TextModel.set_vocab_gpt2():
self.gguf_writer.add_tokenizer_model("gpt2")
self.gguf_writer.add_tokenizer_pre(tokpre)
self.gguf_writer.add_token_list(tokens)
self.gguf_writer.add_token_types(toktypes)

special_vocab = gguf.SpecialVocab(self.dir_model, load_merges=True)
special_vocab.add_to_gguf(self.gguf_writer)

# The tokenizer _does_ add a BOS token (via post_processor type
# TemplateProcessing) but does not set add_bos_token to true in the
Expand Down
16 changes: 8 additions & 8 deletions docs/ops.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ Legend:
| ARANGE | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| ARGMAX | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
| ARGSORT | ❌ | ✅ | ✅ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ✅ | ❌ | ❌ |
| CEIL | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | ❌ |
| CEIL | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | ✅ | 🟡 | ✅ | ❌ | ❌ |
| CLAMP | ❌ | ✅ | ✅ | ✅ | ✅ | 🟡 | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| CONCAT | ❌ | ✅ | ✅ | 🟡 | ✅ | 🟡 | ✅ | ✅ | ✅ | ❌ | ❌ |
| CONT | ❌ | 🟡 | ✅ | ✅ | 🟡 | 🟡 | 🟡 | ✅ | 🟡 | ❌ | ❌ |
| CONT | ❌ | 🟡 | ✅ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ❌ | ❌ |
| CONV_2D | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
| CONV_2D_DW | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
| CONV_3D | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| CONV_3D | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| CONV_TRANSPOSE_1D | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| CONV_TRANSPOSE_2D | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
| COS | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
Expand All @@ -46,7 +46,7 @@ Legend:
| EXPM1 | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ |
| FILL | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
| FLASH_ATTN_EXT | ❌ | 🟡 | ✅ | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | ❌ | ❌ |
| FLOOR | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| FLOOR | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| GATED_DELTA_NET | ❌ | ❌ | ✅ | ❌ | 🟡 | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ |
| GATED_LINEAR_ATTN | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
| GEGLU | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | 🟡 | ✅ | ❌ | ❌ |
Expand Down Expand Up @@ -84,10 +84,10 @@ Legend:
| REPEAT_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| RMS_NORM | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| RMS_NORM_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| ROLL | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| ROLL | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| ROPE | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| ROPE_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| ROUND | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| ROUND | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| RWKV_WKV6 | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| RWKV_WKV7 | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| SCALE | ❌ | 🟡 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
Expand Down Expand Up @@ -116,6 +116,6 @@ Legend:
| TIMESTEP_EMBEDDING | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
| TOP_K | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| TRI | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
| TRUNC | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| TRUNC | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
| UPSCALE | ❌ | 🟡 | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | ❌ | ❌ | ❌ |
| XIELU | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
| XIELU | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
Loading
Loading