perf(cuda): reduce redundant per-token synchronizations

.github/workflows/fork-sync.yml

@@ -0,0 +1,22 @@
+name: Fork Sync
+
+on:
+  schedule:
+    - cron: '30 7 * * *'
+  workflow_dispatch:
+    inputs:
+      skip_rebase:
+        description: 'Only sync main, skip rebase of ht'
+        type: boolean
+        default: false
+
+jobs:
+  sync:
+    uses: heiervang-technologies/.github/.github/workflows/fork-sync-reusable.yml@main
+    with:
+      upstream_repo: 'ggml-org/llama.cpp'
+      upstream_branch: 'master'
+      fork_branch: 'ht'
+      skip_rebase: ${{ inputs.skip_rebase || false }}
+    secrets:
+      gh_pat: ${{ secrets.HAI_GH_PAT }}

.github/workflows/python-lint.yml

@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - master
+      - ht
     paths: [
       '.github/workflows/python-lint.yml',
       '**/*.py'

CONTRIBUTING.md

@@ -1,30 +1,51 @@
-# Contributors
+# HT Fork Management
 
-The project differentiates between 3 levels of contributors:
+This is a [Heiervang Technologies](https://github.com/heiervang-technologies) fork. For full details on fork workflow, sync procedures, and contribution process, see the [Fork Management Guide](https://github.com/orgs/heiervang-technologies/discussions/3).
 
-- Contributors: people who have contributed before (no special privileges)
-- Collaborators (Triage): people with significant contributions, who may be responsible for some parts of the code, and are expected to maintain and review contributions for the code they own
-- Maintainers: responsible for reviewing and merging PRs, after approval from the code owners
+## Branch Conventions
+
+| Branch | Purpose |
+|--------|---------|
+| `master` | Clean fast-forward mirror of upstream — never commit directly |
+| `ht` | HT-specific changes — **default branch**, all PRs target this |
+| `feat/*` | Feature branches created from `ht`, squash-merged back via PR |
+
+## Sync Workflow
+
+```bash
+# Update master from upstream
+git checkout master
+git fetch upstream
+git merge --ff-only upstream/master
+git push origin master
+
+# Rebase ht onto updated master
+git checkout ht
+git rebase master
+git push --force-with-lease origin ht
+```
 
-# AI Usage Policy
+## Commit Standards
 
-> [!IMPORTANT]
-> This project does **not** accept pull requests that are fully or predominantly AI-generated. AI tools may be utilized solely in an assistive capacity.
->
-> Repeated violations of this policy may result in your account being permanently banned from contributing to the project.
->
-> Detailed information regarding permissible and restricted uses of AI can be found in the [AGENTS.md](AGENTS.md) file.
+- Use [conventional commits](https://www.conventionalcommits.org/) (e.g., `feat:`, `fix:`, `chore:`)
+- Maintain clean, linear history — one commit per logical change
+- Squash fix-up commits before merging
 
-Code that is initially generated by AI and subsequently edited will still be considered AI-generated. AI assistance is permissible only when the majority of the code is authored by a human contributor, with AI employed exclusively for corrections or to expand on verbose modifications that the contributor has already conceptualized (e.g., generating repeated lines with minor variations).
+For all questions and inquiries about this fork, use [HT Discussions](https://github.com/orgs/heiervang-technologies/discussions).
 
-If AI is used to generate any portion of the code, contributors must adhere to the following requirements:
+---
+
+# Contributors
+
+This fork differentiates between 3 levels of contributors:
+
+- Contributors: people who have contributed before (no special privileges)
+- Collaborators (Triage): people with significant contributions, who may be responsible for some parts of the code, and are expected to maintain and review contributions for the code they own
+- Maintainers: responsible for reviewing and merging PRs, after approval from the code owners
 
-1. Explicitly disclose the manner in which AI was employed.
-2. Perform a comprehensive manual review prior to submitting the pull request.
-3. Be prepared to explain every line of code they submitted when asked about it by a maintainer.
-4. It is strictly prohibited to use AI to write your posts for you (bug reports, feature requests, pull request descriptions, Github discussions, responding to humans, ...).
+# Agentic Contributions
 
-For more info, please refer to the [AGENTS.md](AGENTS.md) file.
+We welcome contributions from AI agents and assistants. Unlike upstream, we recognize that modern development workflows increasingly involve AI tools, and we embrace that reality. Whether you're a human developer or working with an AI assistant, we're happy to review your pull request. Standard code quality expectations apply—contributions should be well-tested, documented, and maintainable—but we evaluate the code, not how it was written.
 
 # Pull requests (for contributors & collaborators)
 

README.md

@@ -1,6 +1,35 @@
-# llama.cpp
+![ht-llama.cpp](media/ht-llama-banner.png)
 
-![llama](https://user-images.githubusercontent.com/1991296/230134379-7181e485-c521-4d23-a0d6-f7b3b61ba524.png)
+# ht-llama.cpp
+
+_Heiervang Technologies fork of [llama.cpp](https://github.com/ggml-org/llama.cpp)_
+
+[HT Discussions](https://github.com/orgs/heiervang-technologies/discussions) | [Fork Management Guide](https://github.com/orgs/heiervang-technologies/discussions/3) | [Upstream: llama.cpp](https://github.com/ggml-org/llama.cpp)
+
+---
+
+## HT Fork Changes
+
+This is the [Heiervang Technologies](https://github.com/heiervang-technologies) fork of [llama.cpp](https://github.com/ggml-org/llama.cpp). The `ht` branch contains the following changes on top of upstream `master`:
+
+| Change | Description | Contributed back? |
+|--------|-------------|-------------------|
+| Agentic contributions policy | We welcome AI-assisted and AI-generated contributions—see [CONTRIBUTING.md](CONTRIBUTING.md) | N/A |
+
+Unlike upstream, we accept contributions from AI agents and assistants. We judge code by its quality, not its authorship.
+
+### Branch Strategy
+
+| Branch | Purpose |
+|--------|---------|
+| `master` | Clean fast-forward mirror of upstream `master` — never commit directly |
+| `ht` | HT-specific changes on top of `master` — **default branch** |
+
+Feature branches are created from `ht` and squash-merged back via PR.
+
+For questions or inquiries, use the [HT Discussions](https://github.com/orgs/heiervang-technologies/discussions) page. For details on fork workflow and sync procedures, see the [Fork Management Guide](https://github.com/orgs/heiervang-technologies/discussions/3).
+
+---
 
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 [![Release](https://img.shields.io/github/v/release/ggml-org/llama.cpp)](https://github.com/ggml-org/llama.cpp/releases)

common/arg.cpp

@@ -3058,6 +3058,14 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
             params.use_jinja = value;
         }
     ).set_examples({LLAMA_EXAMPLE_SERVER, LLAMA_EXAMPLE_COMPLETION, LLAMA_EXAMPLE_CLI, LLAMA_EXAMPLE_MTMD}).set_env("LLAMA_ARG_JINJA"));
+    add_opt(common_arg(
+        {"--remap-developer-role"},
+        "remap the OpenAI \"developer\" role to \"system\" before applying chat templates "
+        "(needed for models whose templates reject unknown roles, e.g. Qwen3.5)",
+        [](common_params & params) {
+            params.remap_developer_role = true;
+        }
+    ).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_REMAP_DEVELOPER_ROLE"));
     add_opt(common_arg(
         {"--reasoning-format"}, "FORMAT",
         "controls whether thought tags are allowed and/or extracted from the response, and in which format they're returned; one of:\n"

common/common.h

@@ -588,6 +588,7 @@ struct common_params {
     std::string api_prefix    = "";                                                                         // NOLINT
     std::string chat_template = "";                                                                         // NOLINT
     bool use_jinja = true;                                                                                  // NOLINT
+    bool remap_developer_role = false; // remap "developer" role to "system" for template compatibility
     bool enable_chat_template = true;
     bool force_pure_content_parser = false;
     common_reasoning_format reasoning_format = COMMON_REASONING_FORMAT_DEEPSEEK;

common/preset.cpp

@@ -4,6 +4,8 @@
 #include "log.h"
 #include "download.h"
 
+#include "gguf.h"
+
 #include <fstream>
 #include <sstream>
 #include <filesystem>
@@ -444,6 +446,141 @@ common_presets common_preset_context::load_from_models_dir(const std::string & m
     return out;
 }
 
+// helper: read GGUF metadata to determine if file is a LoRA adapter
+// returns true if it is an adapter, and fills out architecture string
+static bool gguf_is_lora_adapter(const std::string & path, std::string & architecture) {
+    struct gguf_init_params params = {
+        /*.no_alloc = */ true,
+        /*.ctx      = */ nullptr,
+    };
+    struct gguf_context * ctx = gguf_init_from_file(path.c_str(), params);
+    if (!ctx) {
+        return false;
+    }
+
+    bool is_adapter = false;
+
+    int64_t type_key = gguf_find_key(ctx, "general.type");
+    if (type_key >= 0) {
+        const char * type_val = gguf_get_val_str(ctx, type_key);
+        if (type_val && std::string(type_val) == "adapter") {
+            is_adapter = true;
+
+            int64_t arch_key = gguf_find_key(ctx, "general.architecture");
+            if (arch_key >= 0) {
+                const char * arch_val = gguf_get_val_str(ctx, arch_key);
+                if (arch_val) {
+                    architecture = arch_val;
+                }
+            }
+        }
+    }
+
+    gguf_free(ctx);
+    return is_adapter;
+}
+
+common_models_dir_result common_preset_context::load_from_models_dir_with_lora(const std::string & models_dir) const {
+    if (!std::filesystem::exists(models_dir) || !std::filesystem::is_directory(models_dir)) {
+        throw std::runtime_error(string_format("error: '%s' does not exist or is not a directory\n", models_dir.c_str()));
+    }
+
+    std::vector<local_model> models;
+    std::vector<common_lora_adapter_info> adapters;
+
+    auto scan_subdir = [&models, &adapters](const std::string & subdir_path, const std::string & name) {
+        auto files = fs_list(subdir_path, false);
+        common_file_info model_file;
+        common_file_info first_shard_file;
+        common_file_info mmproj_file;
+        std::vector<common_file_info> other_gguf_files;
+
+        for (const auto & file : files) {
+            if (string_ends_with(file.name, ".gguf")) {
+                if (file.name.find("mmproj") != std::string::npos) {
+                    mmproj_file = file;
+                } else if (file.name.find("-00001-of-") != std::string::npos) {
+                    first_shard_file = file;
+                } else {
+                    other_gguf_files.push_back(file);
+                }
+            }
+        }
+
+        // check each non-shard, non-mmproj gguf file for adapter type
+        for (const auto & file : other_gguf_files) {
+            std::string arch;
+            if (gguf_is_lora_adapter(file.path, arch)) {
+                std::string adapter_name = file.name;
+                string_replace_all(adapter_name, ".gguf", "");
+                adapters.push_back({
+                    /* name         */ adapter_name,
+                    /* path         */ file.path,
+                    /* architecture */ arch,
+                });
+                LOG_INF("%s: discovered LoRA adapter '%s' (arch: %s) in %s\n",
+                    __func__, adapter_name.c_str(), arch.c_str(), subdir_path.c_str());
+            } else {
+                // treat as model file (last one wins, matching original behavior)
+                model_file = file;
+            }
+        }
+
+        local_model model{
+            /* name        */ name,
+            /* path        */ first_shard_file.path.empty() ? model_file.path : first_shard_file.path,
+            /* path_mmproj */ mmproj_file.path
+        };
+        if (!model.path.empty()) {
+            models.push_back(model);
+        }
+    };
+
+    auto files = fs_list(models_dir, true);
+    for (const auto & file : files) {
+        if (file.is_dir) {
+            scan_subdir(file.path, file.name);
+        } else if (string_ends_with(file.name, ".gguf")) {
+            std::string file_name = file.name;
+            string_replace_all(file_name, ".gguf", "");
+
+            // check if this is a LoRA adapter
+            std::string arch;
+            if (gguf_is_lora_adapter(file.path, arch)) {
+                adapters.push_back({
+                    /* name         */ file_name,
+                    /* path         */ file.path,
+                    /* architecture */ arch,
+                });
+                LOG_INF("%s: discovered LoRA adapter '%s' (arch: %s)\n",
+                    __func__, file_name.c_str(), arch.c_str());
+            } else {
+                local_model model{
+                    /* name        */ file_name,
+                    /* path        */ file.path,
+                    /* path_mmproj */ ""
+                };
+                models.push_back(model);
+            }
+        }
+    }
+
+    // convert local models to presets
+    common_models_dir_result result;
+    for (const auto & model : models) {
+        common_preset preset;
+        preset.name = model.name;
+        preset.set_option(*this, "LLAMA_ARG_MODEL", model.path);
+        if (!model.path_mmproj.empty()) {
+            preset.set_option(*this, "LLAMA_ARG_MMPROJ", model.path_mmproj);
+        }
+        result.presets[preset.name] = preset;
+    }
+    result.adapters = std::move(adapters);
+
+    return result;
+}
+
 common_preset common_preset_context::load_from_args(int argc, char ** argv) const {
     common_preset preset;
     preset.name = COMMON_PRESET_DEFAULT_NAME;