Hybrid: Leader LLM (GPU) & subagent LLM (NPU)
Coding agents often fill the leader model’s context with raw file reads, search results and tool outputs. FLM’s low TTFT and efficient NPU execution could make it an ideal parallel coding subagent while a larger leader model runs on the GPU.
The NPU model could inspect code, compress tool results into focused evidence and handle small, well-scoped edits before returning only the relevant information to the GPU leader.
A particularly useful FLM capability would be multiple independent context slots sharing one loaded model:
- an ephemeral scout context, reset after every task for consistently low TTFT;
- a small persistent maintainer context for project-local follow-ups and incremental edits.
Both would use the same weights, but keep separate session/KV state. This could turn the NPU into a continuously available coding assistant without duplicating model memory or polluting the leader’s context - which should be ideally without a full compression as long as possible.
For an initial implementation, a smaller compute-efficient coding model would already be valuable. As a more ambitious experiment, native binary or ternary support for Bonsai 27B could be extremely interesting: strong coding capability with unusually compact weights. Native 1-bit execution may be a stretch goal, but it could become a unique showcase for FLM and Ryzen AI NPUs. See also #619
Would multi-context model sessions and this type of GPU-leader/NPU-subagent workload fit FLM’s roadmap?
Hybrid: Leader LLM (GPU) & subagent LLM (NPU)
Coding agents often fill the leader model’s context with raw file reads, search results and tool outputs. FLM’s low TTFT and efficient NPU execution could make it an ideal parallel coding subagent while a larger leader model runs on the GPU.
The NPU model could inspect code, compress tool results into focused evidence and handle small, well-scoped edits before returning only the relevant information to the GPU leader.
A particularly useful FLM capability would be multiple independent context slots sharing one loaded model:
Both would use the same weights, but keep separate session/KV state. This could turn the NPU into a continuously available coding assistant without duplicating model memory or polluting the leader’s context - which should be ideally without a full compression as long as possible.
For an initial implementation, a smaller compute-efficient coding model would already be valuable. As a more ambitious experiment, native binary or ternary support for Bonsai 27B could be extremely interesting: strong coding capability with unusually compact weights. Native 1-bit execution may be a stretch goal, but it could become a unique showcase for FLM and Ryzen AI NPUs. See also #619
Would multi-context model sessions and this type of GPU-leader/NPU-subagent workload fit FLM’s roadmap?