Add RK3576 NPU support in mainline Linux and Mesa

[alchark]

RK3576 uses a similar (maybe the same) neural processor as RK3588, which has basic support under mainline Linux (via the rocket driver) and Mesa (teflon layer).

These will need porting to RK3576 to enable NPU on Flipper.

Actually supported features may be limited at the moment, so both the kernel part and the Mesa frontend would likely need more work to enable running interesting models on the NPU.

[tomkel]

You should get in touch with Tomeu for this one: https://gitlab.freedesktop.org/tomeu

[th3cavalry]

I have a photonicat2 which uses the rk3576. I've been tinkering with some stuff on it. I might be able to help with some of the NPU stuff.

[tomeuv]

I have a photonicat2 which uses the rk3576. I've been tinkering with some stuff on it. I might be able to help with some of the NPU stuff.

Feel free to send me any questions to tomeu@tomeuvizoso.net and I will try to find time to answer them.

[Crashkeyz]

“I’d like to contribute to RK3576 NPU bring‑up.

I’ve reviewed the RK3588 rocket driver and Mesa teflon layer, and I’m starting to compare the RK3576 NPU registers and DT requirements.
I can take on the initial device‑tree work and early probing for the NPU.
Let me know if anyone is already working on this or if there’s a preferred direction.”

[Crashkeyz]

crashkeyz@gmail.com

[Crashkeyz]

npu: npu@fdf00000 {
compatible = "rockchip,rk3576-npu";
reg = <0x0 0xfdf00000 0x0 0x100000>;
interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cru CLK_NPU_CORE>, <&cru CLK_NPU_BUS>;
clock-names = "core", "bus";
resets = <&cru SRST_NPU_CORE>, <&cru SRST_NPU_BUS>;
reset-names = "core", "bus";
power-domains = <&power RK3576_PD_NPU>;
iommus = <&npu_mmu>;
status = "disabled";
};arm64: dts: rockchip: add initial RK3576 NPU node (WIP)rk3576-npu-bringup-wip

I know some of the values my not be correct but I am eager to try

[th3cavalry]

Regarding the RK3576 NPU support, we've just shared the NPU infrastructure patches (Power Domains, QoS, GRF) that we stabilized for the PhotoniCAT 2 project in flipperdevices/flipper-linux-kernel#9.

Our rk3576.dtsi already includes the PD_NPUTOP hierarchy and all seven NPU QoS nodes required to enable the RKNPU2 stack without vendor BSP hacks. This should provide the necessary "wiring" to start the Mesa/Kernel driver work mentioned here.

Reference: flipperdevices/flipper-linux-kernel#9

[lordbuffcloud]

I think the useful split here is to keep the first target very small.

1. Kernel/DT plumbing: flipper-linux-kernel#9 may cover the power domains, QoS, GRF, and other wiring needed before Rocket/Mesa can do anything useful. I would first verify that an enabled NPU node boots cleanly, brings up clocks/resets/power domains, and attaches to the IOMMU without faults.
2. Rocket/Mesa support: Mesa's Teflon docs currently list Rocket as tested on RK3588/QuartzPro64, not RK3576, so I would treat RK3576 as a new bring-up target even if the block is similar.
3. First userspace test: use the existing Teflon MobileNetV1 UINT8 classification test before trying heavier models. That should separate device bring-up from model/op coverage.

Small success ladder:

• NPU node enabled in rk3576.dtsi with bindings reviewed
• boot log shows no power-domain, clock, reset, or IOMMU errors
• Rocket sees the device
• libteflon.so loads the Rocket backend
• MobileNetV1 UINT8 classification runs
• expand model/op coverage after that

That keeps three failure classes separate: DT bring-up, Rocket RK3576 support, and Teflon model/op support.

Refs:

• [Contribution] RK3576 Mainline Support: TRNG, Thermal OTP, and NPU Infrastructure flipper-linux-kernel#9
• https://docs.mesa3d.org/teflon.html

[ZlodeiBaal]

NPU support is a little bit tricky because it does not lead directly to LLM support. Probably, you need to create an additional task, "support target LLM", which will be heavily correlated with this NPU support task, and will define a lot of subtasks here.

Here is what you really need to do to run LLMs:

The documented Mesa/Teflon path is a TensorFlow Lite delegate, with the public test coverage focused on small INT8 vision models such as MobileNet/SSDLite. Running an LLM/VLM workload through the same fully open path would likely require several additional layers of work:

• RK3576-specific NPU bring-up and validation in the open driver stack.
• Much broader operator coverage than the currently documented CNN-style models require.
• A clear model/runtime path for LLMs: for example, LiteRT / LiteRT-LM / TFLite-compatible export, or another open runtime integration. Right now, it's only TFLite inference and not LiteRT
• Quantization and tensor layout/packing work suitable for transformer blocks.
• Attention/KV-cache handling and efficient fallback/scheduling between NPU, CPU, and possibly GPU.

Rockchip’s own RKLLM stack is useful as a reference point because it shows that LLM deployment on RK3576-class hardware is a real target, and it lists RK3576 plus models such as LLaMA, Qwen, Phi, Gemma, MiniCPM, etc.:
https://github.com/airockchip/rknn-llm
However, RKLLM is a different software path from the fully open Mesa/Teflon approach. So I would not assume that “RKLLM supports a model” automatically means that the same model is close to working through mainline Linux + Mesa/Teflon.
They will not be transferred directly to your code, and basically, it needs to be rewritten.

And an important point. ~two years ago, I tested which LLM can actually run on 3576. And not all networks that worked on 3588 worked on 3576.

Everything here is at a pretty high level, and each task requires a lot of detail.

---

I understand that it may not align with the project's current ideology. But to have LLMs fast, I recommend using the Armbian approach, which has just two branches. One with fully opened code and one with vendor-precompiled libraries. If you will go this way - instead of rewriting everything, you could just use the existing stack.

[tomeuv]

NPU support is a little bit tricky because it does not lead directly to LLM support.

Rockchip’s own RKLLM stack is useful as a reference point because it shows that LLM deployment on RK3576-class hardware is a real target, and it lists RK3576 plus models such as LLaMA, Qwen, Phi, Gemma, MiniCPM, etc.: https://github.com/airockchip/rknn-llm However, RKLLM is a different software path from the fully open Mesa/Teflon approach. So I would not assume that “RKLLM supports a model” automatically means that the same model is close to working through mainline Linux + Mesa/Teflon. They will not be transferred directly to your code, and basically, it needs to be rewritten.

I'm currently working on transformer support on Executorch for EthosU, btw. This will be a Gallium frontend that will be able to use all NPU gallium backends, including rocket. So if you get the vision models running with Teflon on the rk3576, then by that moment we may have most pieces ready for accelerating LLM models as well.

[n2dgtlhmcde]

Following the roadmap laid out above — we've prepared a patch series that covers step 1 (DT plumbing) and step 2 (Rocket compatible string), ready for hardware validation.

What the patches cover

0001 — accel/rocket: add rockchip,rk3576-rknn-core compatible string
0002 — dt-bindings: document RK3576 rknn-core
0003 — arm64: dts: rockchip: full NPU subsystem in rk3576.dtsi
         ├ PD_NPU → PD_NPUTOP → PD_NPU0 / PD_NPU1 hierarchy
         ├ npu_grf syscon @ 0x27720000
         ├ 7x NIC-400 QoS nodes (0x27714000–0x2771a000)
         ├ IOMMU @ 0x27710000
         ├ OPP table (300–950 MHz, vendor-sourced voltages)
         └ npu_core0 @ 0x27700000, npu_core1 @ 0x27708000 (disabled)

The DT infra in patch 3 builds directly on the power domain hierarchy, QoS nodes, and GRF work shared by the PhotoniCAT 2 project in #9. Register addresses and clock names are cross-checked against friendlyarm/kernel-rockchip (nanopi6-v6.1.y) and the RK3576 TRM.

The series is untested on hardware — we don't have an RK3576 board yet. The next step on our end is a NanoPi M5 to run through your success ladder: clean boot log → Rocket probes → libteflon.so loads → MobileNetV1 UINT8 classification.

Patches

Available as a [PATCH RFC 0/3] series — link to fork/branch once pushed.

If anyone with an RK3576 board (ROCK 4D, Sige5, NanoPi M5) wants to test before we have hardware, the patches are ready to apply against mainline 6.18+.

Open items before v1

• rocket_core.c clock probe table — RK3576 clock set differs from RK3588 (follow-on patch)
• IOMMU compatible string needs hardware confirmation
• Board enablement DTS (ROCK 4D, NanoPi M5) as a separate series after boot is confirmed

Happy to reformat as git send-email to linux-rockchip + dri-devel once the DT bring-up is confirmed on hardware.

RFCPATCHRK3576.zip