When `llama.cpp`'s support for this will be stable. Hopefully, there will be an official API for this after https://github.com/ggml-org/llama.cpp/pull/11292 is implemented.
When
llama.cpp's support for this will be stable.Hopefully, there will be an official API for this after ggml-org/llama.cpp#11292 is implemented.