IQuest-Coder-V1-40B-Instruct-int4 Version !

[gvyfmb8cvt-star]

I used vllm-compressor to perform int4 quantization on this 40B model, which allows this model to be deployed using VLLM on a single 4090 machine. Feel free to try it out!

https://huggingface.co/xhxlb/IQuest-Coder-V1-40B-Instruct-int4

[zelong518]

lgtm