Does not run on CPU only, needs CUDA for compression

[wthie]

./rabbitllm/utils/compression.py", line 192, in compress_layer_state_dict
    v_quant, quant_state = bnb.functional.quantize_nf4(v.cuda(), blocksize=64)
                                                       ^^^^^^^^

[wthie]

After digging through the code I switched off quantisation at the cuda level and everything comes live. The project makes a good impression for all the structure and abstraction and is a really good educational piece, the quantisation/compression though in my opinion would require its own abstraction, deserve its own directory . Is anyone interested in such a rework or does this go the usual way that at some point there is a hard fork, a new name, a new owner?