cross entropy loss during training with xlora

[crossxxd]

I saw discussions about training in other issues, and I have run train and inference code successfully. Training code is mainly based on SFTTrainer and I think only next-token prediction loss is used. If I want to add cross entropy loss mentioned in the paper, what should I do?

[EricLBuehler]

To use cross-entropy loss, we configure the training loop to use CE loss.

[crossxxd]

I have rewritten the trainer from transformers lib and added the cross entropy of the xlora classifier's category output. There is no problem for now. Thanks for your reply!

[EricLBuehler]

@crossxxd, we do not train for the X-LoRA classifier's scalings output in the paper, although you could try that. We just train the model as normal, with the CE loss on the output of the model. This works because the gradients propagate up to the X-LoRA classifier's output, and because the output is a result of the X-LoRA classifier we are training the X-LoRA classifier.

[crossxxd]

It seems that I misunderstood the definition of loss in the paper. For now, I am using the loss on the output of the model combined with the loss on the scalings output of the xlora classifier for overall training. The total loss can converge and xlora model works fine.