Unable to include negative samples

[fjay314159]

I tried several models and with different configuration, but when I include negative samples (i.e. ner array is empty), there is always this error:

RuntimeError: The input size 0, plus negative padding 0 and 0 resulted in a negative output size, which is invalid. Check dimension 1 of your input.

I also tried adding labels to the label property as suggested in another issue, but to no avail. What am I doing wrong?

[TimKoornstra]

Same problem here, I didn't have this problem when running version 0.2.21

[lrdsouza]

Same here! I tried finetuning GLiNER with a batch size of 4 and got this error. I increased the batch size to 8 and was able to finetune without problems. Although not ideal for my problem, it's the best I could do so far.

[Ingvarstep]

This is the limitation of span-level GLiNER models' data preprocessing for training. I will work on better handling negative samples in the near future. For now, you can try to fine-tune token-level models ("span_mode": "token_level") that support negative samples. An example of such a model is glner-multitask.