Training error

[luckcodingdog]

Thanks for your excellent work.
I have tried to reproduce the work, but when i run the command of
sh scripts/train/train-starvector-8b-im2svg.sh.
The environment is same as what you said and flash-attent is version of 2.7.3.

Attempted to call variable.set_data(tensor), but variable and tensor have incompatible tensor type.
File "/data/zhuxt/star-vector/starvector/model/llm/starcoder2.py", line 28, in init
model.resize_token_embeddings(len(self.tokenizer))
File "/data/zhuxt/star-vector/starvector/model/models/starvector_v2.py", line 16, in _get_svg_transformer
return StarCoderModel(config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zhuxt/star-vector/starvector/model/models/starvector_base.py", line 29, in init
self.svg_transformer = self._get_svg_transformer(config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zhuxt/star-vector/starvector/model/models/starvector_v2.py", line 10, in init
super().init(config, **kwargs)
File "/data/zhuxt/star-vector/starvector/model/starvector_arch.py", line 142, in init
self.model = StarVectorStarCoder2(config=config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zhuxt/star-vector/starvector/model/builder.py", line 28, in model_builder
model = StarVectorForCausalLM.from_pretrained(model_name, **args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zhuxt/star-vector/starvector/train/train.py", line 97, in main
model = model_builder(config)
^^^^^^^^^^^^^^^^^^^^^
File "/data/zhuxt/star-vector/starvector/train/train.py", line 274, in
main(config=get_config())
RuntimeError: Attempted to call variable.set_data(tensor), but variable and tensor have incompatible tensor type.

[joanrod]

Hi! Contact me at joanrg.ai@gmail.com