what's the problem? Anybody give me some idea

[foxking-c]

[rank0]: File "/home/zhjzhj/temp/dp/diffusion-pipe/models/flux2.py", line 250, in forward
[rank0]: txt = self.txt_in(txt)
[rank0]: ^^^^^^^^^^^^^^^^
[rank0]: File "/home/zhjzhj/miniconda3/envs/diffusion-pipe/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
[rank0]: return self._call_impl(*args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/home/zhjzhj/miniconda3/envs/diffusion-pipe/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
[rank0]: return forward_call(*args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/home/zhjzhj/temp/dp/diffusion-pipe/utils/../submodules/ComfyUI/comfy/ops.py", line 164, in forward
[rank0]: return self.forward_comfy_cast_weights(*args, **kwargs)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/home/zhjzhj/temp/dp/diffusion-pipe/utils/../submodules/ComfyUI/comfy/ops.py", line 157, in forward_comfy_cast_weights
[rank0]: x = torch.nn.functional.linear(input, weight, bias)
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: RuntimeError: mat1 and mat2 shapes cannot be multiplied (836x2560 and 12288x4096)
[rank0]:[W118 00:43:11.000620966 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

[neph1]

Sounds like incorrectly configured models (ie incompatible ones). Difficult to say without the config itself.