Feature/qwen3

[zpitroda]

• Updated models from Qwen 2.5 to Qwen 3 equivalents
• Updated transformers and torch python packages

[imaffe]

I am not sure what is Second Me's model update policy. As a community user I definitely want to use Qwen3 considering its SOTA capabilities (but I haven't tested yet so maybe there will be some in & outs).

Huge thanks to your work!

On top of that, It would be nice to add Qwen 3 on top of existing Qwen 2.5 model support, like adding a new support model instead of replace the existing Qwen 2.5 model.

[zpitroda]

I am not sure what is Second Me's model update policy. As a community user I definitely want to use Qwen3 considering its SOTA capabilities (but I haven't tested yet so maybe there will be some in & outs).

Huge thanks to your work!

On top of that, It would be nice to add Qwen 3 on top of existing Qwen 2.5 model support, like adding a new support model instead of replace the existing Qwen 2.5 model.

I was wondering that as well, I'm testing right now to ensure qwen 3 doesn't break anything but yeah I don't know if the Second Me team currents actually wants to update . I can also keep the 2.5 models and add the option for 3 as well if that's preferable?

[zpitroda]

Everything is working except during inference it outputs the <think></think> blocks because llama.cpp hasn't updated the qwen3 template to support {"enable_thinking": false} kwargs. There is already a PR so hopefully it'll be updated asap

[kevinaimonster]

Hi, I've test it a bit, it fails when downloading models...

2025-05-06 14:30:21 [INFO] trainprocess_service.py:265 - Starting model download: Qwen3-1.7B
2025-05-06 14:30:21 [ERROR] trainprocess_service.py:285 - Download model failed: cannot access local variable 'base_dir' where it is not associated with a value
2025-05-06 14:30:21 [ERROR] trainprocess_service.py:1109 - Step model_download failed

[zpitroda]

Hi, I've test it a bit, it fails when downloading models...

2025-05-06 14:30:21 [INFO] trainprocess_service.py:265 - Starting model download: Qwen3-1.7B
2025-05-06 14:30:21 [ERROR] trainprocess_service.py:285 - Download model failed: cannot access local variable 'base_dir' where it is not associated with a value
2025-05-06 14:30:21 [ERROR] trainprocess_service.py:1109 - Step model_download failed

Sorry about that! The base_dir variable was accidentally indented into the "if" block above it but should be working now

[kevin-mindverse]

Hi, it works now. so once the conflict is resolved, it will be added to develop branch :)

[zpitroda]

@kevin-mindverse sounds good! I think I have a temporary solution to the extra block being outputted until the llama.pp pr is merged, I'll try to have that and the conflict fixes pushed tomorrow

[zpitroda]

Ok I actually quickly pushed what I think should do it, shouldn't break anything but haven't double check it's fixed

[zpitroda]

@yingapple I'm away from my computer this week and unable to test, but it should hopefull now only add no_think flags when using a qwen3 model