Skip to content

请问MiniCPM-o 4.5现在支持LLaMA-Factory微调吗 #1073

@18128381510

Description

@18128381510

我的minicpmo4_5_lora_sft.yaml文件配置为:

model

model_name_or_path: /home/MiniCPM-o/checkpoints/MiniCPM-o-4_5 # 可以是MiniCPM-V或者MiniCPM-o的本地模型
trust_remote_code: true

method

stage: sft # sft训练
do_train: true
finetuning_type: lora # lora微调
lora_target: q_proj,v_proj # lora层插入哪里

dataset

dataset: mllm_demo # 改成你上面data/data_info.json的文件下新增的键名
template: minicpm_o # 不要改
cutoff_len: 3072 # 包多模态在哪占用的模型token长度
max_samples: 1000 #最多用多少条数据
overwrite_cache: true
preprocessing_num_workers: 16

output

output_dir: saves/minicpmo4_5/lora/sft
logging_steps: 1
save_steps: 100 #多少步报错一次
plot_loss: true # 是否绘制损失函数
overwrite_output_dir: true #是否覆盖之前的保存
save_total_limit: 10

train

per_device_train_batch_size: 2 #训练batch_size
gradient_accumulation_steps: 1 #梯度累积次数
learning_rate: 1.0e-5 # 学习率
num_train_epochs: 20.0 #最多训练轮次
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
save_only_model: true

eval

do_eval: false

执行:llamafactory-cli train configs/minicpmo4_5_lora_sft.yaml
会报错:ValueError: Audio feature extractor was not found, please check and update your model file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions