-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
我的minicpmo4_5_lora_sft.yaml文件配置为:
model
model_name_or_path: /home/MiniCPM-o/checkpoints/MiniCPM-o-4_5 # 可以是MiniCPM-V或者MiniCPM-o的本地模型
trust_remote_code: true
method
stage: sft # sft训练
do_train: true
finetuning_type: lora # lora微调
lora_target: q_proj,v_proj # lora层插入哪里
dataset
dataset: mllm_demo # 改成你上面data/data_info.json的文件下新增的键名
template: minicpm_o # 不要改
cutoff_len: 3072 # 包多模态在哪占用的模型token长度
max_samples: 1000 #最多用多少条数据
overwrite_cache: true
preprocessing_num_workers: 16
output
output_dir: saves/minicpmo4_5/lora/sft
logging_steps: 1
save_steps: 100 #多少步报错一次
plot_loss: true # 是否绘制损失函数
overwrite_output_dir: true #是否覆盖之前的保存
save_total_limit: 10
train
per_device_train_batch_size: 2 #训练batch_size
gradient_accumulation_steps: 1 #梯度累积次数
learning_rate: 1.0e-5 # 学习率
num_train_epochs: 20.0 #最多训练轮次
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
save_only_model: true
eval
do_eval: false
执行:llamafactory-cli train configs/minicpmo4_5_lora_sft.yaml
会报错:ValueError: Audio feature extractor was not found, please check and update your model file.