Problem about training on single task close_jar

[dctwan15]

Thank you very much for your contribution to this work.

I tried to retrain the model on a single task close_jar. After 240k iterations, I found that the training log position and rotation accuracy were close to 1, but the validation log rotation accuracy was still relatively low (only about 0.3).I stopped training and ran inference, and the accuracy was 0. In many cases, the gripper posture had a large deviation. I ran the inference script on the training set again and found that the situation was similar to that during inference.

My first question is: Is this normal? Should I continue training until the rotation accuracy on the validation set is also relatively high?

My second question is: Why is the training prediction accuracy in the training log close to 1, but there are still many failures when running the inference script on the training set?

[ubless607]

Did you check MohitShridhar/RLBench#1?

[nickgkan]

Hi,
Yes, @ubless607 is right, there is a bug in that task. You need to implement the fix in the pull request above, as we also say here https://github.com/nickgkan/3d_diffuser_actor/blob/master/docs/GETTING_STARTED_RLBENCH.md#step-1-install-rlbench-with-the-peract-setup.

[dctwan15]

@nickgkan Thank you for your attention and answering my question in your busy schedule.

Sorry, I did overlook the bug in the success condition of close_jar before. Now I have corrected this according to the instructions of MohitShridhar/RLBench#1. But the same problem still exists. I will describe the specific situation in which I encountered failure in inference in detail, hoping to get your help.

Case 1: Path not found

Case 2: The posture of the gripper will have a large deviation in the first step, and it will directly turn upside down instead of facing down to grab the lid.
It is precisely because of this that I think the problem I encountered may not be caused by the bug in the success condition in MohitShridhar/RLBench#1.

Note: I have set single_task_gripper_loc_bounds=1 during inference

[dctwan15]

Attached is my training script.
I specified tasks as close_jar only and adjusted batch_size and batch_size_val to 4 to suit my server.
And no code modification was made except to correct the success condition of RLBench close_jar task.

main_dir=Actor_18Peract_100Demo_multitask

dataset=/datasets/peract/Peract_packaged/train
valset=/datasets/peract/Peract_packaged/val

lr=1e-4
dense_interpolation=1
interpolation_length=2
num_history=3
diffusion_timesteps=100
B=4
C=120
ngpus=7
quaternion_format=xyzw

CUDA_LAUNCH_BLOCKING=1 torchrun --nproc_per_node $ngpus --master_port $RANDOM \
    main_trajectory.py \
    --tasks close_jar \
    --dataset $dataset \
    --valset $valset \
    --instructions instructions/peract/instructions.pkl \
    --gripper_loc_bounds tasks/18_peract_tasks_location_bounds.json \
    --num_workers 1 \
    --train_iters 600000 \
    --embedding_dim $C \
    --use_instruction 1 \
    --rotation_parametrization 6D \
    --diffusion_timesteps $diffusion_timesteps \
    --val_freq 4000 \
    --dense_interpolation $dense_interpolation \
    --interpolation_length $interpolation_length \
    --exp_log_dir $main_dir \
    --batch_size $B \
    --batch_size_val 4 \
    --cache_size 600 \
    --cache_size_val 0 \
    --keypose_only 1 \
    --variations {0..199} \
    --lr $lr\
    --num_history $num_history \
    --cameras left_shoulder right_shoulder wrist front\
    --max_episodes_per_task -1 \
    --quaternion_format $quaternion_format \
    --run_log_dir single_task_close_jar

[nickgkan]

Thanks for sharing the training script, from what I understand, your curves look good, right?

Did you change the quaternion_format to be the same in both training and eval scripts?

Actually, can you share your eval script as well, if the above suggestions doesn't fix your problem?

[dctwan15]

My training process curves do look good.

And I have confirmed that this parameter is the same quaternion_format=xyzw when training and evaluating.

Attached is my evaluation script, hope to get your help.

exp=3d_diffuser_actor

# close_jar insert_onto_square_peg light_bulb_in meat_off_grill open_drawer place_shape_in_shape_sorter place_wine_at_rack_location push_buttons put_groceries_in_cupboard put_item_in_drawer put_money_in_safe reach_and_drag slide_block_to_color_target stack_blocks stack_cups sweep_to_dustpan_of_size turn_tap place_cups
tasks=(
    close_jar
)
data_dir=/datasets/peract/raw/test/
num_episodes=100
gripper_loc_bounds_file=tasks/18_peract_tasks_location_bounds.json
use_instruction=1
max_tries=2
verbose=1
interpolation_length=2
single_task_gripper_loc_bounds=1
embedding_dim=120
cameras="left_shoulder,right_shoulder,wrist,front"
fps_subsampling_factor=5
lang_enhanced=0
relative_action=0
seed=0
# checkpoint=train_logs/diffuser_actor_peract.pth
checkpoint=train_logs/Actor_18Peract_100Demo_multitask/single_task_close_jar/best.pth
quaternion_format=wxyz  # IMPORTANT: change this to be the same as the training script IF you're not using our checkpoint

num_ckpts=${#tasks[@]}
for ((i=0; i<$num_ckpts; i++)); do
    CUDA_LAUNCH_BLOCKING=1 python online_evaluation_rlbench/evaluate_policy.py \
    --tasks ${tasks[$i]} \
    --checkpoint $checkpoint \
    --diffusion_timesteps 100 \
    --fps_subsampling_factor $fps_subsampling_factor \
    --lang_enhanced $lang_enhanced \
    --relative_action $relative_action \
    --num_history 3 \
    --test_model 3d_diffuser_actor \
    --cameras $cameras \
    --verbose $verbose \
    --action_dim 8 \
    --collision_checking 0 \
    --predict_trajectory 1 \
    --embedding_dim $embedding_dim \
    --rotation_parametrization "6D" \
    --single_task_gripper_loc_bounds $single_task_gripper_loc_bounds \
    --data_dir $data_dir \
    --num_episodes $num_episodes \
    --output_file eval_logs/$exp/seed$seed/${tasks[$i]}.json  \
    --use_instruction $use_instruction \
    --instructions instructions/peract/instructions.pkl \
    --variations {0..60} \
    --max_tries $max_tries \
    --max_steps 25 \
    --seed $seed \
    --gripper_loc_bounds_file $gripper_loc_bounds_file \
    --gripper_loc_bounds_buffer 0.04 \
    --quaternion_format $quaternion_format \
    --interpolation_length $interpolation_length \
    --dense_interpolation 1
done

[nickgkan]

Hi,

I know that you said the quaternion_format is the same in training and inference but in the evaluation script you pasted the quaternion_format is not the same as in the training script.

It seems you're evaluating on the test set, do you want to try evaluating on the training set. You mentioned that you did that but it doesn't look like that in the script. That will erase all our doubts about low validation accuracy.

Lastly, can you evaluate the pre-trained checkpoint using your code?

[dctwan15]

OMG, I'm so sorry, I'm sorry for my carelessness and assumption.

You are right, I did use different quaternion_format parameters in the training script and the evaluation script.
Specifically, it was xyzw in the training script and wxyz in the evaluation script.

I corrected this and re-evaluated, and the result was correct, which solved my previous problem.

Thank you very much for your patient analysis and help.