Hello!
I am a researcher with a strong interest in the fields of reinforcement learning and robotic control. I have been following your work on TD-MPC closely. I believe your work is highly valuable, as it provides new ideas and methods for solving decision-making and planning problems in complex tasks, and it has a positive impact on the development of the entire field.
I found your open-source code on GitHub, which has greatly facilitated my in-depth learning and research. However, I noticed that the current open-source code seems to be mainly designed and optimized for tasks in the Deepmind control suite. In your paper, there are detailed experimental data regarding Metaworld. Recently, I tried to apply your method to tasks such as Hammer and window open in the Metaworld environment, hoping to verify the performance of TD-MPC in more complex and challenging tasks.
During the reproduction process, I followed the parameter settings provided in your appendix and directly applied the relevant parameters to the Hammer task in Metaworld. Unfortunately, after multiple experiments, the average success rate across five random seeds was only 10%. This result is significantly lower than what I observed in the Deepmind control suite tasks, which left me somewhat puzzled.
I understand that different simulation environments and tasks have significant differences in state space, action space, and task objectives, which may be one of the main reasons for the poor reproduction results. Therefore, I would like to ask you the following questions:
Have you ever tried to apply TD-MPC to other tasks in the Metaworld environment? If so, could you share some relevant experimental experiences and insights?
Are there any specific parameter tuning suggestions for the Metaworld environment, or do we need to make some targeted modifications to the existing code to better adapt to the characteristics of Metaworld tasks?
Could you provide some code snippets or parameter configuration examples for experiments in Metaworld? This would be of great help for me to further explore the potential of TD-MPC in the Metaworld environment.
I understand that you are likely very busy, but I would be extremely grateful if you could take some time to reply to my questions. I believe that by further validating and improving TD-MPC in the Metaworld environment, we can better understand its applicability and limitations in complex tasks, which will provide more valuable references for future research and applications.
Thank you again for your contributions to the open-source community. I look forward to your reply!
Contact Information: 51265901036@stu.ecnu.edu.cn
Reproduction Environment:
Simulation Environment: Metaworld
Task Names: Hammer, window open & close
Parameters Used: Parameters provided in the appendix
Hardware Configuration: GPU RTX 3090
Software Environment: Ubuntu 20.04, PyTorch
Steps to Reproduce the Issue:
Clone your TD-MPC code repository to my local machine.
Modify the relevant parameters in the code according to the settings provided in the appendix.
Run the Hammer task in the Metaworld environment.
Observe the experimental results and find that the success rate is only 10%.
Hello!
I am a researcher with a strong interest in the fields of reinforcement learning and robotic control. I have been following your work on TD-MPC closely. I believe your work is highly valuable, as it provides new ideas and methods for solving decision-making and planning problems in complex tasks, and it has a positive impact on the development of the entire field.
I found your open-source code on GitHub, which has greatly facilitated my in-depth learning and research. However, I noticed that the current open-source code seems to be mainly designed and optimized for tasks in the Deepmind control suite. In your paper, there are detailed experimental data regarding Metaworld. Recently, I tried to apply your method to tasks such as Hammer and window open in the Metaworld environment, hoping to verify the performance of TD-MPC in more complex and challenging tasks.
During the reproduction process, I followed the parameter settings provided in your appendix and directly applied the relevant parameters to the Hammer task in Metaworld. Unfortunately, after multiple experiments, the average success rate across five random seeds was only 10%. This result is significantly lower than what I observed in the Deepmind control suite tasks, which left me somewhat puzzled.
I understand that different simulation environments and tasks have significant differences in state space, action space, and task objectives, which may be one of the main reasons for the poor reproduction results. Therefore, I would like to ask you the following questions:
Have you ever tried to apply TD-MPC to other tasks in the Metaworld environment? If so, could you share some relevant experimental experiences and insights?
Are there any specific parameter tuning suggestions for the Metaworld environment, or do we need to make some targeted modifications to the existing code to better adapt to the characteristics of Metaworld tasks?
Could you provide some code snippets or parameter configuration examples for experiments in Metaworld? This would be of great help for me to further explore the potential of TD-MPC in the Metaworld environment.
I understand that you are likely very busy, but I would be extremely grateful if you could take some time to reply to my questions. I believe that by further validating and improving TD-MPC in the Metaworld environment, we can better understand its applicability and limitations in complex tasks, which will provide more valuable references for future research and applications.
Thank you again for your contributions to the open-source community. I look forward to your reply!
Contact Information: 51265901036@stu.ecnu.edu.cn
Reproduction Environment:
Simulation Environment: Metaworld
Task Names: Hammer, window open & close
Parameters Used: Parameters provided in the appendix
Hardware Configuration: GPU RTX 3090
Software Environment: Ubuntu 20.04, PyTorch
Steps to Reproduce the Issue:
Clone your TD-MPC code repository to my local machine.
Modify the relevant parameters in the code according to the settings provided in the appendix.
Run the Hammer task in the Metaworld environment.
Observe the experimental results and find that the success rate is only 10%.