ADD and AMP Training on Hand Animated Reference Motions

[lnayak10]

Hi @xbpeng , I have been training AMP and ADD on some of my hand animated reference motions ( dynamically infeasible) for Go2 and Go2w. It seems like AMP and ADD returns are zero consistently throughout the iterations, my guess is the discriminator is trained on one cycle of reference motion hence it is over fitting, to find any state transition generated by policy to be fake. But, when I tried runnning policies from ADD, it is somewhat following the motions, but is way worse than Deepmimic trained policy. Do you have any suggestions or thoughts on this?

This is log from training AMP on one cycle of trot motion which is hand animated.

[xbpeng]

The AMP and ADD environments have no environment reward, There is only a discriminator reward from the agent. The returns are logged for the environment reward, therefore they will always be 0 for AMP and ADD.

Do you have videos of what the reference motion and policy looks like?

[lnayak10]

This the policy tracking performance for trot walking. I'm a little bit confused in terms of the implementation of AMP, the update_reward function does nothing in the amp_env.py file, so where is the discriminator based reward is applied as written in AMP paper ?

[xbpeng]

The discriminator rewards are calculated in the agent class:

MimicKit/mimickit/learning/amp_agent.py

Line 106 in 28f1f21

disc_r = self._calc_disc_rewards(norm_disc_obs)

Your reference motion doesn't move the legs very much. It could be that the policy still gets a reasonably high reward even without moving the legs. You probably need to tune the hyperparameters for the discriminator, like the gradient penalty. Or try modifying the reference motion to lift its legs more.